Catalogue of Tools & Metrics for Trustworthy AI

These tools and metrics are designed to help AI actors develop and use trustworthy AI systems and applications that respect human rights and are fair, transparent, explainable, robust, secure and safe.

TrojAI Detect & Defend



TrojAI Detect & Defend

TrojAI is an AI Security platform that protects AI/ML applications and models at both build and runtime. It is an enterprise grade, customizable, and scalable solution, meeting the most stringent needs of large enterprises in complex environments. 

 

(1) TrojAI Detect - Secure AI model behavior at build time

 

TrojAI Detect automatically pentests AI models, assessing risks and deficiencies in model behavior before AI applications are deployed. In an evolving AI threat landscape, ensuring AI models behave as expected becomes a CISO imperative. Leveraging a robust attack library and hundreds of manipulations, TrojAI Detect helps AI Security teams thoroughly pentest AI applications against threats and risks such as prompt injections, jailbreaks, data leakages, biased, toxic and harmful content, hallucinations, reasoning inconsistencies, and more.

 

Key Benefits:
- Hundreds of readily available tests, easily configurable and customizable
- Coverage for GenAI, NLP, and tabular models
- Support for leading standards (OWASP, MITRE, NIST)

 

(2) TrojAI Defend - Monitor and secure AI applications

 

TrojAI Defend provides real-time monitoring and protection against active AI threats like prompt injection, jailbreaks, model DoS, data leakages, and harmful, toxic, and inappropriate content. As enterprises continue to build and deploy AI applications, AI attack surfaces expand, leading to a proliferation of threats on AI applications. TrojAI Defend helps CISOs of the world's largest organizations identify and mitigate AI risks and threats targeting AI models and applications in production, ensuring safe, secure and compliant AI.

 

Key Benefits:
- Extensible AI/ML rules engine, easily customizable
- Flexible controls to monitor, alert, block, redact, and log
- Supports leading standards (OWASP, MITRE, NIST)

Use Cases

There is no use cases for this tool yet.

Would you like to submit a use case for this tool?

If you have used this tool, we would love to know more about your experience.

Add use case
catalogue Logos

Disclaimer: The tools and metrics featured herein are solely those of the originating authors and are not vetted or endorsed by the OECD or its member countries. The Organisation cannot be held responsible for possible issues resulting from the posting of links to third parties' tools and metrics on this catalogue. More on the methodology can be found at https://oecd.ai/catalogue/faq.