TrustLLM is a framework and open-source toolkit for evaluating the trustworthiness of large language models. It was presented at ICML 2024 by an international team led by researchers in the United States. It proposes principles for trustworthy models and a benchmark covering truthfulness, safety, fairness, robustness, privacy and machine ethics. The benchmark draws on more than 30 datasets. A public leaderboard ranks 16 leading models. The code is released under the MIT licence.

The MLCommons Jailbreak Benchmark v1.0 is a benchmark for evaluating how well large language models withstand jailbreak attacks, which are prompts designed to bypass safety safeguards. It was published by MLCommons, an industry and academic consortium. The benchmark applies known jailbreak techniques to prompts across eleven hazard categories. Results are compared with a safety baseline to show how much attacks weaken a system’s safety. Systems are graded on how often they produce content that violates safety standards.

xsoft comply is a Python software development kit that records the operations of AI systems to support record-keeping under Korea’s AI Basic Act. It records model calls, tool use, decisions and human reviews, and lets developers attach structured evidence for specific articles of the Act. Raw content is not sent by default, and common personal data is masked. Records are linked so changes can be detected. It is an early-stage product and does not certify compliance.

Agent Barn is an open-source control plane for running and governing AI agents on an organisation's own infrastructure. It was built by AAI Labs in Lithuania. Agents connect to team chat tools such as Slack and Microsoft Teams, and act on systems such as GitHub and Jira. Access roles control who manages each agent. Audit logs record conversations and tool calls, and costs are attributed to each agent. It is released under the Apache 2.0 licence.

The CREST Accreditation Standards set requirements for organisations that provide cyber security services, verified through independent assessment. They are published by CREST, an international not-for-profit body. Three components address AI. Domain 7 requires all accredited organisations to govern their use of AI, including validating AI-supported outputs. Annex B sets requirements for using AI in penetration testing, with human validation of findings. A third standard covers security testing of generative AI and large language model-enabled systems.

Mandare is an open-source accountability system for fleets of AI agents. It gives each agent a signed identity and a signed mandate setting its spending caps, permissions and approval requirements. A gateway enforces these limits outside the agent, with an offline kill switch. Every action, including refusals, is recorded in a tamper-evident ledger that an independent witness can check. Raw data never leaves the user's machine. It is released under the AGPL-3.0 and Apache 2.0 licences.

AiEnsured is a commercial testing platform for AI products, designed to support the deployment of responsible AI. It is developed in India. The platform brings together fairness and bias evaluation, explainability, adversarial robustness testing and privacy and GDPR compliance checks. It also generates test cases automatically, including rare and extreme inputs. Further features cover experiment management, model comparison and performance testing. After deployment, it monitors models for concept drift, which can reduce accuracy over time.

The VeritasChain Protocol (VCP) is an open specification for cryptographically verifiable audit trails in algorithmic and AI-driven trading. It is maintained by the VeritasChain Standards Organization. VCP lets regulators and auditors verify mathematically that records of trading decisions, orders, risk controls and AI model use are complete and unaltered. A privacy module allows personal data to be erased while keeping the audit trail intact. VCP is designed to support MiFID II, EU AI Act and GDPR compliance.

FAIAS (Financial AI Independent Assurance Standard) is a baseline for reviewing generative, agentic and predictive AI systems in financial institutions. It was developed by Wonder Attah. FAIAS sets out 109 review questions across nine domains, such as data, security and fairness, each with a safe, non-production test procedure. Systems are reviewed in proportion to their risk. The author describes it as emerging work, not a certification scheme. It is released under a CC BY 4.0 licence.

DelusionEval is a dataset for evaluating how AI chatbots behave when conversations reinforce users' delusional beliefs. It was developed by Stanford University. It contains 725 excerpts from real, anonymised conversations, each labelled with one of 18 chatbot behaviours. These include harmful behaviours, such as endorsing delusions or facilitating self-harm, and protective ones, such as discouraging violence. Researchers use it to audit chatbot safety and develop detection methods. Access is restricted to non-commercial research.

Noisegate is an open-source demonstration of a gateway that lets AI agents query sensitive data without exposing individual records. Its privacy protections do not depend on the AI being trustworthy. A trusted layer checks every query against a policy, adds calibrated noise to answers and limits each user's privacy budget. Working attack demonstrations show its protections in action. The developer describes it as a demonstration, not a production product. It is released under the Apache 2.0 licence.

Diffprivlib is an open-source Python library for differential privacy, developed by IBM Research in Dublin. It adds calibrated random noise to data analysis and machine learning, so that no individual can be identified. The library provides privacy mechanisms, machine learning models with built-in privacy, data analysis tools and a privacy budget tracker. Its models work like those of scikit-learn, making them easy to adopt. It is intended for research and education, and was archived in September 2026.

pyPANTERA is an open-source Python package for obfuscating text using differential privacy. It was developed at the University of Padua. The package replaces words in a text with substitutes, so the original cannot be directly recovered, while keeping the text useful for tasks such as search. It implements seven published mechanisms in one framework, allowing users to reproduce and compare them. It won the Best Resource Paper Award at CIKM 2024 and is released under the GPL-3.0 licence.

AI Compliancy is an online EU AI Act risk assessment, obligation and report tool for UK small and medium-sized businesses that use AI built into everyday software. Users check whether the Act can reach them, record the software they use, assess what they use each AI feature for, work through the obligations that follow, and produce a dated report.

Inner Warden is an open-source security agent for Linux and macOS servers. It detects attacks such as brute-force attempts and privilege escalation, and alerts operators in real time. It can use AI models to recommend responses, but AI remains advisory unless operators allow automatic action. Inner Warden can also monitor autonomous AI agents and block risky commands. All actions are reversible and recorded in an audit trail. It runs locally and is released under the MIT licence.

Sponsio is an open-source runtime safety tool for AI agents. It checks every agent action against deterministic rules, called agent contracts, before the action is executed. Contracts are based on formal methods and can allow, block, escalate or redirect actions. Every decision is logged in an audit trail. Users can apply ready-made contract bundles or draft rules in plain English. Sponsio works with major agent frameworks in Python and TypeScript, under the Apache 2.0 licence.

KSAFE-MM is a Korean-language benchmark for evaluating safety risks in multimodal large language models. It was developed by K-intelligence at KT in South Korea. The benchmark contains 14,135 query-image pairs across 11 risk categories, including hate, violence, privacy and weaponisation. One subset tests Korean culture-specific risks and applies jailbreak strategies such as role-play. Researchers use it to check whether models respond safely. It is available for research under a CC BY-NC 4.0 licence.

Doberman is an open-source security layer for AI coding agents such as Claude Code and Codex. It checks every action an agent takes before it runs. Routine actions pass, sensitive actions need human approval and dangerous actions are blocked. Any uncertainty results in the action being denied. Protections can tighten automatically, but weakening them requires human approval and is logged. Doberman is written in Python and released under the Apache 2.0 licence.

IndoBias is a benchmark for evaluating social bias in large language models in Indonesian and three local languages. It was developed by researchers at the Mohamed bin Zayed University of Artificial Intelligence and Universitas Indonesia. One track tests whether models prefer stereotypical sentences over neutral ones. A second track tests whether models describe local groups and institutions more positively or negatively. The results show strong stereotypical bias in current models, especially regarding ideology and religion in local languages.

F2Bench is a benchmark for evaluating fairness in large language models that also considers factual accuracy. It was developed by researchers at Inner Mongolia University and Xiamen University. The benchmark contains 2 568 test instances across ten demographic categories, including intersectional combinations. One task tests whether models accept biased conclusions after gradual stereotypical suggestions in a conversation. A second task tests whether models avoid stereotypes while still reflecting real statistics accurately. The code and dataset are publicly available.