TrustLLM is a framework and open-source toolkit for evaluating the trustworthiness of large language models. It was presented at ICML 2024 by an international team led by researchers in the United States. It proposes principles for trustworthy models and a benchmark covering truthfulness, safety, fairness, robustness, privacy and machine ethics. The benchmark draws on more than 30 datasets. A public leaderboard ranks 16 leading models. The code is released under the MIT licence.

The MLCommons Jailbreak Benchmark v1.0 is a benchmark for evaluating how well large language models withstand jailbreak attacks, which are prompts designed to bypass safety safeguards. It was published by MLCommons, an industry and academic consortium. The benchmark applies known jailbreak techniques to prompts across eleven hazard categories. Results are compared with a safety baseline to show how much attacks weaken a system’s safety. Systems are graded on how often they produce content that violates safety standards.

DelusionEval is a dataset for evaluating how AI chatbots behave when conversations reinforce users' delusional beliefs. It was developed by Stanford University. It contains 725 excerpts from real, anonymised conversations, each labelled with one of 18 chatbot behaviours. These include harmful behaviours, such as endorsing delusions or facilitating self-harm, and protective ones, such as discouraging violence. Researchers use it to audit chatbot safety and develop detection methods. Access is restricted to non-commercial research.

Sponsio is an open-source runtime safety tool for AI agents. It checks every agent action against deterministic rules, called agent contracts, before the action is executed. Contracts are based on formal methods and can allow, block, escalate or redirect actions. Every decision is logged in an audit trail. Users can apply ready-made contract bundles or draft rules in plain English. Sponsio works with major agent frameworks in Python and TypeScript, under the Apache 2.0 licence.

KSAFE-MM is a Korean-language benchmark for evaluating safety risks in multimodal large language models. It was developed by K-intelligence at KT in South Korea. The benchmark contains 14,135 query-image pairs across 11 risk categories, including hate, violence, privacy and weaponisation. One subset tests Korean culture-specific risks and applies jailbreak strategies such as role-play. Researchers use it to check whether models respond safely. It is available for research under a CC BY-NC 4.0 licence.

Doberman is an open-source security layer for AI coding agents such as Claude Code and Codex. It checks every action an agent takes before it runs. Routine actions pass, sensitive actions need human approval and dangerous actions are blocked. Any uncertainty results in the action being denied. Protections can tighten automatically, but weakening them requires human approval and is logged. Doberman is written in Python and released under the Apache 2.0 licence.

IndoBias is a benchmark for evaluating social bias in large language models in Indonesian and three local languages. It was developed by researchers at the Mohamed bin Zayed University of Artificial Intelligence and Universitas Indonesia. One track tests whether models prefer stereotypical sentences over neutral ones. A second track tests whether models describe local groups and institutions more positively or negatively. The results show strong stereotypical bias in current models, especially regarding ideology and religion in local languages.

The Safety Evaluation Benchmark for Vision LLMs is an open dataset for testing the safety and robustness of models that answer questions about images. It was developed by researchers led by the University of California, Santa Cruz. One part tests how models handle unusual images, such as sketches, and misleading questions. A second part tests resistance to attacks, including altered images and jailbreak prompts. The dataset contains about 2 000 samples and is released under the Apache 2.0 licence.

PyDP is an open-source Python library for differential privacy, created by OpenMined. It wraps Google's Differential Privacy library, giving Python users access to tested algorithms. PyDP produces summary statistics, such as means, sums and medians, from sensitive datasets while adding calibrated noise to protect individuals. Users set a privacy parameter that controls the trade-off between privacy and accuracy. PyDP runs on Linux, macOS and Windows. It is released under the Apache 2.0 licence.

FLARE-AI is an open-source AI flaw and incident reporting platform that lets anyone document a flaw in any AI system and route a single, standardised report to multiple developers and coordinators at once. It enables any AI actor to document vulnerabilities, biases, or incidents and route a single, standardised (JSON-LD) report to multiple developers, coordinators, and registries in the ecosystem.

The REFRAIME Legal Toolkit provides legal practitioners, public authorities, and civil society organisations with a structured resource for identifying and addressing the impact of AI systems on fundamental rights under the EU Charter. Developed by a consortium including the Center for the Study of Democracy, the European Center for Not-for-Profit Law, and the University of Malta, and co-funded by the European Union, the toolkit combines knowledge articles, sixteen real-world case studies grounded in actual case law (including ACLU v. Clearview AI, SCHUFA before the CJEU, and the Dutch Childcare Benefits case), an interactive glossary, a curated directory of EU, Council of Europe, UN, and OECD instruments, and a 43-point checklist for monitoring compliance with Fundamental Rights Impact Assessment obligations under Article 27 of the EU AI Act.

The AIM Framework (Awareness, Identification, Mitigation) presents a stepwise approach for the implementation of risk management strategies. The framework is intended for AI developers working in private, academic, or public sectors. It features a checklist with indicative scenarios for awareness-raising and training purposes.

SafeIMG is a safety-oriented benchmark for evaluating synthetic-image detection and visual-evidence verification in high-risk public- and personal-safety scenarios. It provides scenario-specific synthetic images generated for assessing how misleading visuals could impact safety and accountability. Researchers and developers can use it to verify and compare image authenticity methods, and to improve robustness of safeguards for trustworthy visual content.

ModelBench is a benchmarking tool for running safety-focused evaluations on AI models and producing detailed reports on performance against the benchmark suite. It is intended for developers and researchers who need to verify and compare safety/robustness behavior before deployment and during ongoing evaluation. The outputs support auditing and model validation for safety-related trustworthiness objectives.

OpenART is an open-source framework designed to evaluate the security of AI agents in dynamic, long-horizon, and stateful environments. It stress-tests agent runtimes against multi-step state poisoning, privilege escalation, and tool-use vulnerabilities across more than 10,000 benchmark scenarios.

IndicSafeEval is a multilingual benchmark framework for evaluating the safety and robustness of large language models (LLMs) against persuasion-based jailbreak attacks in Indian languages. It combines safety-critical content categories with multiple human-like persuasion strategies and evaluates model responses across several languages. The framework is designed to identify safety and alignment failures in non-English settings and can be used to assess and compare model behavior, support model validation, and monitor safety performance.

The AI Slop Side Effect Database documents indirect harms to legitimate users, creators, researchers, and organisations caused by the proliferation of low-quality AI-generated content and by countermeasures introduced to control it. It classifies cases across gatekeeping failures, content contamination, discriminatory bias, institutional invisibility, and service self-contamination, with evidence levels, affected parties, sources, and analytical commentary.

Ster is an open-source framework for intervening in large language model behavior at the level of internal activations to reduce harmful outputs and hallucinations. It helps users improve safety and robustness by enabling targeted controls that affect what the model produces, supporting transparency by making activation-level intervention mechanisms available for inspection and explanation.

Human Approval Gate is a free, platform-neutral educational kit that helps leaders, educators, operators, and small teams define what a qualified person must check before AI-assisted work can affect a real decision or action. It includes a practical guide, printable worksheet, facilitator notes, and ten synthetic test cases. Its CLEAR test holds the consequence, names the reviewer and evidence, preserves accept, revise, reject, and escalate outcomes, and records the decision and recovery path.

GovAI is the official Australian Government AI platform. Announced in July 2025, GovAI offers Australian Public Service (APS) staff secure access to advanced AI capabilities through government-controlled infrastructure. A key aspect of the platform is the Interactive Learning Environment that provides hands-on-training using multiple AI models, enabling staff to build practical AI skills in a secure, risk-free environment.