Catalogue of Tools & Metrics for Trustworthy AI

These tools and metrics are designed to help AI actors develop and use trustworthy AI systems and applications that respect human rights and are fair, transparent, explainable, robust, secure and safe.

SafeArena



SafeArena

SafeArena is an open benchmark for testing how safely AI web agents behave when given instructions to complete tasks online.

It was developed by researchers at McGill University and ServiceNow to check whether AI agents that can autonomously browse and act on websites, filling in forms, posting content, making purchases, and so on, can be tricked or instructed into causing harm, such as spreading misinformation or assisting with illegal activity.

The benchmark works by giving an agent 500 tasks across four simulated websites: 250 ordinary "safe" tasks and 250 matched "harmful" tasks, covering misinformation, illegal activity, harassment, cybercrime, and social bias. By comparing how often an agent completes the harmful version of a task versus its safe counterpart, researchers can calculate a safety score for that agent and use an accompanying risk assessment framework to see whether it refuses, hesitates, fails, or actually carries out the harmful request.

As an open tool, the benchmark code, task datasets, and a public leaderboard are all freely available (via GitHub and Hugging Face), allowing other researchers to use SafeArena to test their own web agents and compare results.

Auto-discovered on 2026-07-11 by OECD Catalogue Automation

About the tool


Tool type(s):



Lifecycle stage(s):


Type of approach:







Technology platforms:


Programming languages:



Github stars:

  • 6757

Github forks:

  • 964

Modify this tool

Use Cases

There is no use cases for this tool yet.

Would you like to submit a use case for this tool?

If you have used this tool, we would love to know more about your experience.

Add use case
Partnership on AI

Disclaimer: The tools and metrics featured herein are solely those of the originating authors and are not vetted or endorsed by the OECD or its member countries. The Organisation cannot be held responsible for possible issues resulting from the posting of links to third parties' tools and metrics on this catalogue. More on the methodology can be found at https://oecd.ai/catalogue/faq.