Blog Live data Countries

Priority issues

Tools

Resources

About

AI Risk & Accountability

AI has risks and all actors must be accountable.

AI, Data & Privacy

Data and privacy are primary policy issues for AI.

Generative AI

Managing the risks and benefits of generative AI.

Future of Work

How AI can and will affect workers and working environments

AI Index

The OECD AI will be a synthetic measurement framework on Trustworthy Artificial Intelligence (AI)

AI Incidents

To manage risks, governments must track and understand AI incidents and hazards.

Data Governance

Expertise on data governance to promote its safe and faire use in AI

Responsible AI

The responsible development, use and governance of human-centred AI systems

Innovation & Commercialisation

How to drive cooperation on AI and transfer research results into products

AI Compute & Climate

AI computing capacities and their environmental impact.

AI & Health

AI can help health systems overcome their most urgent challenges.

AI Futures

AI’s potential futures.

WIPS

Programme on Work, Innovation, Productivity and Skills in AI.

Catalogue Tools & Metrics

Explore tools & metrics to build and deploy AI systems that are trustworthy.

AI Incidents Monitor

Gain valuable insights on global AI incidents.

OECD AI Principles

The first IGO standard to promote innovative and trustworthy AI

Policy areas

Browse OECD work related to AI across policy areas.

Publications

Find OECD publications related to AI.

Videos

Watch videos about AI policy the issues that matter most.

Context

AI is already a crucial part of most people’s daily routines.

What we do

Countries and stakeholder groups join forces to shape trustworthy AI.

Network of Experts

Experts from around the world advise the OECD and contribute to its work.

Partners

OECD.AI works closely with many partners.

Home
Tools & metrics
Tools

Catalogue of Tools & Metrics for Trustworthy AI

These tools and metrics are designed to help AI actors develop and use trustworthy AI systems and applications that respect human rights and are fair, transparent, explainable, robust, secure and safe.

Overview Tools Metrics About the catalogue

Contribute to the catalogue

Show tools Show use cases

Type

Approach

Technical

Educational

Procedural

Tool type

Objective

Usage rights

Copyleft/Share alike

Open source/Permissive

Creative commons

Free of charge

Restricted access

Non-commercial

Research purposes only

Fee-based

One-time license

Other

Origin

Stakeholder group

Academia

Business

Civil society

Government

International organisation

Technical community

Trade union/workers

Other

Country of origin

Organisation

Scope

Lifecycle stage(s)

Operate & monitor

Deploy

Verify & validate

Build & interpret model

Collect & process data

Plan & design

Target group(s)

Academia/educators/students

Civil society/specific social groups

Private sector

Professionals

Public sector

Technical community

Trade Union/worker representatives

Other

Target user(s)

System integrators

Policy makers

All employees

Business leader

Data scientist

Developer

Government

HR manager

IT specialist

Management

Project manager

Researcher

System operators

Other

Target sector(s)

Impacted stakeholders

Consumers

Employees

Management

Regulators

Specific policy communities

Other

Purpose(s)

Content generation

Event/anomaly detection

Forecasting/prediction

Goal-driven optimisation

Interaction support/chatbots

Personalisation/recommenders

Reasoning with knowledge structures/planning

Recognition/object detection

Tool readiness

Implemented in multiple projects

In development

Project stage

Published document

SUBMIT A TOOL

If you have a tool that you think should be featured in the Catalogue of AI Tools & Metrics, we would love to hear from you!

SUBMIT

List of tools (5)

Holistic AI Open Source Library

TechnicalProceduralUnited KingdomUploaded on Mar 27, 2023

The Holistic AI library is an open-source tool to assess and improve the trustworthiness of AI systems. The library offers a set of techniques to easily measure and mitigate Bias across a variety of tasks.

Objective(s)

Related lifecycle stage(s)

Operate & monitor Deploy Verify & validate Build & interpret model Collect & process data Plan & design

Holistic AI risk mitigation roadmaps

TechnicalProceduralUploaded on Mar 27, 2023

A set of guides to help enterprises mitigate some of the most common AI risks, presenting step-by-step solutions to protect against technical risks.

Objective(s)

Fairness Performance Privacy & Data Governance & Traceability Human Agency & Control Robustness Safety Transparency

Related lifecycle stage(s)

Operate & monitor Deploy Verify & validate Build & interpret model Collect & process data Plan & design

Holistic AI Audits

TechnicalProceduralUploaded on Mar 27, 2023<1 day

A bespoke AI risk audit solution tailor-made to identify your enterprise project’s AI risks, comprising deep technical, quantitative analysis.

Objective(s)

Fairness Performance Privacy & Data Governance & Traceability Human Agency & Control Robustness Transparency

Related lifecycle stage(s)

Operate & monitor Deploy Verify & validate Build & interpret model Collect & process data Plan & design

Holistic AI Bias Audits

TechnicalProceduralUploaded on Mar 27, 2023<1 day

A fully independent and impartial audit to ensure compliance with the bias audit requirements for New York City Local Law 144 and upcoming regulations.

Objective(s)

Fairness Human Agency & Control Transparency

Related lifecycle stage(s)

Operate & monitor Deploy Verify & validate Build & interpret model Collect & process data Plan & design

Holistic AI Governance, Risk and Compliance Platform

TechnicalProceduralUploaded on Mar 27, 2023<1 day

A scalable and systematic solution empowering enterprise to adopt and scale AI with confidence and enhance business performance.

Objective(s)

Fairness Performance Privacy & Data Governance & Traceability Human Agency & Control Robustness Transparency

Related lifecycle stage(s)

Operate & monitor Deploy Verify & validate Build & interpret model Collect & process data Plan & design

Disclaimer: The tools and metrics featured herein are solely those of the originating authors and are not vetted or endorsed by the OECD or its member countries. The Organisation cannot be held responsible for possible issues resulting from the posting of links to third parties' tools and metrics on this catalogue. More on the methodology can be found at https://oecd.ai/catalogue/faq.

Receive the OECD's artificial intelligence newsletter!

Home
Terms & conditions
Privacy policy