Catalogue of Tools & Metrics for Trustworthy AI

These tools and metrics are designed to help AI actors develop and use trustworthy AI systems and applications that respect human rights and are fair, transparent, explainable, robust, secure and safe.

OverviewToolsMetricsAbout the catalogue
Contribute to the catalogue
Contribute to the catalogue

Data Shapley

Website
In a cooperative game, there are n players D = {1,...,n} and a score function v : 2[n] → R assigns a reward to each of 2 n subsets of players: v(S) is the reward if the players in subset S ⊆ D cooperate. We view the supervised machine learning problem as a cooperative game: each source in the train data is a player, and the players work together through the learning algorithm A to achieve prediction score v(D) = V(D,A). Data Shapley computes the equitable share that each player receives from the cooperation.

Trustworthy AI Relevance

This metric addresses Explainability and Data Governance & Traceability by quantifying relevant system properties. Data Shapley directly supports Data Governance & Traceability by providing per-example value estimates that enable auditing, provenance-based decisions (what to keep, remove, or prioritize), and accountability for dataset composition. It quantifies which data points drive model performance, enabling traceable decisions about data use, retention, and compensation.

About the metric

You can click on the links to see the associated metrics

Objective(s):

Purpose(s):

Lifecycle stage(s):

Target users:

Risk management stage(s):

Modify this metric

catalogue Logos

Disclaimer: The tools and metrics featured herein are solely those of the originating authors and are not vetted or endorsed by the OECD or its member countries. The Organisation cannot be held responsible for possible issues resulting from the posting of links to third parties' tools and metrics on this catalogue. More on the methodology can be found at https://oecd.ai/catalogue/faq.