These tools and metrics are designed to help AI actors develop and use trustworthy AI systems and applications that respect human rights and are fair, transparent, explainable, robust, secure and safe.
Scope
SUBMIT A METRIC
If you have a tool that you think should be featured in the Catalogue of AI Tools & Metrics, we would love to hear from you!
Submit This page includes technical metrics and methodologies for measuring and evaluating AI trustworthiness and AI risks. These metrics are often represented through mathematical formulas that assess the technical requirements for achieving trustworthy AI in a particular context. They can help to ensure that a system is fair, accurate, explainable, transparent, robust, safe, or secure.
Normalized Mutual Information (NMI)
Normalized Mutual Information is a metric calculated between two clusterings and is a normalization of the Mutual Information (MI) score to scale the results between 0 (no mutual information) and 1 (perfect correlation).
NMI can support Data Governan...
Objectives:
Data Shapley
In a cooperative game, there are n players D = {1,...,n} and a score function v : 2[n] → R assigns a reward to each of 2 n subsets of players: v(S) is the reward if the players in subset S ⊆ D cooperate. We view the supervised machine learning problem as a coo...
Objectives:
Beta Shapley
Beta Shapley is a unified data valuation framework that naturally arises from Data Shapley by relaxing the efficiency axiom. The Beta(α, β)-Shapley value considers the pair of hyperparameters (α, β) which decides the weight distribution on [n]. Beta(1,1)-Shapl...
Objectives:
