Catalogue of Tools & Metrics for Trustworthy AI

These tools and metrics are designed to help AI actors develop and use trustworthy AI systems and applications that respect human rights and are fair, transparent, explainable, robust, secure and safe.

OverviewToolsMetricsAbout the catalogue
Contribute to the catalogue
Contribute to the catalogue

Mean rank

4 citations of this metric
Github
Website

Mean rank is the average ranking position of the items predicted by the model among all the possible items.

Trustworthy AI Relevance

This metric addresses Robustness and Fairness by quantifying relevant system properties. Robustness: Mean rank quantifies how well a model consistently places relevant items near the top across queries or conditions. Changes or increases in mean rank under perturbed inputs, OOD data, or adversarial/noisy conditions are a direct signal of decreased reliability.

About the metric

You can click on the links to see the associated metrics

Objective(s):

Purpose(s):

Lifecycle stage(s):

Target users:

Risk management stage(s):

Github stars:

  • 7100

Github forks:

  • 720

Modify this metric

Partnership on AI

Disclaimer: The tools and metrics featured herein are solely those of the originating authors and are not vetted or endorsed by the OECD or its member countries. The Organisation cannot be held responsible for possible issues resulting from the posting of links to third parties' tools and metrics on this catalogue. More on the methodology can be found at https://oecd.ai/catalogue/faq.