Catalogue of Tools & Metrics for Trustworthy AI

These tools and metrics are designed to help AI actors develop and use trustworthy AI systems and applications that respect human rights and are fair, transparent, explainable, robust, secure and safe.

OverviewToolsMetricsAbout the catalogue
Contribute to the catalogue
Contribute to the catalogue

Shapley Variable Importance Cloud (ShapleyVIC)

Website
Following the VIC framework, our proposed ShapleyVIC extends the widely used Shapley-based variable importance measures beyond final models for a comprehensive assessment and has important practical implications.

Trustworthy AI Relevance

This metric addresses Explainability and Robustness by quantifying relevant system properties. Explainability: ShapleyVIC produces per‑feature Shapley value distributions (a 'cloud') rather than a single point estimate, so it improves the comprehensibility and fidelity of explanations by revealing the range, central tendency, and uncertainty of feature importance. That helps stakeholders understand how features drive predictions and when explanations are reliable. Robustness: By aggregating Shapley values over multiple re-trainings, subsamples, or model perturbations, ShapleyVIC quantifies the stability of importance scores under data/model variation.

References

1. Nikulin, M.S., 2001. Hellinger distance. In Encyclopedia of Mathematics. EMS Press. 2. Hellinger, E., 1909. Neue Begründung der Theorie quadratischer Formen von unendlichvielen Veränderlichen. Journal für die reine und angewandte Mathematik, 136, pp.210–271. 3. Jeffreys, H., 1946. An invariant form for the prior probability in estimation problems. Proceedings of the Royal Society of London. Series A. Mathematical and Physical Sciences, 186(1007), pp.453–461. 4. Liese, F. and Miescke, K.-J., 2008. Statistical Decision Theory: Estimation, Testing, and Selection. Springer. 5. Pardo, L., 2006. Statistical Inference Based on Divergence Measures. Chapman and Hall/CRC.

About the metric

You can click on the links to see the associated metrics

Objective(s):

Purpose(s):

Lifecycle stage(s):

Target users:

Risk management stage(s):

Modify this metric

Partnership on AI

Disclaimer: The tools and metrics featured herein are solely those of the originating authors and are not vetted or endorsed by the OECD or its member countries. The Organisation cannot be held responsible for possible issues resulting from the posting of links to third parties' tools and metrics on this catalogue. More on the methodology can be found at https://oecd.ai/catalogue/faq.