These tools and metrics are designed to help AI actors develop and use trustworthy AI systems and applications that respect human rights and are fair, transparent, explainable, robust, secure and safe.
Out-of-distribution (OOD) generalization
Robustness Metrics provides lightweight modules in order to evaluate the robustness of classification models. OOD generalization is defined as, e.g. a non-expert human would be able to classify similar objects, but possibly changed viewpoint, scene setting or clutter.
The library includes popular out-of-distribution datasets (ImageNetV2, ImageNet-C, etc.) and can be readily applied to benchmark arbitrary models and is not limited to vision models: any mapping from input -> logits will do.
Trustworthy AI Relevance
This metric addresses Robustness, Safety by quantifying relevant system properties. OOD generalization is fundamentally about ensuring AI systems remain reliable and perform safely when faced with novel or shifted data distributions, which aligns closely with the Robustness objective. By improving OOD generalization, the system reduces the risk of unexpected failures or harmful outputs in new environments, thereby supporting the Safety objective.
Related use cases :
Towards Out-Of-Distribution Generalization: A Survey
Uploaded on Oct 25, 2022Classic machine learning methods are built on the i.i.d. assumption that training and testing data are independent and identically distributed. However, in real scena...
About the metric
You can click on the links to see the associated metrics
Objective(s):
Lifecycle stage(s):
Target users:
Partnership on AI
