Catalogue of Tools & Metrics for Trustworthy AI
These tools and metrics are designed to help AI actors develop and use trustworthy AI systems and applications that respect human rights and are fair, transparent, explainable, robust, secure and safe.
Given a model and an input text sequence, perplexity measures how likely the model is to generate the input text sequence. This can be used in two main ways:
- to evaluate how well the model has learned the distribution of the text it was trained on. In this case, the model input should be the trained model to be evaluated, and the input texts should be the text that the model was trained on.
- to evaluate how well a selection of text matches the distribution of text that the input model was trained on. In this case, the model input should be a trained model, and the input texts should be the text to be evaluated.
About the metric
You can click on the links to see the associated metrics