Catalogue of Tools & Metrics for Trustworthy AI
These tools and metrics are designed to help AI actors develop and use trustworthy AI systems and applications that respect human rights and are fair, transparent, explainable, robust, secure and safe.
Perplexity
Given a model and an input text sequence, perplexity measures how likely the model is to generate the input text sequence. This can be used in two main ways:
- to evaluate how well the model has learned the distribution of the text it was trained on. In this case, the model input should be the trained model to be evaluated, and the input texts should be the text that the model was trained on.
- to evaluate how well a selection of text matches the distribution of text that the input model was trained on. In this case, the model input should be a trained model, and the input texts should be the text to be evaluated.
Related use cases :
Mirostat: A Neural Text Decoding Algorithm that Directly Controls Perplexity
Uploaded on Nov 1, 2022Neural text decoding is important for generating high-quality texts using language models. To generate high-quality text, popular decoding algorithms like top-k, top-p (nucleus...
How much complexity does an RNN architecture need to learn syntax-sensitive dependencies?
Uploaded on Nov 1, 2023About the metric
You can click on the links to see the associated metrics
Objective(s):
Purpose(s):
Lifecycle stage(s):
Target users:
