Catalogue of Tools & Metrics for Trustworthy AI

These tools and metrics are designed to help AI actors develop and use trustworthy AI systems and applications that respect human rights and are fair, transparent, explainable, robust, secure and safe.

OverviewToolsMetricsAbout the catalogue
Contribute to the catalogue
Contribute to the catalogue

Agent Goal Accuracy

Github
Website

Agent Goal Accuracy is a metric used to evaluate the effectiveness of a language model in accurately identifying and achieving a user’s intended goals during an interaction. This binary metric assigns a score of 1 if the AI successfully accomplishes the user’s goal and 0 if it does not. It is particularly valuable in assessing the performance of AI agents in task-oriented dialogues, where the objective is to fulfill specific user requests.

 

Formula:

Agent Goal Accuracy = (Number of Successfully Achieved Goals) / (Total Number of Goals)

References

Agentic or Tool use - Ragas

About the metric

You can click on the links to see the associated metrics

Metric type(s):

Objective(s):

Purpose(s):

Target sector(s):

Lifecycle stage(s):

Usage rights:

Target users:

Risk management stage(s):

Github stars:

  • 7100

Github forks:

  • 723

Modify this metric

catalogue Logos

Disclaimer: The tools and metrics featured herein are solely those of the originating authors and are not vetted or endorsed by the OECD or its member countries. The Organisation cannot be held responsible for possible issues resulting from the posting of links to third parties' tools and metrics on this catalogue. More on the methodology can be found at https://oecd.ai/catalogue/faq.