Catalogue of Tools & Metrics for Trustworthy AI

These tools and metrics are designed to help AI actors develop and use trustworthy AI systems and applications that respect human rights and are fair, transparent, explainable, robust, secure and safe.

SUBMIT A METRIC USE CASE

If you have a metric use case that you think should be featured in the Catalogue of Tools & Metrics for Trustworthy AI, we would love to hear from you!

SUBMIT

Interaction support/chatbotsRecognition/object detectionUploaded on Jun 5, 2024
Multimodal Large Language Models (MLLMs) excel in generating responses based on visual inputs. However, they often suffer from a bias towards generating responses similar to their ...

Objective(s)


Reasoning with knowledge structures/planningRecognition/object detectionUploaded on Jun 5, 2024
The BigCode project, an open-scientific collaboration focused on the responsible development of Large Language Models for Code (Code LLMs), introduces StarCoder2. In partnership wi...

Recognition/object detectionUploaded on Jun 5, 2024
In Multiple Object Tracking (MOT), tracking-by-detection methods have stood the test for a long time, which split the process into two parts according to the definition: object det...

Recognition/object detectionUploaded on Jun 5, 2024
This paper introduces fourteen novel datasets for the evaluation of Large Language Models' safety in the context of enterprise tasks. A method was devised to evaluate a model's saf...

Recognition/object detectionUploaded on Jun 5, 2024
Retrieval-Augmented Generation (RAG) is a prevalent approach to infuse a private knowledge base of documents with Large Language Models (LLM) to build Generative Q\&A (Question-Ans...

Recognition/object detectionUploaded on Jun 5, 2024
We propose a novel model-selection method for dynamic real-life networks. Our approach involves training a classifier on a large body of synthetic network data. The data is generat...

Recognition/object detectionUploaded on Jun 5, 2024
Multimodal Large Language Models (MLLMs) have experienced significant advancements recently. Nevertheless, challenges persist in the accurate recognition and comprehension of intri...

Recognition/object detectionUploaded on Jun 5, 2024
In this paper, we introduce an open-vocabulary panoptic segmentation model that effectively unifies the strengths of the Segment Anything Model (SAM) with the vision-language CLIP ...

Objective(s)


Recognition/object detectionUploaded on Jun 5, 2024
Vision Transformer (ViT) self-attention mechanism is characterized by feature collapse in deeper layers, resulting in the vanishing of low-level visual features. However, such feat...

Reasoning with knowledge structures/planningRecognition/object detectionUploaded on Jun 5, 2024
In this work, we introduce Mini-Gemini, a simple and effective framework enhancing multi-modality Vision Language Models (VLMs). Despite the advancements in VLMs facilitating basic...

Recognition/object detectionUploaded on Jun 5, 2024
Today's deep learning methods focus on how to design the most appropriate objective functions so that the prediction results of the model can be closest to the ground truth. Meanwh...

Event/anomaly detectionReasoning with knowledge structures/planningRecognition/object detectionUploaded on Jun 5, 2024
Table-based reasoning with large language models (LLMs) is a promising direction to tackle many table understanding tasks, such as table-based question answering and fact verificat...

Recognition/object detectionUploaded on Jun 5, 2024
The task of stock earnings forecasting has received considerable attention due to the demand investors in real-world scenarios. However, compared with financial institutions, it is...

Recognition/object detectionUploaded on Jun 5, 2024
Image classifiers often rely on convolutional neural networks (CNN) for their tasks, which are inherently more heavyweight than multilayer perceptrons (MLPs), which can be problema...

Objective(s)


Reasoning with knowledge structures/planningRecognition/object detectionUploaded on Jun 5, 2024
Large language models (LLMs) have shown great potential in complex reasoning tasks, yet their performance is often hampered by the scarcity of high-quality and reasoning-focused tr...

Objective(s)


Interaction support/chatbotsRecognition/object detectionUploaded on May 21, 2024
Multimodal Large Language Models (MLLMs) excel in generating responses based on visual inputs. However, they often suffer from a bias towards generating responses similar to their ...

Objective(s)


Reasoning with knowledge structures/planningRecognition/object detectionUploaded on May 21, 2024
The BigCode project, an open-scientific collaboration focused on the responsible development of Large Language Models for Code (Code LLMs), introduces StarCoder2. In partnership wi...

Recognition/object detectionUploaded on May 21, 2024
In Multiple Object Tracking (MOT), tracking-by-detection methods have stood the test for a long time, which split the process into two parts according to the definition: object det...

Recognition/object detectionUploaded on May 21, 2024
This paper introduces fourteen novel datasets for the evaluation of Large Language Models' safety in the context of enterprise tasks. A method was devised to evaluate a model's saf...

Recognition/object detectionUploaded on May 21, 2024
Retrieval-Augmented Generation (RAG) is a prevalent approach to infuse a private knowledge base of documents with Large Language Models (LLM) to build Generative Q\&A (Question-Ans...

catalogue Logos

Disclaimer: The tools and metrics featured herein are solely those of the originating authors and are not vetted or endorsed by the OECD or its member countries. The Organisation cannot be held responsible for possible issues resulting from the posting of links to third parties' tools and metrics on this catalogue. More on the methodology can be found at https://oecd.ai/catalogue/faq.