Exact Match

A given predicted string’s exact match score is 1 if it is the exact same as its reference string, and is 0 otherwise.

• Example 1: The exact match score of prediction “Happy Birthday!” is 0, given its reference is “Happy New Year!”.
• Example 2: The exact match score of prediction “The Colour of Magic (1983)” is 1, given its reference is also “The Colour of Magic (1983)”.

The exact match score of a set of predictions is the sum of all of the individual exact match scores in the set, divided by the total number of predictions in the set.

• Example: The exact match score of the set {Example 1, Example 2} (above) is 0.5.

Trustworthy AI Relevance

This metric addresses Robustness and Transparency by quantifying relevant system properties. Robustness: EM directly measures whether a model consistently returns exactly correct outputs, so it is useful for assessing reliability under different conditions (e.g., dataset shifts, noisy inputs, model variants). Tracking EM across test slices, perturbations, or OOD examples helps identify brittleness, regressions, or failure modes — all central to robustness evaluation.