These tools and metrics are designed to help AI actors develop and use trustworthy AI systems and applications that respect human rights and are fair, transparent, explainable, robust, secure and safe.
WATCH: Adaptive Monitoring for AI Deployments via Weighted-Conformal Martingales
WATCH (Weighted Adaptive Testing for Changepoint Hypotheses) is a statistical monitoring framework for detecting unsafe behavior in deployed AI/ML systems. The method introduces weighted-conformal test martingales (WCTMs), a generalisation of conformal test martingales that extends nonparametric sequential testing beyond simple exchangeability checks. It provides a way to raise alarms during operation when sequential evidence indicates unsafe behavior or distributional changes, supporting continuous verification. In practice, WATCH continuously monitors an AI system's inputs and outputs after deployment. It is designed to:
- Adapt online to mild, benign shifts in the input data distribution (covariate shift) without raising unnecessary alarms;
- Quickly detect harmful distribution shifts that could degrade system safety or performance;
- Diagnose the cause of a detected shift by identifying whether it results from concept shift or extreme out-of-support covariate shift
It is intended for researchers and practitioners who need formal, post-deployment safety/robustness monitoring guarantees for high-stakes AI systems.
Auto-discovered on 2026-09-09 by OECD Catalogue Automation
About the metric
You can click on the links to see the associated metrics
Metric type(s):
Objective(s):
Lifecycle stage(s):
Usage rights:
Target users:
Risk management stage(s):
Partnership on AI
