Metric Hub

Metric Hub is an online platform, developed by Germany's national metrology institute, the Physikalisch-Technische Bundesanstalt (PTB), and partners in the EU TEF-Health project. It supports the quantitative assessment of data quality for machine learning in medicine. Its core is the metric library: 60 data quality metrics for evaluating whether training and test data are fit for purpose. By making data quality measurable, it helps build the evidence of trustworthiness that clinicians, patients and regulators require, including the EU AI Act's data requirements for high-risk systems.

The library puts into practice the METRIC-framework, which defines 26 data quality dimensions grouped into five clusters: measurement process, timeliness, representativeness, informativeness and consistency. The metrics are mapped to the 14 dimensions that can be assessed quantitatively, such as completeness, noisy labels, target class balance and distribution drift. Each metric has a standardised metric card covering its definition, value range, applicability, prerequisites, pitfalls and recommendations.

To avoid computing every metric for every use case, users follow decision trees for each dimension. The trees translate use-case properties, such as data modality, variable type, machine learning task and availability of reference data, into a recommended set of metrics. The recommended workflow is to identify the dimensions most relevant to the risk and intended use, select metrics using the decision trees, implement them with the metric cards, and document the rationale. The approach was demonstrated on the PTB-XL electrocardiogram dataset and supports documentation for conformity assessment, audits and post-market monitoring.