Conditional Entropy

We discuss information-theoretic anonymity metrics, that use entropy over the distribution of all possible recipients to quantify anonymity. We identify a common misconception: the entropy of the distribution describing the potential receivers does not always decrease given more information. We show the relation of these a-posteriori distributions with the Shannon conditional entropy, which is an average over all possible observations.

Trustworthy AI Relevance

This metric addresses Robustness and Privacy by quantifying relevant system properties. Robustness: Conditional entropy H(Y|X) quantifies how uncertain a model's outputs (or true labels) are given inputs. In practice, increases in conditional entropy under perturbation or new-test distributions indicate degraded predictive certainty and can signal distribution shift, OOD inputs, or brittleness to noise/adversarial changes.