These tools and metrics are designed to help AI actors develop and use trustworthy AI systems and applications that respect human rights and are fair, transparent, explainable, robust, secure and safe.
Cohen's Kappa coefficient
Cohen's kappa coefficient is a statistic that is used to measure inter-rater reliability (and also intra-rater reliability) for qualitative (categorical) items. It is generally thought to be a more robust measure than simple percent agreement calculation, as κ takes into account the possibility of the agreement occurring by chance.
References
1. Nikulin, M.S., 2001. Hellinger distance. In Encyclopedia of Mathematics. EMS Press. 2. Hellinger, E., 1909. Neue Begründung der Theorie quadratischer Formen von unendlichvielen Veränderlichen. Journal für die reine und angewandte Mathematik, 136, pp.210–271. 3. Jeffreys, H., 1946. An invariant form for the prior probability in estimation problems. Proceedings of the Royal Society of London. Series A. Mathematical and Physical Sciences, 186(1007), pp.453–461. 4. Liese, F. and Miescke, K.-J., 2008. Statistical Decision Theory: Estimation, Testing, and Selection. Springer. 5. Pardo, L., 2006. Statistical Inference Based on Divergence Measures. Chapman and Hall/CRC.
About the metric
You can click on the links to see the associated metrics
Objective(s):
Purpose(s):
Lifecycle stage(s):
Target users:
