AI Bias in Mammogram Interpretation Based on Age and Race

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

A study of nearly 5,000 mammograms revealed that an FDA-approved AI algorithm showed bias in performance based on patient age and race, leading to false positives. This highlights potential human rights violations due to discrimination in healthcare, as the AI system's training lacked demographic diversity.[AI generated]

Why's our monitor labelling this an incident or hazard?

The AI system is explicitly involved as it is FDA-approved and used for mammography screening. The study shows the AI's use has directly led to harm in the form of disproportionate false positives for Black women, causing anxiety and unnecessary medical procedures, which qualifies as harm to groups of people. This is an AI Incident because the harm is realized and directly linked to the AI system's use and its biased performance due to training data limitations. The article also discusses ongoing investigation into false negatives, but the current realized harm suffices for classification as an AI Incident.[AI generated]
AI principles
FairnessRespect of human rightsSafetyHuman wellbeingAccountability

Industries
Healthcare, drugs, and biotechnology

Affected stakeholders
ConsumersWomen

Harm types
PsychologicalPhysical (injury)Economic/PropertyHuman or fundamental rights

Severity
AI incident

Business function:
Monitoring and quality control

AI system task:
Recognition/object detection


Articles about this incident or hazard

Thumbnail Image

AI More Likely To Wrongly Indicate Breast Cancer In Black Women

2024-05-22
Forbes
Why's our monitor labelling this an incident or hazard?
The AI system is explicitly involved as it is FDA-approved and used for mammography screening. The study shows the AI's use has directly led to harm in the form of disproportionate false positives for Black women, causing anxiety and unnecessary medical procedures, which qualifies as harm to groups of people. This is an AI Incident because the harm is realized and directly linked to the AI system's use and its biased performance due to training data limitations. The article also discusses ongoing investigation into false negatives, but the current realized harm suffices for classification as an AI Incident.
Thumbnail Image

Age, race impact AI performance on digital mammograms, study finds

2024-05-21
Medical Xpress - Medical and Health News
Why's our monitor labelling this an incident or hazard?
The event involves the use of an AI system (an FDA-approved AI algorithm for mammogram interpretation) whose performance varies by patient demographics, leading to false positive results. False positives in medical diagnostics can cause harm through unnecessary anxiety, additional testing, and potential overtreatment, which constitutes harm to health (a). Since the AI system's use has directly led to these false positives, this qualifies as an AI Incident. The study highlights realized harm (false positives) linked to the AI system's use, not just potential harm, so it is not merely a hazard or complementary information.
Thumbnail Image

Patient characteristics influence AI performance in mammogram interpretation

2024-05-22
News-Medical.net
Why's our monitor labelling this an incident or hazard?
The event involves an AI system (an FDA-approved AI algorithm for mammogram interpretation) and its use in clinical settings. However, the article describes research findings about variability in AI performance related to patient demographics without indicating that this variability has directly or indirectly caused harm such as injury, rights violations, or other significant harms. There is no report of actual incidents of harm or disruption caused by the AI system. The focus is on understanding performance differences and recommending considerations for future AI software development and deployment. Therefore, this is best classified as Complementary Information, as it provides important context and insights into AI system performance and potential areas for improvement, but does not describe an AI Incident or AI Hazard.
Thumbnail Image

AI's Breast Cancer Blind Spots Exposed by New Study

2024-05-21
SciTechDaily
Why's our monitor labelling this an incident or hazard?
The AI system (an FDA-approved mammography interpretation algorithm) is explicitly involved. The study shows that the AI's use leads to false positive results disproportionately affecting Black and older patients, which can cause harm such as unnecessary anxiety, additional testing, or treatment. This constitutes indirect harm to health due to biased AI outputs. Therefore, this qualifies as an AI Incident because the AI system's use has directly led to harm (false positives) impacting patient health and potentially violating equitable healthcare principles.
Thumbnail Image

Age, race impact AI performance on digital mammograms

2024-05-21
EurekAlert!
Why's our monitor labelling this an incident or hazard?
The article involves an AI system explicitly (an FDA-approved AI algorithm for mammogram interpretation) and discusses its use and performance. While it identifies disparities in false positive results linked to patient demographics, it does not report actual harm or incidents resulting from these disparities. The study's findings highlight potential risks and limitations that could plausibly lead to harm if unaddressed, but no specific incident or harm is described. Therefore, this is not an AI Incident or AI Hazard. The article primarily provides research findings that inform understanding and future improvements, fitting the definition of Complementary Information.
Thumbnail Image

New Study Finds Age and Race Impact Artificial Intelligence (AI) Performance on Digital Mammograms

2024-05-21
Imaging Technology News
Why's our monitor labelling this an incident or hazard?
The event involves an AI system explicitly mentioned as FDA-approved for interpreting mammograms. The study reveals that the AI's use leads to differential false positive results based on race and age, which can indirectly cause harm to patients through misdiagnosis or unnecessary follow-up procedures, thus impacting health. This constitutes an AI Incident because the AI system's use has directly led to realized harm (false positives) affecting patient health. The article does not merely discuss potential future harm or general AI research but reports on actual performance issues with an AI system in clinical use.
Thumbnail Image

AI Accuracy on Digital Mammograms Affected by Age, Race

2024-05-21
Mirage News
Why's our monitor labelling this an incident or hazard?
The AI system is explicitly involved as it interprets mammograms and generates risk scores. The study found that the AI's false positive rates vary significantly by patient race and age, indicating that the AI's outputs are not equally accurate across demographic groups. This disparity can lead to harm (a) injury or harm to health of persons, through unnecessary follow-up procedures and psychological distress. Since the harm is realized and linked to the AI system's use, this qualifies as an AI Incident. The article does not merely discuss potential risks or future hazards, but documents actual performance issues causing harm.
Thumbnail Image

FDA-Approved AI-Algorithm Reveals Higher False Positive Rates in Key Patient Groups

2024-05-22
Inside Precision Medicine
Why's our monitor labelling this an incident or hazard?
The article involves an AI system (an FDA-approved AI breast cancer screening algorithm) and discusses its use and performance. While it reveals disparities in false positive rates among different ethnic and age groups, it does not report any direct or indirect realized harm such as injury, rights violations, or operational disruption. The study is retrospective and analytical, highlighting potential risks and recommending transparency and improvements. There is no indication that these false positives have caused actual harm or that a near-miss or plausible future harm event is occurring. The main focus is on understanding and improving AI system fairness and transparency, which fits the definition of Complementary Information rather than an AI Incident or AI Hazard.