AI Chest X-Ray Model Found to Exhibit Race and Sex Bias, Raising Patient Safety Concerns

The article explicitly involves an AI system (a chest X-ray foundation model) used for disease detection. The study found that the AI system underperforms for certain racial and sex subgroups, indicating bias that could cause harm to patients if used clinically. This constitutes a violation of rights related to equitable healthcare and harm to health (a). Since the harm is realized in the model's performance and its potential clinical application, this qualifies as an AI Incident rather than a mere hazard or complementary information.

The event involves an AI system explicitly described as a chest X-ray foundation model used for disease detection. The study demonstrates that the AI system's use leads to biased performance affecting different racial and sex subgroups, which can cause harm to patients' health and violate rights to equitable medical treatment. Since the harm is realized in the form of uneven and potentially unsafe clinical application, this qualifies as an AI Incident under the framework, specifically harm to health and violation of rights.

The article explicitly discusses an AI system (a chest X-ray foundation model) used for disease detection that demonstrates significant bias affecting different racial and sex subgroups. This bias leads to uneven and potentially unsafe clinical outcomes, which can harm patient health and violate rights to equitable medical treatment. The harm is realized through the AI system's use and its biased outputs, fulfilling the criteria for an AI Incident under the definitions provided.

The article explicitly discusses an AI system (a chest X-ray foundation model) used for disease detection that demonstrates bias causing uneven performance across racial and sex subgroups. This bias can lead to misdiagnosis or missed diagnosis, which constitutes harm to health (a). The study highlights that the model may be unsafe for clinical applications due to these biases, indicating realized or imminent harm. The AI system's use is directly linked to these harms through its biased outputs. Therefore, this qualifies as an AI Incident under the framework because the AI system's use has directly led to harm or risk of harm to persons' health and potentially violates equitable treatment principles.

The article explicitly discusses an AI system (a chest X-ray foundation model) used for disease detection that exhibits bias based on race and sex, resulting in uneven performance and potential clinical inaccuracy. This bias can directly harm patients by leading to misdiagnosis or missed diagnosis, which is injury or harm to health (harm category a). The AI system's development and use have directly led to this harm, fulfilling the criteria for an AI Incident. The article does not merely warn of potential future harm but reports actual bias found in a deployed model, indicating realized harm. Hence, the event is classified as an AI Incident.