Ombudsman Warns Against Automated AI Reviews in Medical Disability Decisions

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

Bulgarian Ombudsman Velislava Delcheva opposes a proposed law enabling an automated AI system to flag and trigger reviews of medical disability decisions (TELK/NELK). She warns this could lead to wrongful loss of rights and payments without human oversight, urging legal safeguards and transparency before implementation.[AI generated]

Why's our monitor labelling this an incident or hazard?

The event involves an automated software system used in medical disability assessments, which can be reasonably inferred to involve AI or algorithmic decision-making given the description of risk detection and automated evaluation. The Ombudsman warns that this system could lead to wrongful denial or revocation of rights and payments, which is a violation of fundamental rights and harm to individuals. However, the article does not report actual cases of harm occurring yet, but rather highlights the risk and calls for safeguards, human oversight, and transparency. Thus, the event describes a plausible future harm scenario linked to AI system use, fitting the definition of an AI Hazard. It is not an AI Incident because no direct or indirect harm has materialized yet. It is not Complementary Information because the article focuses on the risk and critique of the system's design and deployment, not on responses or updates to a past incident. It is not Beneficial Use because the AI system is not solely a countermeasure to external harm but is involved in decisions that could themselves cause harm. It is not Unrelated because the event clearly involves an AI system and its potential impact on rights.[AI generated]
AI principles
Respect of human rightsTransparency & explainability

Industries
Healthcare, drugs, and biotechnologyGovernment, security, and defence

Affected stakeholders
General public

Harm types
Economic/PropertyHuman or fundamental rights

Business function:
Citizen/customer service

AI system task:
Event/anomaly detection


Articles about this incident or hazard