Utah Approves AI System for Automated Acne Prescriptions

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

Utah has authorized Nolla Health's AI system to independently prescribe acne treatments via its app, marking the first U.S. approval for AI-driven, end-to-end medical prescriptions. The pilot program starts with physician oversight but plans to reduce human involvement, raising concerns about potential health risks from AI-driven clinical decisions.[AI generated]

Why's our monitor labelling this an incident or hazard?

The AI system is explicitly involved in prescribing medication, a healthcare decision that directly affects patient health. Although no harm is reported, the autonomous nature of the AI prescribing without human oversight introduces a credible risk of injury or harm to patients if the AI malfunctions or errs. This fits the definition of an AI Hazard, as the event plausibly could lead to harm but no incident has yet occurred.[AI generated]
AI principles
SafetyAccountability

Industries
Healthcare, drugs, and biotechnology

Affected stakeholders
Consumers

Harm types
Physical (injury)

Business function:
Other

AI system task:
Goal-driven organisation


Articles about this incident or hazard