FDA's artificial intelligence is supposed to revolutionize drug approvals. It's making up nonexistent studies.
2025-07-23
CNN International
The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.
The FDA's generative AI tool, Elsa, designed to expedite drug approval processes, has been found to hallucinate—fabricating non-existent studies and misinterpreting real research. Employees report that Elsa's unreliable outputs require extensive human verification, raising concerns about potential risks to drug safety if its outputs are trusted without oversight.[AI generated]
Why's our monitor labelling this an incident or hazard?
Elsa is an AI system explicitly mentioned as being used in the FDA's drug approval workflow. Its malfunction—hallucinating nonexistent studies and misrepresenting research—directly affects the reliability of critical health regulatory work. This can lead to harm to public health if erroneous information influences drug approval decisions. Although the AI is currently used with human oversight and is optional, the presence of hallucinations and misrepresentations in a high-stakes health context meets the criteria for an AI Incident due to the direct link to potential harm to health. The article details realized issues with the AI's outputs, not just potential risks, so it is not merely a hazard or complementary information.[AI generated]