
The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.
A Simon Fraser University study found that popular AI chatbots, including ChatGPT and Llama, frequently direct users seeking health advice to for-profit clinics selling unregulated and potentially unsafe treatments. Nearly a quarter of chatbot responses referenced commercial sources, raising concerns about misinformation and harm to vulnerable patients.[AI generated]
Why's our monitor labelling this an incident or hazard?
The AI chatbots are generative AI systems explicitly mentioned as providing health-related advice. Their outputs have directly led users to unregulated, for-profit medical providers, which can mislead users and cause harm to their health or decision-making. This is a clear case of harm to people through misinformation and violation of rights to accurate health information. The study documents realized harm rather than just potential risk, so this is an AI Incident rather than a hazard or complementary information.[AI generated]