AI Chatbots Spread False Medical Information After Experiment With Fabricated Disease

The event involves multiple AI chatbots generating and spreading false medical information about a non-existent disease, which is a direct consequence of their training data and response generation processes. This misinformation can harm individuals by misleading them about health conditions, potentially causing inappropriate health actions or anxiety, which constitutes harm to health and communities. The AI systems' outputs are central to the harm, fulfilling the criteria for an AI Incident. Although the original experiment was designed to be low risk, the real-world impact of AI systems treating the fictitious disease as real and disseminating false information is a clear harm. The event also includes responses and mitigation attempts but the primary focus is on the harm caused by AI-generated misinformation.

The event involves AI systems (ChatGPT, Gemini, Microsoft Copilot) that use advanced language models to generate medical advice. The AI systems were misled by fabricated scientific papers and propagated false medical information as if it were true. This misinformation has already caused harm by polluting medical knowledge and potentially misleading patients and doctors, which fits the definition of harm to communities and harm to health. The AI systems' use and malfunction (hallucination and acceptance of false data) directly led to this harm. Hence, this is an AI Incident rather than a hazard or complementary information.

An AI system (large language models/chatbots) is explicitly involved, as it accepted and propagated fabricated medical information. The harm is indirect but real: misinformation about a non-existent disease can mislead users, potentially causing health-related harm or confusion. The event also highlights risks in AI-generated content being taken as authoritative, leading to violations of trust and possibly health harm. Hence, this qualifies as an AI Incident due to the realized harm from AI's role in spreading false medical information.

The event involves the use of AI systems (large language models) in academic writing processes, which directly produce fabricated citations ('hallucinated citations'). These fabricated references constitute a violation of intellectual property rights and academic integrity, harming the scientific community and public trust in science. The harm is ongoing and increasing, with concrete examples and data supporting the presence of these AI-generated false citations in published literature. The article also discusses responses and mitigation efforts, but the primary focus is on the realized harm caused by AI misuse or malfunction. Hence, the classification as an AI Incident is appropriate.

The event involves AI systems (large language models) that have accepted and propagated fabricated medical information, demonstrating a malfunction in their information validation capabilities. While no direct harm has yet occurred, the plausible future harm includes patients receiving incorrect medical advice, leading to health risks and resource misallocation. The article also discusses the broader implications for trust in academic and medical information due to AI hallucinations. Since the harm is potential and not yet realized, and the AI's role is central to the propagation of false information, this qualifies as an AI Hazard rather than an AI Incident. The article also includes responses from AI companies and experts, but the main focus is on the experiment revealing AI vulnerabilities and the plausible risks arising from them.

The event involves AI systems (large language models) that generated and propagated false medical information about a fabricated disease. The AI's outputs led to misinformation spreading among the public and academic communities, with some researchers citing the fake disease in their work. This misinformation can cause harm to individuals and communities by misleading them about health conditions and potentially prompting unnecessary medical actions. The AI's role is pivotal as it accepted and disseminated the false information as factual, thus directly contributing to the harm. Although the false disease was intentionally fabricated as a test, the AI's failure to detect the falsity and its propagation of the misinformation constitute an AI Incident due to realized harm from AI-generated false medical advice and misinformation.

The event involves AI systems (LLMs) that were used and malfunctioned by generating and disseminating false medical information, which was then indirectly linked to harm through misinformation in healthcare contexts and academic literature. The AI's failure to correctly identify the fictitious disease and its propagation of false medical advice constitutes an AI Incident because it directly led to misinformation that can harm public health and trust. The citation of the false information in a medical journal further demonstrates real-world harm resulting from AI-generated misinformation. Therefore, this qualifies as an AI Incident due to realized harm from AI misuse and malfunction in a critical domain (health).

The event involves AI systems (large language models like ChatGPT, Google's Gemini, Perplexity AI, and Microsoft's Copilot) that were used to generate and spread false medical information about a fabricated disease. The AI systems' outputs directly misled users by presenting the fake condition as real, which can cause harm to individuals' health by influencing their medical decisions improperly. The harm is realized as misinformation is actively spread and trusted by users, fulfilling the criteria for an AI Incident under harm to health. The event is not merely a potential risk but an actual occurrence of AI-driven misinformation causing harm.

An AI system is explicitly involved as the experiment tests AI tools' responses to fabricated medical information. The AI systems' use in generating confident but false medical explanations demonstrates a misuse or misunderstanding of AI outputs in a health context. Although no direct harm is reported, the plausible risk of users relying on AI-generated false medical advice constitutes a credible potential for harm to health. Therefore, this event qualifies as an AI Hazard because it shows how AI-generated misinformation in health could plausibly lead to harm, even though no actual injury or health harm has yet occurred.

An AI system (large language models like ChatGPT, Gemini, etc.) was used and misled into generating false medical information, which could directly harm users by spreading misinformation about health. This constitutes harm to health through misinformation, fulfilling the criteria for an AI Incident because the AI's outputs have directly led to potential harm by misinforming users about a non-existent disease.

The event involves AI systems (large language models/chatbots) that have directly propagated false medical information, leading to misinformation that can harm individuals' health decisions and undermine trust in scientific research. The AI's role in repeating and legitimizing the fake diagnosis is pivotal to the harm. The harm is realized, not just potential, as the AI systems are actively providing false medical advice and influencing peer-reviewed publications. This fits the definition of an AI Incident because it involves harm to health and harm to communities through misinformation, caused directly or indirectly by AI system use.

The event involves AI systems (large language models) that, through their use, have directly led to the dissemination of false medical information about a fabricated illness. This misinformation can cause harm to people's health by misleading them about symptoms and treatments, fulfilling the criteria for harm to health (a). The AI's role is pivotal as it amplified and spread the fake condition widely and quickly, beyond the original fake studies. Therefore, this qualifies as an AI Incident due to the realized harm from AI-generated misinformation affecting health and communities.

The event involves AI systems (large language model chatbots) that absorbed and repeated false medical information, thereby spreading misinformation about a non-existent illness. This misinformation can harm individuals' health decisions and mislead communities, fulfilling the criteria for harm to health and communities. The AI's role in propagating the falsehood is direct and pivotal. Therefore, this qualifies as an AI Incident due to realized harm from AI-generated misinformation in a sensitive domain (health).

The AI systems involved are chatbots that use internet data to generate responses, thus qualifying as AI systems. The prank led to these AI systems providing false medical information, which poses a direct risk of harm to users' health if they act on such advice. Since the AI's use has directly led to misinformation that could cause injury or harm to people, this qualifies as an AI Incident under the definition of harm to health caused by AI system use.

The event involves AI systems (large language models) and their use (training on misinformation) leading to the replication and potential dissemination of false medical information. While the article does not describe a specific incident of harm occurring, it highlights a credible risk that such AI behavior could lead to harm to health (harm category a). Therefore, this qualifies as an AI Hazard because the development and use of these AI systems could plausibly lead to an AI Incident involving health harm.

The AI systems (ChatGPT, Gemini) were used or relied upon to generate or validate information about a fabricated disease, which was then accepted as legitimate by people, demonstrating a direct link between AI outputs and the spread of misinformation. This misinformation can harm public understanding and health decisions, fulfilling the criteria for harm to communities and health. The article explicitly states that the AI models treated the fake disease as real, thus playing a pivotal role in the misinformation spread. Hence, this is an AI Incident rather than a hazard or complementary information, as the harm is realized and not merely potential.

The article explicitly mentions AI systems (large language models) treating a fabricated disease as real, which is a clear example of AI hallucination leading to misinformation. While misinformation can cause harm to communities and individuals, the article does not document a concrete incident of harm resulting from this misinformation, only the plausible risk and societal implications. Therefore, this event fits best as Complementary Information, as it provides important context and understanding about AI's role in misinformation and the challenges of trust and deception, without describing a specific AI Incident or an imminent AI Hazard.

The article involves AI systems (large language models) that processed and propagated false information, which is an example of AI hallucination. However, it does not report a concrete AI Incident where harm has occurred, nor does it describe a specific AI Hazard with plausible future harm. The main focus is on understanding and raising awareness of the risks of deception involving AI, including human biases and trust issues. Therefore, this is best classified as Complementary Information, as it provides context and insight into AI-related risks without reporting a new incident or hazard.

While the article involves AI systems (large language models) and their role in spreading a fictional disease as if it were real, it does not report a specific incident where harm has occurred or is imminent. The spread of misinformation is a recognized risk, but the article frames it as a general concern and an example of AI hallucination rather than a documented AI Incident or Hazard. It also includes broader reflections on human nature and trust, and a social experiment on deception, which are not AI Incidents or Hazards. Thus, the article is best classified as Complementary Information, providing context and insight into AI's societal impacts and challenges without describing a direct or plausible harm event.

While the article involves AI systems (large language models) that contributed to the spread of false information, it does not report a concrete incident of harm resulting from this misinformation. The harm discussed is potential or general societal risk rather than a specific realized harm. The article also focuses on human cognitive biases and the need for critical thinking rather than on a particular AI incident or hazard. Therefore, it is best classified as Complementary Information, providing context and insight into AI-related misinformation issues without describing a distinct AI Incident or AI Hazard.

While the article involves AI systems (large language models) and discusses their role in spreading misinformation, it does not report a concrete incident of harm caused by these AI systems. The harm described is potential and general rather than specific and realized. The article also focuses on societal and cognitive aspects of misinformation and trust rather than on a particular AI incident or hazard. Therefore, it fits best as Complementary Information, providing context and insight into AI-related misinformation risks without describing a new AI Incident or AI Hazard.

The event involves AI systems (LLMs) that have been used and have propagated false medical information, but no direct or indirect harm (such as injury, rights violations, or community harm) has been reported as having occurred. The article focuses on the potential for harm through misinformation dissemination and the need for critical human oversight. Therefore, this event is best classified as Complementary Information, as it provides important context and understanding about AI's role in spreading misinformation and the challenges in verifying AI-generated content, without describing a realized AI Incident or a plausible AI Hazard causing harm.

The AI systems (ChatGPT, Gemini, Perplexity) are explicitly involved as they generated responses about a fabricated disease, which originated from manipulated and false data sources. The AI's role in spreading this misinformation is direct, as the models repeated and presented the false information as factual. Although the article does not report actual injuries, the potential for harm to health through misleading medical advice is clear and plausible. This fits the definition of an AI Incident because the AI's use has indirectly led to harm (or at least a significant risk of harm) to people's health by providing false medical information. The event is not merely a hazard since the misinformation is already being disseminated by AI systems, nor is it complementary information or unrelated news.

The event involves AI systems (LLMs) that have been shown to reproduce fabricated medical information, which could plausibly lead to harm such as misinformation-induced health risks or erosion of trust in medical knowledge. The experiment explicitly demonstrates the AI's role in spreading false information, fulfilling the criteria for AI system involvement and plausible future harm. However, since no direct or indirect harm has yet materialized from this misinformation (no injuries, no rights violations, no disruption), it does not qualify as an AI Incident. The event is not merely complementary information because it centers on the demonstration of AI's potential to cause harm, not just a response or update. Hence, the classification is AI Hazard.

An AI system (LLMs such as ChatGPT and Gemini) was involved in the use phase, where it reproduced and spread false medical information about a non-existent disease. This misinformation can harm communities by causing confusion, unnecessary concern, or inappropriate health actions, which qualifies as harm to communities under the framework. Since the AI systems directly propagated false information that could mislead users about health, this constitutes an AI Incident. The event is not merely a potential risk but an actual occurrence of AI systems spreading misinformation, fulfilling the criteria for an AI Incident.

The event involves AI systems (large language models) that reproduced false medical information, which is a form of misinformation that can harm communities by misleading users. However, the article frames this as a deliberate, educational experiment to demonstrate AI's limitations and risks, not as an uncontrolled incident causing actual harm. Since no direct or indirect harm has materialized yet, but there is a plausible risk of harm from AI systems spreading misinformation, this qualifies as an AI Hazard rather than an AI Incident. It is more than complementary information because it reports on the AI systems' role in spreading false information, but no harm has been realized yet.

The event involves AI systems (large language models/chatbots) that processed and accepted fabricated scientific information as true, leading to the spread of misinformation. This misinformation has already caused harm by entering scientific literature and misleading researchers, which is a violation of the reliability and integrity of information (harm to communities and potentially health). The AI systems' use and malfunction (failure to detect falsehoods) directly led to this harm. Hence, it meets the criteria for an AI Incident rather than a hazard or complementary information. The article does not merely warn about potential harm but documents actual misinformation spread involving AI systems.

An AI system (large language models/chatbots) is explicitly involved, and its use has directly led to harm in the form of misinformation spreading within scientific literature and AI outputs. This misinformation can harm communities by undermining trust in medical information and potentially influencing health decisions based on false data. Therefore, this qualifies as an AI Incident because the AI system's use has directly led to harm to communities through the spread of false medical information.