Singapore Health Ministry Disables Chatbot After Misleading COVID-19 Advice

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

Singapore's Ministry of Health disabled its 'Ask Jamie' AI chatbot after it provided misleading and inappropriate responses to COVID-19 queries, such as suggesting safe sex and polio vaccines instead of relevant health advice. The malfunction raised concerns about public health misinformation and prompted a system review.[AI generated]

Why's our monitor labelling this an incident or hazard?

Ask Jamie is an AI system employing NLP to generate responses. The chatbot's misaligned reply to a COVID-19 question constitutes a malfunction in its use, resulting in the dissemination of incorrect health advice. This misinformation can harm public health understanding and trust, thus qualifying as harm to communities. Since the harm has occurred due to the AI system's malfunction, this event qualifies as an AI Incident rather than a hazard or complementary information.[AI generated]
AI principles
SafetyRobustness & digital securityHuman wellbeing
Industries
Healthcare, drugs, and biotechnologyGovernment, security, and defence
Affected stakeholders
General public
Harm types
Public interest
Severity
AI incident
Business function:
Citizen/customer service
AI system task:
Interaction support/chatbots

Articles about this incident or hazard

Thumbnail Image

MOH temporarily disables Ask Jamie chatbot after 'misaligned replies'

2021-10-05
CNA
Why's our monitor labelling this an incident or hazard?
Ask Jamie is an AI system employing NLP to generate responses. The chatbot's misaligned reply to a COVID-19 question constitutes a malfunction in its use, resulting in the dissemination of incorrect health advice. This misinformation can harm public health understanding and trust, thus qualifying as harm to communities. Since the harm has occurred due to the AI system's malfunction, this event qualifies as an AI Incident rather than a hazard or complementary information.
Thumbnail Image

Gaffes by Ask Jamie chatbot on MOH site leaves netizens bemused and

2021-10-04
The Straits Times
Why's our monitor labelling this an incident or hazard?
The Ask Jamie chatbot is an AI system using natural language processing to provide health-related advice. Its malfunction caused it to give incorrect advice about Covid-19, instead giving family planning advice, which is misleading and could harm users' health decisions. The event involves the use and malfunction of an AI system leading to indirect harm to health by misinformation. The chatbot's removal indicates recognition of the harm caused. Hence, this is an AI Incident as per the definitions provided.
Thumbnail Image

Singapore Health Ministry takes down 'Ask Jamie' chatbot which gave 'misaligned' Covid-19 advice | Malay Mail

2021-10-05
Malay Mail
Why's our monitor labelling this an incident or hazard?
The 'Ask Jamie' chatbot is an AI system employing natural language processing to provide health-related advice. Its malfunction caused it to give misleading and incorrect Covid-19 advice, which can directly harm individuals relying on this information for health decisions. The Ministry's action to disable the chatbot confirms the recognition of harm caused by the AI system's outputs. Therefore, the event meets the criteria for an AI Incident due to the AI system's malfunction leading to potential injury or harm to health.
Thumbnail Image

Singapore government chatbot fluffs COVID questions

2021-10-06
theregister.com
Why's our monitor labelling this an incident or hazard?
The chatbot is an AI system using natural language processing to answer health-related questions. Its malfunction resulted in inappropriate and misleading health advice, which constitutes harm to the health of individuals (harm category a). The event involves the AI system's use and malfunction leading to realized harm through misinformation. Therefore, this qualifies as an AI Incident.
Thumbnail Image

GG! Singapore MOH Disables Their Chatbot After It Gave Out Outrageous Replies

2021-10-07
WORLD OF BUZZ
Why's our monitor labelling this an incident or hazard?
The chatbot 'Ask Jamie' is an AI system designed to provide health advice. Its malfunction led to the dissemination of misleading health information, which could indirectly cause harm to individuals relying on it for Covid-19 guidance. Although no direct harm is reported yet, the potential for harm from incorrect medical advice is significant. Therefore, this event qualifies as an AI Incident due to the AI system's malfunction causing or potentially causing harm to health.