New Zealand Develops AI Tool to Redirect Extremist Users to Deradicalization Support

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

ThroughLine, contracted by OpenAI, Anthropic, and Google, is developing an AI system in New Zealand to detect users exhibiting violent extremist tendencies on platforms like ChatGPT and redirect them to human and chatbot-based deradicalization support. The tool aims to prevent harm but is still in testing.[AI generated]

Why's our monitor labelling this an incident or hazard?

The event involves the use of an AI system (a chatbot and detection system) to identify and intervene with users showing violent extremist tendencies, which is a clear AI system involvement. However, the article does not report any actual harm or incident caused by the AI system; rather, it discusses the development and testing of a tool aimed at preventing harm. Therefore, it fits the definition of an AI Hazard, as the AI system's use could plausibly lead to preventing or managing harm related to extremism, but no incident has yet occurred. The article also discusses the broader context of safety concerns and potential misuse but does not describe a realized AI Incident or complementary information focused on responses to a past incident.[AI generated]
AI principles
Privacy & data governanceRespect of human rights

Industries
Government, security, and defence

Severity
AI hazard

Business function:
Citizen/customer service

AI system task:
Event/anomaly detectionInteraction support/chatbots


Articles about this incident or hazard

Thumbnail Image

Crisis contractor for OpenAI, Anthropic eyes a move to combat extremism By Reuters

2026-04-13
Investing.com
Why's our monitor labelling this an incident or hazard?
The event involves an AI system designed to detect and intervene in cases of violent extremism on AI platforms, which is a use of AI. However, the article does not report any actual harm caused by the AI system or any malfunction leading to harm. Instead, it focuses on the development and testing of a tool to prevent harm by redirecting at-risk users to support services. This aligns with societal and governance responses to AI safety concerns. Since no AI Incident or AI Hazard is described, and the main focus is on the development and potential deployment of a safety tool, the classification as Complementary Information is appropriate.
Thumbnail Image

Crisis contractor for OpenAI, Anthropic eyes a move to combat extremism

2026-04-14
ETCIO.com
Why's our monitor labelling this an incident or hazard?
The event involves the use of an AI system (a chatbot and detection system) to identify and intervene with users showing violent extremist tendencies, which is a clear AI system involvement. However, the article does not report any actual harm or incident caused by the AI system; rather, it discusses the development and testing of a tool aimed at preventing harm. Therefore, it fits the definition of an AI Hazard, as the AI system's use could plausibly lead to preventing or managing harm related to extremism, but no incident has yet occurred. The article also discusses the broader context of safety concerns and potential misuse but does not describe a realized AI Incident or complementary information focused on responses to a past incident.
Thumbnail Image

Crisis contractor for OpenAI, Anthropic eyes a move to combat extremism

2026-04-13
Economic Times
Why's our monitor labelling this an incident or hazard?
The article explicitly mentions AI systems used to detect signs of mental health crises and violent extremism, with the AI system routing users to support services. No actual harm or incident is reported; the tool is still in development and being tested. The potential for harm exists if the AI system fails to properly identify or manage extremist tendencies, or if follow-up mechanisms are insufficient, which could lead to harm to individuals or communities. Thus, the event fits the definition of an AI Hazard, as it plausibly could lead to an AI Incident in the future but has not yet done so.
Thumbnail Image

Crisis contractor for OpenAI, Anthropic eyes a move to combat extremism - The Economic Times

2026-04-14
Economic Times
Why's our monitor labelling this an incident or hazard?
The event involves the use of AI systems (chatbots and detection algorithms) to identify and intervene with users exhibiting extremist tendencies. The AI system's development and use are central to the event. However, the article does not report any actual harm caused by the AI system or its malfunction; rather, it discusses a proactive approach to prevent harm. This fits the definition of an AI Hazard, as the AI system's use could plausibly lead to preventing an AI Incident related to violent extremism. The article also discusses the broader context of AI safety and moderation but does not describe a realized harm or incident caused by AI.
Thumbnail Image

Crisis contractor for OpenAI, Anthropic eyes a move to combat extremism

2026-04-13
The Jerusalem Post
Why's our monitor labelling this an incident or hazard?
The article explicitly mentions the use of AI chatbots to detect violent extremist tendencies and route users to support, indicating AI system involvement. The event concerns the development and potential use of this AI system to prevent violent extremism, which is a significant harm. However, no actual harm caused by the AI system or its malfunction is reported; rather, the system is intended as a safety intervention. Thus, it is not an AI Incident. The event plausibly could lead to harm prevention or mitigation, making it an AI Hazard. It is not merely complementary information because the main focus is on the development of a new AI tool with potential impact on extremism-related harms, not on updates or responses to past incidents.
Thumbnail Image

Crisis contractor for OpenAI, Anthropic eyes a move to combat extremism

2026-04-13
Free Malaysia Today
Why's our monitor labelling this an incident or hazard?
The event involves the use of AI systems (chatbots and detection algorithms) to identify and intervene in cases of violent extremism and mental health crises. However, the article does not report any actual harm caused by the AI system or its malfunction; rather, it discusses the development and testing of a tool intended to prevent such harms. The AI system's involvement is in its use for intervention and support, with the potential to reduce harm. Since no direct or indirect harm has occurred yet, but there is a plausible risk of harm that the tool aims to address, this qualifies as Complementary Information about ongoing AI safety and governance efforts rather than an AI Incident or AI Hazard.
Thumbnail Image

AI Deradicalization Tools: How Chatbots Could Help Combat Violent Extremism Online - EconoTimes

2026-04-13
EconoTimes
Why's our monitor labelling this an incident or hazard?
The event involves the use and development of an AI system (a deradicalization chatbot) intended to mitigate harm by intervening in cases of online radicalization. Since the system is still under development and no harm has yet occurred or been reported, this constitutes a plausible future risk mitigation tool rather than an incident. The article focuses on the potential and ongoing development of this AI tool, which could plausibly lead to preventing AI-related harms. Therefore, it fits the definition of an AI Hazard, as it involves an AI system whose use could plausibly lead to preventing or mitigating harm related to violent extremism online.