ChatGPT Search Results Direct New Zealanders to Scam and Malicious Sites

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

ESET research found that ChatGPT's search results have routinely directed New Zealand users to unsafe websites, including scams, phishing pages, and malicious downloads. The AI's recommendations have led to increased cybersecurity risks, as users trust AI-generated links, resulting in potential data theft and financial harm.[AI generated]

Why's our monitor labelling this an incident or hazard?

ChatGPT is an AI system generating recommendations that lead users to harmful websites, which constitutes direct involvement of AI in causing harm. The harm includes exposure to scams, phishing, and malicious downloads, which can injure users' security and property. Since the AI system's outputs are directly linked to these harms, this qualifies as an AI Incident under the framework, specifically harm to persons and property through cybersecurity threats.[AI generated]
AI principles
SafetyRobustness & digital security
Industries
Digital security
Affected stakeholders
Consumers
Harm types
Economic/PropertyHuman or fundamental rights
AI system task:
Organisation/recommenders

Articles about this incident or hazard