ChatGPT Misused for Malware Creation and Subscription Scams

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

Security researchers demonstrated that OpenAI's ChatGPT and Codex can be exploited to generate malicious code and phishing emails, enabling cyberattacks. Additionally, deceptive apps using ChatGPT's name have trapped users in expensive subscriptions, causing financial harm. These incidents highlight the risks of AI misuse for cybercrime and fraud.[AI generated]

Why's our monitor labelling this an incident or hazard?

The event involves AI systems explicitly, namely ChatGPT, an AI language model. The harm arises from the use of AI-based apps that mislead users into expensive subscription traps, causing financial harm (a form of harm to persons). The AI system's use is central to the incident, as the apps rely on ChatGPT's AI capabilities but exploit users through deceptive business practices. Therefore, this qualifies as an AI Incident due to realized harm (financial loss) linked directly to the use of AI systems in a misleading manner.[AI generated]
AI principles
SafetyRobustness & digital securityAccountabilityTransparency & explainabilityPrivacy & data governanceHuman wellbeing

Industries
Digital securityFinancial and insurance servicesConsumer servicesIT infrastructure and hosting

Affected stakeholders
Consumers

Harm types
Economic/PropertyReputationalPsychologicalPublic interest

Severity
AI incident

Business function:
Marketing and advertisementCitizen/customer serviceOther

AI system task:
Content generationInteraction support/chatbots

In other databases

Articles about this incident or hazard

Thumbnail Image

Achtung Abo-Fallen im App Store: 180 Euro für kostenloses KI-Tool ChatGPT

2022-12-20
Chip
Why's our monitor labelling this an incident or hazard?
The event involves AI systems explicitly, namely ChatGPT, an AI language model. The harm arises from the use of AI-based apps that mislead users into expensive subscription traps, causing financial harm (a form of harm to persons). The AI system's use is central to the incident, as the apps rely on ChatGPT's AI capabilities but exploit users through deceptive business practices. Therefore, this qualifies as an AI Incident due to realized harm (financial loss) linked directly to the use of AI systems in a misleading manner.
Thumbnail Image

Chance oder Gefahr? Was Schulen und Unis zu ChatGPT sagen

2022-12-21
Zweites Deutsches Fernsehen
Why's our monitor labelling this an incident or hazard?
The content focuses on the potential problems and risks associated with the use of ChatGPT in schools and universities, which could plausibly lead to harms such as undermining education quality or misinformation. Since no actual harm or incident is described, but plausible future harm is discussed, this fits the definition of an AI Hazard rather than an AI Incident or Complementary Information.
Thumbnail Image

Künstliche Intelligenz von ChatGPT: Gedichtanalyse, Goethe, Enter

2022-12-21
taz.de
Why's our monitor labelling this an incident or hazard?
The article centers on the use and implications of ChatGPT, an AI language model, in schools and education policy. While it mentions potential risks like reduced student independence and bias reproduction, no direct or indirect harm has occurred or is reported. The discussion is about possible future challenges and educational strategies, making it a contextual and informative piece rather than a report of an AI Incident or Hazard. Therefore, it fits best as Complementary Information, providing background and societal response to AI integration in education.
Thumbnail Image

KI-Wunder ChatGPT kann bösartige E-Mails und Code generieren

2022-12-22
PRESSEPORTAL
Why's our monitor labelling this an incident or hazard?
The article explicitly involves AI systems (ChatGPT and Codex) used to create malicious content and code. Although the event is a controlled demonstration by security researchers and no actual cyberattack harm has occurred, the potential for such AI-generated malicious outputs to be used in real cyberattacks is a plausible future harm. Therefore, this qualifies as an AI Hazard because it plausibly could lead to AI Incidents involving harm to property, communities, or individuals through cyberattacks.
Thumbnail Image

KI hilft Hackern: "ChatGPT, schreib mir einen Virus!"

2022-12-20
Computerwoche
Why's our monitor labelling this an incident or hazard?
The article explicitly states that ChatGPT, an AI system, was used to generate malicious code and phishing emails that enable cyberattacks. The researchers successfully created a full infection chain using ChatGPT, demonstrating the AI's role in facilitating harmful cyber activities. This constitutes direct involvement of an AI system in causing harm or enabling harm, fulfilling the criteria for an AI Incident. The harm includes potential injury to persons via cyberattacks, harm to property, and disruption of systems. Therefore, this event is classified as an AI Incident rather than a hazard or complementary information.
Thumbnail Image

纽约市教育部门禁止学生和教师使用 ChatGPT

2023-01-07
煎蛋
Why's our monitor labelling this an incident or hazard?
ChatGPT is an AI system explicitly mentioned. The event stems from the use of this AI system and concerns about its impact on education quality and academic integrity. However, the article does not report any realized harm or incident caused by ChatGPT but rather a precautionary ban to prevent potential negative effects. Therefore, this qualifies as an AI Hazard, as the AI system's use could plausibly lead to harms such as cheating or misinformation in education, but no direct or indirect harm has yet occurred according to the article.
Thumbnail Image

澎湖休憩園區打造美麗花海(1) (圖)

2023-01-10
Yahoo News (Taiwan)
Why's our monitor labelling this an incident or hazard?
The article mentions an AI system (ChatGPT) and its use in education, but it does not describe any direct or indirect harm resulting from its use, nor does it indicate a plausible future harm event. The content mainly provides background information and expert commentary on the benefits and risks of AI in learning, which fits the definition of Complementary Information rather than an Incident or Hazard.
Thumbnail Image

雲林縣政府舉辦年節畜禽產品推廣活動 (圖)

2023-01-10
Yahoo News (Taiwan)
Why's our monitor labelling this an incident or hazard?
The article mentions an AI system (ChatGPT) and its use in education but does not describe any realized harm or incident resulting from its use. It also does not present a specific credible risk or hazard scenario. The content is primarily informational and advisory, discussing the benefits and cautions of AI use without detailing any incident or hazard. Therefore, it fits the category of Complementary Information, as it provides context and expert views on AI's impact in education without reporting an AI Incident or AI Hazard.
Thumbnail Image

日本加強防疫 1/12起澳門入境旅客須持陰性證明 (圖)

2023-01-09
Yahoo News (Taiwan)
Why's our monitor labelling this an incident or hazard?
The article mentions an AI system (ChatGPT) and its use in education but does not describe any realized harm or incident resulting from its use. It also does not indicate a credible risk of harm or hazard stemming from the AI system. The focus is on general commentary and expert caution, which fits the definition of Complementary Information rather than an AI Incident or AI Hazard.
Thumbnail Image

有人用ChatGPT作弊!學者憂心:少了這動作,將加劇散播誤導性訊息

2023-01-09
Yahoo News (Taiwan)
Why's our monitor labelling this an incident or hazard?
The event involves an AI system (ChatGPT) whose use has led to concerns about cheating and the spread of misleading information. While there is mention of realized misuse (students cheating) and the potential for harm (misinformation causing real-world damage), the article primarily focuses on the risks and concerns rather than documenting a specific harmful incident with direct consequences. Therefore, it fits best as an AI Hazard, as it plausibly could lead to AI Incidents such as misinformation harm or academic dishonesty but does not describe a concrete incident of harm itself.
Thumbnail Image

用来改论文!纽约的学校禁止ChatGPT

2023-01-08
中关村在线
Why's our monitor labelling this an incident or hazard?
The event involves the use of an AI system (ChatGPT) and concerns about its misuse leading to academic dishonesty, which could harm students' education (harm to groups of people). Since no actual harm or incident has been reported yet, but the risk is credible and has led to preventive measures (banning ChatGPT on school networks and devices), this fits the definition of an AI Hazard. The article focuses on the potential for harm and the school's response rather than describing a specific AI Incident or realized harm.
Thumbnail Image

趣AI | 纽约市禁止学生和教师使用ChatGPT 以防止作弊

2023-01-07
chinaz.com
Why's our monitor labelling this an incident or hazard?
The article explicitly mentions ChatGPT, an AI system, and discusses its use and potential misuse in schools. However, it does not report any realized harm or incident caused by ChatGPT but rather a preventive ban to avoid cheating and concerns about content safety and accuracy. This fits the definition of Complementary Information, as it details a governance response to potential AI-related risks without describing an AI Incident or AI Hazard itself.
Thumbnail Image

AI助寫作業 「墮學習」衝擊校園 | 聯合新聞網

2023-01-09
UDN
Why's our monitor labelling this an incident or hazard?
The article clearly involves an AI system (ChatGPT) and its use in completing academic work. However, it does not describe any realized harm such as cheating scandals causing disciplinary actions, violations of rights, or other harms as defined in the framework. The concerns raised are about potential misuse and the need for proper guidance, which are speculative or general risks rather than specific incidents or hazards. Therefore, the article is best classified as Complementary Information, providing context and expert perspectives on AI's impact in education without reporting a concrete AI Incident or AI Hazard.
Thumbnail Image

兄弟 那个帮爷爷卖茶叶的善良女孩 可能是ChatGPT

2023-01-08
驱动之家
Why's our monitor labelling this an incident or hazard?
The article explicitly describes the use of ChatGPT, an AI system, in the development and deployment of malicious software and scam chatbots that have caused harm through fraud and cybercrime. This constitutes direct involvement of AI in causing harm to people and communities, fulfilling the criteria for an AI Incident. The harms include fraud (harm to individuals), unauthorized data theft (violation of rights), and enabling cybercrime. Therefore, this event is classified as an AI Incident.
Thumbnail Image

真实性存疑的ChatGPT收费应用在苹果App Store排行榜上飙升

2023-01-10
凤凰网(凤凰新媒体)
Why's our monitor labelling this an incident or hazard?
The event involves an AI system (a chatbot app using GPT-3 or similar technology) whose use has directly led to financial harm to users by charging for a service that is misleading and of poor quality. This constitutes a violation of consumer rights and could be considered a breach of obligations under applicable law protecting users from deceptive commercial practices. Therefore, this qualifies as an AI Incident due to realized harm (financial exploitation and deception) caused by the AI system's use.
Thumbnail Image

学校封杀,大厂禁用,ChatGPT引发大面积恐慌!

2023-01-07
163.com
Why's our monitor labelling this an incident or hazard?
ChatGPT is an AI system explicitly mentioned. Its use has directly led to concerns about academic cheating (harm to educational integrity) and intellectual property violations (legal risks from code sharing), which are forms of harm to rights and communities. The bans by schools and companies are responses to these realized harms. The article documents actual impacts and reactions, not just potential risks or general commentary. Hence, this is an AI Incident rather than a hazard or complementary information.
Thumbnail Image

微软Office或将支持ChatGPT技术 能自动编写电子邮件

2023-01-09
notebook.cnmo.com
Why's our monitor labelling this an incident or hazard?
The event describes a planned or potential use of AI technology (ChatGPT) in Microsoft Office products, focusing on feature enhancements and known limitations. There is no indication that any harm has occurred or that the AI system has malfunctioned or been misused leading to harm. Therefore, this is not an AI Incident or AI Hazard. It is a general AI-related news item about product plans and capabilities, which fits the definition of Complementary Information as it provides context and understanding of AI developments without reporting specific harms or risks realized or imminent.
Thumbnail Image

有問必答的AI聊天機器人ChatGPT 被發現遭駭客用來協助開發惡意程式 " 網路資訊雜誌

2023-01-09
網路資訊雜誌
Why's our monitor labelling this an incident or hazard?
The event involves the use of an AI system (ChatGPT) to develop malicious software that supports cybercrime activities such as ransomware, spyware, and phishing. This use directly leads to harm to communities and violations of applicable laws protecting rights and security. The article describes realized harm through the creation and sharing of malicious code, not just potential harm. Therefore, this qualifies as an AI Incident under the framework, as the AI system's use has directly led to significant harm.
Thumbnail Image

学校封杀,大厂禁用,ChatGPT引发大面积恐慌!

2023-01-09
新浪财经
Why's our monitor labelling this an incident or hazard?
ChatGPT is an AI system (a large language model) used for generating text and code. The article details how its use has led to concerns about academic cheating (harm to academic integrity) and risks of proprietary code leakage (intellectual property violations). These concerns have prompted bans in schools and companies, indicating that harms are either occurring or have been realized. The involvement of ChatGPT in these harms is direct or indirect, as its outputs facilitate or enable these issues. Thus, this qualifies as an AI Incident under the framework, as the AI system's use has directly or indirectly led to violations of intellectual property rights and academic integrity, which are breaches of obligations intended to protect fundamental and intellectual property rights.
Thumbnail Image

那个帮爷爷卖茶叶的善良女孩 可能是ChatGPT - AI 人工智能 - cnBeta.COM

2023-01-08
cnBeta.COM
Why's our monitor labelling this an incident or hazard?
The article explicitly describes ChatGPT, an AI system, being used by criminals to generate malicious code, phishing emails, ransomware, and scam chatbots that deceive and defraud people. These activities have directly caused harm to individuals through fraud and data theft, fulfilling the criteria for an AI Incident. The harms include violations of rights and harm to communities. The involvement of ChatGPT is central and direct in enabling these harms. The article also distinguishes these as realized harms, not just hypothetical risks, which excludes classification as an AI Hazard or Complementary Information. Hence, the correct classification is AI Incident.