Researchers Demonstrate ChatGPT's Ability to Generate Evasive Polymorphic Malware

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

CyberArk researchers showed that OpenAI's ChatGPT can be manipulated to generate highly evasive polymorphic malware, capable of mutating its code to evade detection by most anti-malware tools. This demonstrates a direct risk of AI systems being used to facilitate cyberattacks and harm digital infrastructure.[AI generated]

Why's our monitor labelling this an incident or hazard?

The event involves an AI system (ChatGPT) being used to produce malicious code (polymorphic malware) through deliberate prompting and bypassing of content filters. The malware's ability to mutate and evade detection constitutes a direct harm to property, communities, and potentially individuals by enabling cyberattacks. Since the AI system's use directly leads to this harm, this qualifies as an AI Incident under the framework's definition, specifically harm to property and communities due to malicious software creation.[AI generated]
AI principles
Robustness & digital securitySafetyAccountabilityTransparency & explainability

Industries
Digital securityIT infrastructure and hostingGovernment, security, and defence

Affected stakeholders
BusinessGovernmentGeneral public

Harm types
Economic/PropertyPublic interestReputational

Severity
AI incident

Business function:
ICT management and information securityResearch and development

AI system task:
Content generation


Articles about this incident or hazard

Thumbnail Image

ChatGPT can apparently make malware code on the fly, too (Mashable!)

2023-01-19
Tech Investor News
Why's our monitor labelling this an incident or hazard?
ChatGPT is an AI system capable of generating code. The report indicates that it can be manipulated to produce malware code, which is a direct misuse of the AI system leading to potential harm. Although the article does not describe a specific realized attack, the creation and potential use of malware generated by AI constitutes a credible risk of harm to cybersecurity and infrastructure. Therefore, this event qualifies as an AI Hazard because it plausibly could lead to an AI Incident involving harm to digital infrastructure or security.
Thumbnail Image

ChatGPT can be tricked into coding malware, say security researchers

2023-01-21
MoneyControl
Why's our monitor labelling this an incident or hazard?
The event involves an AI system (ChatGPT) being used to produce malicious code (polymorphic malware) through deliberate prompting and bypassing of content filters. The malware's ability to mutate and evade detection constitutes a direct harm to property, communities, and potentially individuals by enabling cyberattacks. Since the AI system's use directly leads to this harm, this qualifies as an AI Incident under the framework's definition, specifically harm to property and communities due to malicious software creation.
Thumbnail Image

ChatGPT can apparently make malware code on the fly, too

2023-01-19
Mashable
Why's our monitor labelling this an incident or hazard?
The article explicitly involves an AI system (ChatGPT) being used to generate malware code, which is malicious software that can cause harm to computer systems and users. Although no direct harm or incident is reported, the research reveals a method to circumvent safeguards, indicating a plausible pathway for future AI-enabled cyber harm. This fits the definition of an AI Hazard, as the development and use of the AI system could plausibly lead to an AI Incident involving harm to property or communities through malware attacks. There is no indication that harm has already occurred, so it is not an AI Incident. The article is not merely complementary information or unrelated news, as it focuses on the potential for harm stemming from AI misuse.
Thumbnail Image

ChatGPT Can Apparently Make Malware Code on the Fly, Too

2023-01-20
Mashable SEA
Why's our monitor labelling this an incident or hazard?
The article explicitly involves an AI system (ChatGPT) being manipulated to produce malware code, which is malicious software that can cause harm to computer systems and potentially to users. Although the article does not report a specific incident of harm occurring yet, it highlights a credible risk that the AI's misuse could lead to the creation and deployment of malware, which constitutes a plausible future harm. Therefore, this situation fits the definition of an AI Hazard, as the development and use of the AI system in this way could plausibly lead to an AI Incident involving harm to property, communities, or systems.
Thumbnail Image

ChatGPT Is Pretty Good at Writing Malware, It Turns Out

2023-01-20
Gizmodo
Why's our monitor labelling this an incident or hazard?
The event involves the use of an AI system (ChatGPT) to develop malware, which is a direct cause of harm to property and potentially to communities through cybercrime. The article reports that this is an ongoing and real concern, not merely hypothetical, indicating realized harm or at least active malicious use. Therefore, this qualifies as an AI Incident due to the direct link between AI use and harm caused by malware.
Thumbnail Image

ChatGPT Malware Shows It's Time To Get 'More Serious' About Security | CRN

2023-01-19
CRN
Why's our monitor labelling this an incident or hazard?
The event involves the use of an AI system (ChatGPT) in the development of malware code, which could plausibly lead to significant harms such as cyberattacks and security breaches. Although no actual incident of harm is described, the credible potential for such harm qualifies this as an AI Hazard. The article focuses on the risk and potential misuse rather than a realized AI Incident.
Thumbnail Image

ChatGPT Can Create Mutating Malware That Antiviruses Can't Detect

2023-01-21
TechWorm
Why's our monitor labelling this an incident or hazard?
The article explicitly involves an AI system (ChatGPT) being used to generate polymorphic malware, which is a form of harm to property and cybersecurity. The researchers demonstrated that the AI's outputs can be manipulated to produce malicious code that evades detection, directly linking the AI system's use to a realized harm scenario. This meets the criteria for an AI Incident as the AI system's use has directly led to harm through enabling advanced malware creation and evasion techniques.
Thumbnail Image

ChatGPT Could Create Polymorphic Malware Wave, Researchers Warn

2023-01-19
Dark Reading
Why's our monitor labelling this an incident or hazard?
The event involves the use of an AI system (ChatGPT) to generate malware, which directly relates to the development and use of AI systems leading to harm in cybersecurity contexts. The researchers explicitly state that this is a real concern, indicating that the AI's involvement could directly or indirectly lead to harm by enabling the creation of hard-to-detect malware. Therefore, this qualifies as an AI Incident due to the realized and ongoing risk of harm from AI-generated malware.
Thumbnail Image

Security experts develop method of generating 'highly evasive' polymorphic malware using ChatGPT | IT PRO

2023-01-21
IT PRO
Why's our monitor labelling this an incident or hazard?
The described malware uses an AI system (ChatGPT) in its operation to generate and adapt malicious code dynamically, which directly facilitates harmful activities like file encryption on victim PCs. This constitutes the use of an AI system leading to realized harm (malware infection and potential data loss or disruption), fitting the definition of an AI Incident. The harm is direct and significant, involving harm to property and communities through malware damage and potential ransomware effects.
Thumbnail Image

AI can make new malware

2023-01-19
Accounting Today
Why's our monitor labelling this an incident or hazard?
The article explicitly involves an AI system (ChatGPT) used in the development and operation of polymorphic malware. The malware's ability to mutate and evade detection represents a direct harm to property and cybersecurity. The AI system's involvement is not hypothetical but demonstrated by researchers, indicating realized harm potential. Therefore, this qualifies as an AI Incident due to the direct link between AI use and malicious harm.
Thumbnail Image

ChatGPT Malware Shows It's Time To Get 'More Serious' About ... - CRN

2023-01-19
Business Telegraph
Why's our monitor labelling this an incident or hazard?
The event involves the use of an AI system (ChatGPT) in a way that could plausibly lead to harm, specifically the creation of evasive malware that could cause injury to computer systems and potentially disrupt critical infrastructure or harm communities indirectly. The article does not describe an actual malware attack caused by ChatGPT but warns of the potential for such misuse. Therefore, this qualifies as an AI Hazard because it plausibly could lead to an AI Incident but no realized harm is reported yet.
Thumbnail Image

Resulta que ChatGPT es bastante bueno escribiendo malware

2023-01-23
Gizmodo en Español
Why's our monitor labelling this an incident or hazard?
The article explicitly involves an AI system (ChatGPT) used in the development of malware, which is a form of harm to property and communities. The malware's polymorphic nature allows it to evade security products, increasing the risk and actual occurrence of cyberattacks. The researchers' ability to bypass ChatGPT's safeguards and generate malicious code shows the AI's role in facilitating harmful outcomes. Therefore, this qualifies as an AI Incident due to the direct involvement of AI in causing realized harm through malware creation.
Thumbnail Image

¿Puede una IA dañar nuestro PC con malware?: ChatGPT empieza a programar código malicioso y es alarmante

2023-01-24
Vandal
Why's our monitor labelling this an incident or hazard?
The event involves the use of an AI system (ChatGPT) to develop malware capable of harming computer systems, which constitutes harm to property and potentially to users. The AI's involvement is in its use to generate malicious code, which has directly or indirectly led to a cybersecurity threat. Although no specific incident of infection is reported, the article emphasizes the real and current capability and use of AI to create harmful malware, indicating realized harm in the form of malware development and potential infections. Therefore, this qualifies as an AI Incident due to the direct link between AI use and harm (malware creation and cybersecurity risk).
Thumbnail Image

ChatGPT: La Inteligencia Artificial podría infectar nuestros smartphones y PC con código malicioso

2023-01-24
FayerWayer
Why's our monitor labelling this an incident or hazard?
The article explicitly involves an AI system (ChatGPT) and its use in generating malicious code. The harm described (malware infection causing damage to devices and users) is a recognized form of harm. However, the article does not report any realized incident of malware infection caused by AI-generated code but rather warns about the plausible future risk of such harm. Therefore, this qualifies as an AI Hazard, as the AI system's misuse could plausibly lead to an AI Incident in the future.
Thumbnail Image

Chat GPT, los ciberdelincuentes pueden usarlo como parte de sus estafas

2023-01-26
El Mundo Financiero
Why's our monitor labelling this an incident or hazard?
The article explicitly mentions ChatGPT, an AI system, being used in the creation of personalized phishing emails and malicious code, which directly leads to harm through cybercrime (harm to persons and organizations). Although the article notes that current AI-generated malware may be detected and is not fully autonomous, the realized use of AI in cyberattacks constitutes an AI Incident due to the direct link to harm. The discussion of future risks and defensive uses complements this but does not negate the current incident status.
Thumbnail Image

CyberArk Labs permite a ChatGPT mutar el código malicioso por sí mismo

2023-01-23
Nuevo Periodico
Why's our monitor labelling this an incident or hazard?
The event involves the use of an AI system (ChatGPT) in the creation and mutation of malware that can evade security detection, which directly relates to harm in the form of cybersecurity threats (harm to property, communities, or environment). The AI system's use in generating and mutating malicious code that executes without detection constitutes a direct contribution to the harm. Therefore, this qualifies as an AI Incident because the AI system's use has directly led to a significant harm scenario involving cybersecurity risks.