Artists Use Data Poisoning to Sabotage AI Image Generators

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

Artists are intentionally altering images with tools like Nightshade to 'poison' datasets used by AI image generators such as Midjourney and DALL-E. This sabotage causes the AI to produce incorrect or nonsensical outputs, degrading reliability and utility as a form of protest against unauthorized use of their work.[AI generated]

Why's our monitor labelling this an incident or hazard?

The article explicitly involves AI systems (text-to-image generators) and describes a deliberate intervention (data poisoning) that affects their training and output. This intervention could plausibly lead to harms such as disruption of AI services and intellectual property violations. However, the article does not report any concrete harm or incident resulting from this poisoning, only the potential and ongoing use of the technique. The discussion of technological and governance responses further supports that this is an emerging risk rather than a realized incident. Hence, the classification as an AI Hazard is appropriate.[AI generated]
AI principles
Privacy & data governanceTransparency & explainabilityAccountabilityRobustness & digital securitySafetyRespect of human rights

Industries
Arts, entertainment, and recreationMedia, social platforms, and marketingDigital securityConsumer services

Affected stakeholders
ConsumersBusiness

Harm types
Economic/PropertyReputationalHuman or fundamental rights

Severity
AI hazard

Business function:
Research and developmentMonitoring and quality control

AI system task:
Content generationOther


Articles about this incident or hazard

Thumbnail Image

Artists turn AI against itself with tools designed to protect their work and break the algorithm -- here's how they are 'poisoning the well'

2023-12-18
Tom's Guide
Why's our monitor labelling this an incident or hazard?
The event involves the use of AI systems (image generators like Midjourney and Stable Diffusion) and a new AI-related technique (Nightshade) that manipulates training data to disrupt AI model performance. While no direct harm to people or property is reported, the technique is designed to cause disruption to AI systems by poisoning their training data, which could be seen as causing harm to AI systems' operation. However, the primary focus is on the potential and ongoing conflict between artists and AI developers, with no specific realized harm to individuals or communities described. The article mainly provides context on the evolving ecosystem, technological responses, and regulatory considerations. Therefore, this is best classified as Complementary Information, as it informs about societal and technical responses to AI use and misuse, without reporting a specific AI Incident or AI Hazard.
Thumbnail Image

Data poisoning: how artists are sabotaging AI to take revenge on image generators

2023-12-17
The Conversation
Why's our monitor labelling this an incident or hazard?
The article explicitly involves AI systems (text-to-image generators) and describes a deliberate intervention (data poisoning) that affects their training and output. This intervention could plausibly lead to harms such as disruption of AI services and intellectual property violations. However, the article does not report any concrete harm or incident resulting from this poisoning, only the potential and ongoing use of the technique. The discussion of technological and governance responses further supports that this is an emerging risk rather than a realized incident. Hence, the classification as an AI Hazard is appropriate.
Thumbnail Image

Commentary: How artists are sabotaging AI to take revenge on image generators

2023-12-18
CNA
Why's our monitor labelling this an incident or hazard?
The event involves AI systems (text-to-image generators) and their training data, which is being deliberately manipulated by a tool to cause malfunction or erroneous outputs. This manipulation is a use of AI system development or use that could plausibly lead to harm, such as disruption of AI services or indirect harm to users relying on these systems. Since no actual harm or incident is described, but a credible risk of harm exists, this qualifies as an AI Hazard rather than an AI Incident or Complementary Information.
Thumbnail Image

Data Poisoning: How Artists Are Sabotaging AI To Take Revenge On Image Generators - Stuff South Africa

2023-12-18
Stuff
Why's our monitor labelling this an incident or hazard?
The article explicitly involves AI systems (text-to-image generators like Midjourney and DALL-E) and describes how the use of poisoned training data causes these AI systems to malfunction, producing incorrect or nonsensical images. This malfunction directly harms users by degrading the quality and reliability of AI-generated images, which fits the definition of an AI Incident (harm caused by AI malfunction). The intentional poisoning of data is a misuse of AI development and use, leading to realized harm. Although the harm is indirect (the AI system's outputs are wrong), it is a direct consequence of the AI system's malfunction due to poisoned data. Therefore, this event qualifies as an AI Incident rather than a hazard or complementary information.
Thumbnail Image

MIL-Evening Report: Data poisoning: how artists are sabotaging AI to take revenge on image...

2023-12-17
foreignaffairs.co.nz
Why's our monitor labelling this an incident or hazard?
The event involves the use of AI systems (text-to-image generators) and a deliberate intervention (data poisoning) that could plausibly lead to harm by degrading AI model outputs and disrupting services. However, the article does not report any actual harm occurring yet, only the potential for harm through disrupted AI outputs and service reliability. Therefore, this qualifies as an AI Hazard because the development and use of data poisoning techniques could plausibly lead to AI incidents such as misinformation, degraded AI utility, or service disruption. The article also discusses governance and technological responses, but the main focus is on the potential risk rather than realized harm or a response to a past incident.
Thumbnail Image

Data poisoning: how artists are sabotaging AI to take revenge on image generators

2023-12-17
Tolerance
Why's our monitor labelling this an incident or hazard?
The article describes how data poisoning can cause AI image generators to produce incorrect images, which is a misuse of AI training data. While this could plausibly lead to harm (e.g., misleading outputs), the article does not report a specific event where harm has occurred or a concrete incident of malfunction. Instead, it explains the concept and its implications, which fits the definition of Complementary Information as it enhances understanding of AI risks and ecosystem developments without describing a new AI Incident or Hazard.
Thumbnail Image

Instapundit " Blog Archive " #RESIST: Data poisoning: how artists are sabotaging AI to take revenge on image generators. Resea

2023-12-19
InstaPundit.Com
Why's our monitor labelling this an incident or hazard?
The article explicitly involves AI systems (image generators) and discusses how the deliberate alteration of training images can cause these AI systems to malfunction, producing unpredictable and incorrect outputs. This constitutes a harm to the AI system's reliability and output quality, which can be considered harm to the AI ecosystem and potentially to communities relying on these AI outputs. Since the sabotage is intentional and leads to malfunction of AI systems, it fits the definition of an AI Incident due to the direct impact on the AI system's function and the resulting harm in terms of misinformation or degraded AI utility.
Thumbnail Image

Data poisoning: how artists are sabotaging AI to take revenge on image generators

2023-12-18
The National Tribune
Why's our monitor labelling this an incident or hazard?
The event involves AI systems (text-to-image generators) whose training data is intentionally manipulated (data poisoning) by artists. This manipulation leads to AI systems producing erroneous outputs, which is a malfunction caused by the poisoned data. The harm here is indirect: the AI systems' outputs become unreliable, which can disrupt their intended use and degrade service quality. However, there is no direct physical harm, violation of rights, or damage to critical infrastructure described. The article mainly discusses the phenomenon, its implications, and possible responses, without reporting a specific incident causing significant harm. Therefore, this is best classified as Complementary Information, as it provides context and understanding about AI system vulnerabilities and societal responses rather than reporting a concrete AI Incident or Hazard.