Researchers Demonstrate Deepfake Detectors Can Be Fooled by Adversarial Attacks

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

UC San Diego researchers have shown that AI-based deepfake detectors can be deceived using adversarial examples—manipulated inputs that cause detection systems to misclassify fake videos as real. This vulnerability, demonstrated even without knowledge of the detector’s inner workings, poses a significant risk for undetectable misinformation and malicious media.[AI generated]

Why's our monitor labelling this an incident or hazard?

The event involves AI systems both in the creation of deepfakes and in their detection. The scientists demonstrated that AI-based detectors can be bypassed using adversarial examples, which is a malfunction or exploitation of the AI system's weaknesses. Although no specific harm has been reported as having occurred, the demonstrated vulnerability plausibly could lead to harms such as misinformation, reputational damage, or other societal harms associated with undetected deepfakes. Therefore, this event fits the definition of an AI Hazard, as it plausibly could lead to an AI Incident in the future if exploited.[AI generated]
AI principles
Robustness & digital securitySafetyAccountabilityDemocracy & human autonomy

Industries
Digital securityMedia, social platforms, and marketing

Affected stakeholders
General public

Harm types
Public interest

Business function:
Monitoring and quality control

AI system task:
Recognition/object detection


Articles about this incident or hazard