Real-Time AI Deepfakes Pose Growing Fraud Risk

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

Multiple reports highlight the rising risk of real-time AI deepfakes in digital scams. Marco Ramilli of IdentifAI explains how AI systems can alter faces and voices in live video calls, deceiving verification systems and potentially violating individuals’ rights. This emerging threat is seen as a major AI hazard.[AI generated]

Why's our monitor labelling this an incident or hazard?

The event involves AI systems explicitly (real-time deepfake generation using AI) and their use has directly led to harms in the form of fraud and deception, which constitute harm to individuals and communities. The article provides data on the number of incidents recorded, indicating that these harms are occurring and increasing. Therefore, this qualifies as an AI Incident because the AI system's use has directly led to significant harm through fraudulent activities.[AI generated]
AI principles
Privacy & data governanceRespect of human rightsRobustness & digital securitySafetyTransparency & explainabilityAccountability

Industries
Financial and insurance servicesDigital securityGovernment, security, and defenceMedia, social platforms, and marketingConsumer services

Affected stakeholders
Consumers

Harm types
Economic/PropertyHuman or fundamental rightsReputationalPublic interestPsychological

Severity
AI incident

Business function:
Citizen/customer serviceICT management and information security

AI system task:
Content generation


Articles about this incident or hazard

Thumbnail Image

Deepfake in tempo reale, ecco la nuova frontiera delle frodi con l'intelligenza artificiale

2025-05-10
Giornale di Sicilia
Why's our monitor labelling this an incident or hazard?
The event involves AI systems explicitly (real-time deepfake generation using AI) and their use has directly led to harms in the form of fraud and deception, which constitute harm to individuals and communities. The article provides data on the number of incidents recorded, indicating that these harms are occurring and increasing. Therefore, this qualifies as an AI Incident because the AI system's use has directly led to significant harm through fraudulent activities.
Thumbnail Image

L'IA sempre più utilizzata nelle truffe digitali, i deep fake in tempo reale saranno un incubo per tutti

2025-05-12
Tiscali Notizie
Why's our monitor labelling this an incident or hazard?
The event involves AI systems explicitly (deepfake generation, voice cloning) used in fraudulent activities that have already caused harm to people, such as financial fraud and misinformation. The article provides data on the number of incidents and examples of harm, including targeting vulnerable groups like the elderly. Therefore, this qualifies as an AI Incident because the AI system's use has directly led to harm to persons and communities through deception and fraud.
Thumbnail Image

Deepfake in tempo reale, è la nuova frontiera delle frodi IA - Future Tech - Ansa.it

2025-05-10
ANSA.it
Why's our monitor labelling this an incident or hazard?
The event involves AI systems explicitly (real-time deepfake generation using AI) used maliciously to perpetrate fraud, which constitutes harm to individuals and communities. The article provides data on numerous incidents already occurring, confirming that harm is realized rather than potential. Therefore, this qualifies as an AI Incident due to the direct involvement of AI in causing harm through fraud and deception.
Thumbnail Image

Deepfake e AI generativa: la nuova frontiera delle truffe online e digitali

2025-05-12
HTML.it
Why's our monitor labelling this an incident or hazard?
The event involves AI systems explicitly (AI generative models creating real-time deepfakes) whose use has directly led to harm through digital fraud and deception, fulfilling the criteria for an AI Incident. The harms include violations of trust, financial fraud, and harm to individuals and communities. The article provides data on the increasing number of such incidents, confirming that harm is occurring, not just potential. Therefore, this is classified as an AI Incident.
Thumbnail Image

Deepfake in tempo reale, aumenta il rischio di frodi

2025-05-12
L'opinione delle Libertà
Why's our monitor labelling this an incident or hazard?
The event involves AI systems explicitly used to generate real-time deepfakes that have directly led to fraud and deception harming individuals financially and socially. The article documents actual incidents and increasing trends of such harms, including identity fraud and misinformation, fulfilling the criteria for an AI Incident. The AI system's use in generating manipulated video and audio content that deceives victims and verification systems is central to the harm described.
Thumbnail Image

Trust trap: How deepfake video scams are stealing millions

2025-05-30
The Indian Express
Why's our monitor labelling this an incident or hazard?
The event involves the use of AI systems (deepfake video generation using AI and machine learning) to perpetrate financial scams and identity theft, which are direct harms to individuals' property and personal data. The article provides concrete examples of victims losing money due to these AI-enabled scams, fulfilling the criteria for an AI Incident. The harm is realized, not just potential, and the AI system's role is pivotal in creating convincing fake videos that enable the scams. Therefore, this event qualifies as an AI Incident.
Thumbnail Image

Online trust, offline threat: social media habits aid cyber scams

2025-06-01
VietNamNet News
Why's our monitor labelling this an incident or hazard?
The article explicitly mentions AI systems (deepfake and deepvoice software) being used maliciously to impersonate individuals and authorities, leading to direct financial harm (e.g., a $25 million fraud). This constitutes injury to persons' property (financial loss) and harm to communities through widespread scams. The AI systems' use is central to the harm, fulfilling the criteria for an AI Incident. The article also discusses ongoing harms and responses, but the primary focus is on realized harms caused by AI misuse, not just potential or complementary information.
Thumbnail Image

Are You Talking to a Human? The Dangers of Deepfake Technology

2025-05-30
Lexology
Why's our monitor labelling this an incident or hazard?
The article explicitly mentions the use of deepfake technology, which is an AI system employing deep learning to generate realistic fake images, videos, and audio. The described incidents involve the use of these AI-generated deepfakes to deceive employees and criminals altering ID photos with AI, resulting in financial fraud and harm. This fits the definition of an AI Incident because the AI system's use directly led to harm (financial loss) and violations of rights (fraud). The article also discusses mitigation strategies but the primary focus is on the realized harms caused by AI misuse.
Thumbnail Image

The Legal Accountability of AI-Generated Deepfakes in Election Misinformation

2025-06-01
MarkTechPost
Why's our monitor labelling this an incident or hazard?
The article explicitly discusses AI systems (generative adversarial networks, autoencoders) used to create deepfakes that have been deployed in real-world election misinformation campaigns across multiple countries. These campaigns have caused harm by misleading voters, undermining trust in electoral processes, and potentially influencing election outcomes, which constitutes harm to communities and democratic rights. The harms are realized, not hypothetical, as evidenced by specific examples such as the $6 million fine for a deepfake robocall and viral AI-generated videos affecting elections. The article also covers legal and policy responses, but the primary focus is on the harms caused by AI-generated content. Hence, this is an AI Incident rather than a hazard or complementary information.
Thumbnail Image

Taylor Swift Deepfake Video Viral: What to Learn

2025-06-02
Bangla news
Why's our monitor labelling this an incident or hazard?
The article explicitly states that the deepfake video was created using AI technologies (deep learning, GANs) and that it went viral, misleading millions and causing reputational and emotional harm to Taylor Swift. This meets the definition of an AI Incident because the AI system's use directly led to harm to a person (reputational and emotional harm) and harm to communities (misinformation and public trust erosion). The involvement of AI in the creation and spread of the harmful content is clear, and the harm is realized, not just potential. The article also discusses legal and societal responses, but the primary event is the harmful viral deepfake itself, which is an AI Incident.