Deepfake Celebrities Fuel 'Subway Money' Scam

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

Criminals used AI-powered deepfake videos of celebrities, including Neymar and Anitta, across Instagram, Facebook and TikTok to promote a fake 'Subway Money' game. Users were tricked into creating accounts and making PIX payments for promised rewards, resulting in financial losses even as Meta and TikTok prohibit such deceptive content.[AI generated]

Why's our monitor labelling this an incident or hazard?

The event involves the use of AI technology (deepfake) to create manipulated videos that mislead victims into making payments for a fraudulent game. The AI system's use directly leads to harm by enabling financial scams causing monetary loss to individuals. Therefore, this qualifies as an AI Incident due to realized harm (financial fraud) facilitated by AI-generated content.[AI generated]
AI principles
Privacy & data governanceTransparency & explainabilityAccountabilityRobustness & digital securityRespect of human rightsSafety

Industries
Media, social platforms, and marketingFinancial and insurance servicesDigital security

Affected stakeholders
Consumers

Harm types
Economic/PropertyReputationalHuman or fundamental rightsPsychological

Severity
AI incident

Business function:
Marketing and advertisement

AI system task:
Content generation


Articles about this incident or hazard

Thumbnail Image

Subway Money: entenda golpe que usa deepfake de César Tralli e Anitta

2024-01-15
Tecnologia
Why's our monitor labelling this an incident or hazard?
The event involves the use of AI technology (deepfake) to create manipulated videos that mislead victims into making payments for a fraudulent game. The AI system's use directly leads to harm by enabling financial scams causing monetary loss to individuals. Therefore, this qualifies as an AI Incident due to realized harm (financial fraud) facilitated by AI-generated content.
Thumbnail Image

'Subway Money': golpistas criam deepfakes de famosos para prometer dinheiro em jogo falso - Alagoas 24 Horas: Líder em Notícias On-line de Alagoas

2024-01-14
Alagoas 24 Horas: Líder em Notícias On-line de Alagoas
Why's our monitor labelling this an incident or hazard?
Deepfake technology is an AI system that generates realistic fake videos. Here, it is used maliciously to impersonate celebrities and promote a fraudulent gambling game, leading to financial harm to users who are deceived. The harm is realized as users are tricked into depositing money into a fake game. The involvement of AI in creating the deceptive content directly leads to harm, fulfilling the criteria for an AI Incident.
Thumbnail Image

Neymar e Anitta têm imagem usada em golpe do Subway Money - 15/01/2024 - Tec - Folha

2024-01-15
Folha de S.Paulo
Why's our monitor labelling this an incident or hazard?
The event involves the use of AI systems (deepfake generation) in the malicious use phase to create realistic fake videos of celebrities endorsing a fraudulent scheme. This directly leads to harm (financial loss and data theft) to users, fulfilling the criteria for an AI Incident. The AI system's role is pivotal in enabling the deception and the scam's effectiveness. Therefore, this is classified as an AI Incident.
Thumbnail Image

Subway Money | Jogo de azar usa deepfakes de famosos para promovê-lo

2024-01-15
Tecnologia
Why's our monitor labelling this an incident or hazard?
The event involves AI systems explicitly through the use of deepfake technology to create manipulated videos of celebrities promoting an illegal gambling platform. This use of AI directly leads to harm by deceiving users, causing financial losses, and spreading false information about the legality and benefits of the game. The harms include violation of rights (fraud, misinformation) and harm to communities (misleading many users). The AI system's role is pivotal in enabling the deceptive advertising and consequent harm. Hence, it meets the criteria for an AI Incident rather than a hazard or complementary information.
Thumbnail Image

Subway Money | Jogo de azar usa deepfakes de famosos para promovê-lo

2024-01-15
Terra
Why's our monitor labelling this an incident or hazard?
The article explicitly mentions the use of AI-generated deepfake videos to promote an illegal gambling platform that causes financial harm to users and violates legal and ethical rights. The deepfakes are central to the fraudulent promotion and deception, directly contributing to harm. This fits the definition of an AI Incident because the AI system's use has directly led to violations of rights and harm to communities through misinformation and illegal gambling practices.
Thumbnail Image

Subway Money | Jogo de azar usa deepfakes de famosos para promovê-lo

2024-01-15
Canaltech
Why's our monitor labelling this an incident or hazard?
The article explicitly describes the use of AI deepfake technology to create manipulated videos of famous people promoting an illegal gambling platform. This use of AI directly leads to harm by deceiving users into participating in a game that is illegal and has poor payout practices, causing financial harm and violating consumer rights. The AI system's role is pivotal in creating convincing fake endorsements that facilitate this harm. Therefore, this qualifies as an AI Incident due to realized harm caused by the AI system's malicious use.
Thumbnail Image

Subway Pay paga mesmo? Jogo que promete dinheiro é promovido até por vídeos fakes com IA

2024-01-12
TecMundo
Why's our monitor labelling this an incident or hazard?
The event involves an AI system in the form of AI-generated fake videos used to promote a fraudulent gambling game that causes financial harm to users. The harm is realized as players lose money and are unable to withdraw winnings, constituting injury or harm to persons. The AI's role in generating deceptive promotional content is pivotal in enabling the scam. Hence, it meets the criteria for an AI Incident rather than a hazard or complementary information.
Thumbnail Image

'Subway Money': golpe usa deep fake de celebridades para enganar usuários

2024-01-15
Estadão
Why's our monitor labelling this an incident or hazard?
The event involves AI systems explicitly (deepfake technology) used to create manipulated videos and voices of celebrities to promote a scam game. This use of AI has directly led to harm (financial loss and deception of users), fulfilling the criteria for an AI Incident. The harm is realized, not just potential, as users report losing money and being unable to withdraw promised rewards. Therefore, this is an AI Incident due to direct harm caused by malicious use of AI-generated content.
Thumbnail Image

Golpistas usam imagem de Neymar, Anitta e famosos para divulgar jogo falso Subway Money

2024-01-16
Notícias ao Minuto Brasil
Why's our monitor labelling this an incident or hazard?
The event involves the use of AI systems (deepfake technology) to create fraudulent videos impersonating celebrities, which directly leads to financial harm to users who are scammed into paying money and sharing personal data. The AI system's use is central to the deception and harm, fulfilling the criteria for an AI Incident. The harm includes financial loss and violation of personal data rights, which are significant harms caused by the AI system's malicious use.
Thumbnail Image

Subway Money: golpistas criam deepfakes de famosos em jogo falso

2024-01-14
O Povo
Why's our monitor labelling this an incident or hazard?
The use of AI deepfake technology to create manipulated videos of famous people directly leads to harm by enabling a scam that causes financial loss to victims. This constitutes an AI Incident because the AI system's use is central to the harm occurring (fraud and financial harm).
Thumbnail Image

'Subway Money': golpistas criam deepfakes de famosos para prometer dinheiro em jogo falso

2024-01-14
Jornal Floripa - Notícias de Florianópolis - Santa Catarina Brasil
Why's our monitor labelling this an incident or hazard?
The event involves the use of AI technology (deepfake generation) to create manipulated videos that mislead users into participating in a scam involving fake games and monetary transfers. This use of AI directly leads to harm to individuals (financial loss) and violates rights by deception. The harm is realized, not just potential, as victims are induced to transfer money. Therefore, this qualifies as an AI Incident under the framework, as the AI system's use has directly led to harm to people.