Deepfake Videos Used in Global Phishing Scam

The report describes scammers employing generative AI to produce deepfake videos of celebrities and false news anchors, along with automated creation of over 1,000 phishing domains daily, to trick users into downloading a fraudulent app. This use of AI has directly led to financial losses (harm to property) and breaches of trust. As the harm is occurring, it qualifies as an AI Incident.

Deepfake generation is an AI system producing manipulated video content. Scammers’ use of these AI-generated videos to defraud users constitutes direct misuse of AI, causing real financial harm to victims, meeting the criteria for an AI Incident.

The event involves the use of AI (deepfake generation) to create fraudulent endorsement videos that directly caused harm (financial losses to 25,000–50,000 victims). This is a realized AI-driven incident, not merely a potential risk or policy update, and so qualifies as an AI Incident.

The event describes the actual misuse of AI deepfake technology by scammers, causing real financial harm to victims through deceptive investment schemes. The AI system’s use in generating and deploying these counterfeit endorsements is central to the fraud, constituting an AI Incident.

The event describes the active use of generative AI (deepfake videos) to perpetrate phishing scams, resulting in direct financial harm to users who are misled into downloading fraudulent apps. This constitutes an AI Incident because the AI system’s outputs (deepfake videos) are central to the realized harm (fraud and financial loss).

The article describes real, ongoing harm caused by the use of AI systems (deepfake generation) to commit fraud and phishing attacks. These attacks involve registration of over 1,000 phishing domains daily and use manipulated video content to deceive victims, leading to direct financial harm. Therefore, this qualifies as an AI Incident.

The article details how AI-generated deepfake videos are actively used in scams that deceive users into financial loss, which is a clear harm to individuals and communities. The AI system's use in creating these deepfakes is directly linked to the harm. Although the article also discusses the AI detection tool developed to counteract these scams, the primary focus is on the harm caused by the AI-generated deepfakes. Hence, this qualifies as an AI Incident because the AI system's use has directly led to harm.

The event involves the use of AI systems (deepfake technology) to create manipulated videos and images that are used maliciously to deceive users into investing money in a scam. This misuse of AI has directly led to financial harm to individuals, fulfilling the criteria for an AI Incident. The harm is realized and ongoing, as users are losing money and being misled by AI-generated content. Therefore, this is classified as an AI Incident rather than a hazard or complementary information.

The event involves AI systems explicitly through the use of deepfake technology to create realistic fake videos of celebrities endorsing a fraudulent app. These videos are used to deceive users into downloading a scam app and making payments, causing direct financial harm to victims. The AI system's use in generating these videos is pivotal to the scam's success, fulfilling the criteria for an AI Incident as it has directly led to harm to people (financial injury) and harm to communities (widespread fraud).

The event involves the use of AI systems (deepfake generation technology) to create manipulated videos that are used maliciously to deceive people into downloading fraudulent apps and losing money. The phishing domains and fake app stores further support the scam operation. The harm is realized and ongoing, as users are being cheated financially. Therefore, this qualifies as an AI Incident because the AI system's use directly leads to harm (financial loss and deception) to people across multiple countries.

The article explicitly mentions the use of deepfake AI technology to create manipulated videos that deceive users into downloading dubious apps, supported by over 1,000 phishing domains daily. The harm is direct and realized, as users are targeted for financial scams, fulfilling the criteria of injury or harm to people (financial harm) and harm to communities through deceptive practices. The AI system's use is central to the incident, making this an AI Incident rather than a hazard or complementary information.

The event describes the malicious use of AI (deepfake video generation) by cybercriminals to harm or deceive people. This is an actual occurrence of harm caused by an AI system, fitting the definition of an AI Incident.

Deepfake video generation is a misuse of generative AI that directly harms celebrities’ privacy, reputation and rights through deceptive content. Since the deepfakes have been produced and disseminated, causing realized harm, this qualifies as an AI Incident.

The article explicitly mentions the use of AI (artificial intelligence) to create deepfake videos, which are manipulated media generated by AI systems. These videos are used to deceive users into downloading a suspicious gaming app and investing money with false promises of financial rewards. This constitutes a direct harm to people (financial harm and deception) and harm to communities (spread of misinformation and fraud). Therefore, this event qualifies as an AI Incident because the AI system's use has directly led to realized harm through fraudulent activities and misinformation campaigns.

The article explicitly mentions the use of AI (artificial intelligence) to create deepfake videos, which are manipulated media generated by AI systems. These videos are used in a fraudulent campaign to deceive users into downloading a suspicious gaming app, promising financial rewards that are likely false. This misuse of AI directly causes harm to people by facilitating fraud and deception, impacting their financial well-being and trust. Hence, the event meets the criteria for an AI Incident due to realized harm caused by AI-generated content.

The event involves AI systems used to generate deepfake videos without consent, directly causing harm to individuals (psychological trauma, violation of privacy and rights) and communities (social trust breakdown). The AI system's use in creating and distributing non-consensual explicit content constitutes an AI Incident as it has directly led to significant harm to persons and violation of rights. The article also mentions law enforcement and legislative actions as responses, but the primary focus is on the realized harm caused by AI-generated deepfakes.