AI-Generated Deepfake Scam Impersonates Johor Official on TikTok

The article describes a fake video generated using AI that impersonates a king, which constitutes a misuse of AI technology. The circulation of such content can harm communities by spreading misinformation and potentially violating rights related to reputation and dignity. Since the video has already been circulated and a complaint has been lodged, the harm is realized or ongoing. Therefore, this qualifies as an AI Incident due to the direct involvement of AI-generated content causing harm.

The article explicitly mentions the use of AI to generate a fake image and voice of a public figure to disseminate false information related to a financial aid program, which is intended for fraudulent purposes. This misuse of AI has directly led to harm by enabling a scam, which affects individuals and communities financially and socially. Therefore, this qualifies as an AI Incident due to the realized harm caused by the AI system's use in the scam.

The event explicitly mentions the use of AI to generate a fake video impersonating the King of Malaysia, which is being used to deceive the public. This misuse of AI-generated content directly leads to harm by spreading misinformation and potentially undermining public trust, thus fitting the definition of an AI Incident due to realized harm caused by the AI system's use.

The use of AI to mimic a public official's face and voice to spread false information and attempt to steal personal data directly relates to an AI system's misuse causing harm. The event describes ongoing fraudulent activities leveraging AI-generated content, which is a clear violation of rights and harms communities by enabling scams. Even though no financial losses have been reported yet, the harm is occurring through deception and potential data theft. Therefore, this qualifies as an AI Incident due to realized harm linked to AI misuse.

The event describes the use of AI-generated deepfake videos to impersonate a public figure and deceive the public, which constitutes a violation of rights and potential harm to the community's trust and social order. While no direct harm has been reported yet, the misuse of AI in this manner has already led to a report and police investigation, indicating realized harm in terms of deception and reputational damage. Therefore, this qualifies as an AI Incident due to the direct misuse of AI causing harm through misinformation and impersonation.

The event explicitly mentions AI technology used to impersonate a public official's facial features and voice to deceive the public. The AI system's use directly led to the creation of fraudulent accounts spreading false information and phishing for personal data, which constitutes harm to individuals' rights and potential financial harm. The police have taken enforcement action, but the harm is realized or imminent due to the phishing attempt. Hence, it meets the criteria for an AI Incident rather than a hazard or complementary information.

The article explicitly mentions the use of AI deepfake technology to impersonate officials and deceive the public, which is a direct misuse of AI systems. The harm involves identity theft and exploitation of personal data, which are violations of rights and harm to individuals and communities. Even though no confirmed financial losses have been reported yet, the ongoing scam activities and the involvement of organized crime using AI deepfakes to exploit people meet the criteria for an AI Incident. The AI system's malicious use has directly led to harm or attempted harm, fulfilling the definition of an AI Incident.

The event explicitly involves an AI system generating fake video content used to deceive the public, which constitutes direct use of AI leading to harm in the form of misinformation and reputational damage. Although no financial or physical harm has occurred, the dissemination of false information about a public figure and the potential to mislead the public is a clear harm to communities and a violation of legal frameworks. Therefore, this qualifies as an AI Incident rather than a hazard or complementary information, as harm is already occurring and authorities are responding to it.

The event describes AI-generated videos used to impersonate a public figure and deceive viewers, which is a misuse of AI technology. Although the content aims to deceive and tarnish the image of the royal institution, no actual harm such as financial loss or injury has been reported. The police and regulatory authorities have intervened to remove the content and investigate, indicating the harm is potential rather than realized. Therefore, this qualifies as an AI Hazard because the AI system's use could plausibly lead to harm (e.g., misinformation, reputational damage) but no direct or indirect harm has yet materialized.

The event explicitly mentions the use of AI-generated images and voice to impersonate a Johor state executive councillor in a scam. The AI system's outputs are used to deceive people into providing personal information, which is a violation of privacy and potentially leads to financial harm. The scam is active, with multiple TikTok accounts involved, and investigations are underway. The harm is direct and ongoing, meeting the criteria for an AI Incident as the AI system's use has directly led to harm through fraud and privacy violations.

The event explicitly mentions the use of AI to create deepfake content for fraudulent purposes, which is a clear example of AI system use leading to harm through identity theft and scams. The harm involves violations of personal data privacy and potential financial harm to victims, fitting the definition of an AI Incident. The scam is ongoing, and the AI system's role is pivotal in enabling the impersonation and deception. Therefore, this event qualifies as an AI Incident rather than a hazard or complementary information.

The event explicitly involves an AI system generating deceptive content (an AI-generated video impersonating the King). The use of AI to create and spread false information that could mislead the public and tarnish the image of a public figure constitutes harm to communities and a violation of rights. Although no direct financial harm or victims have been reported yet, the dissemination of false information with intent to deceive is a realized harm under the framework. The police and regulatory authorities' involvement and content removal confirm the seriousness of the incident. Therefore, this event meets the criteria for an AI Incident rather than a hazard or complementary information.

The event explicitly mentions the use of AI to mimic the face and voice of a public figure for fraudulent purposes, which constitutes the use of an AI system. The AI system's use has directly led to harm through fraud (financial harm to individuals) and misinformation (harm to community trust). Therefore, this qualifies as an AI Incident under the definitions provided, as the AI system's use has directly led to violations including harm to people and communities.

The event explicitly mentions the use of AI technology to generate fake video content that deceives the public, which is a direct misuse of an AI system. The harm includes misinformation and reputational damage to a key institution, which falls under harm to communities and violation of rights. The investigation and takedown actions are responses to this incident, not the main focus, so this is not merely complementary information. Since harm is occurring due to the AI-generated content, this qualifies as an AI Incident.

An AI system was used to generate a fake video (deepfake) of the Malaysian King, which is a clear example of AI-generated content causing misinformation. The video aims to deceive viewers by falsely announcing financial aid, which could plausibly lead to harm such as public confusion, reputational damage to the monarchy, or financial scams if exploited further. Since no actual harm has been reported yet but the potential for harm is credible and recognized by authorities, this event qualifies as an AI Hazard rather than an AI Incident. The police and regulatory response further supports the recognition of plausible future harm.

The event explicitly mentions the use of AI technology to imitate the face and voice of a public official, which is a clear AI system involvement. The misuse of this AI system has directly led to fraudulent activities and attempts to steal personal and financial information, causing harm to individuals and communities. Therefore, this qualifies as an AI Incident due to realized harm stemming from the malicious use of AI-generated deepfakes for scams.

The event explicitly mentions the use of AI to generate fake faces and voices to impersonate a public official for fraudulent purposes. This misuse of AI has directly led to the dissemination of false information and attempts to deceive the public, which fits the definition of an AI Incident due to the direct or indirect harm caused or intended (fraud, violation of privacy, potential financial harm). The presence of AI systems is clear, the harm is linked to their use, and the event describes realized harm in the form of deception and risk to personal data, even if financial loss has not yet been reported.

The event explicitly mentions the use of AI to generate videos that impersonate a public figure's voice to promote false financial aid, which is a direct misuse of AI technology causing harm by facilitating scams and misinformation. This meets the criteria for an AI Incident because the AI system's use has directly led to harm through deception and potential financial loss to individuals. The harm is realized as the fake videos are actively being spread and used by fraud syndicates, fulfilling the definition of an AI Incident.

An AI system is explicitly involved as the fake accounts use AI technology to generate the voice of the Agong. The use of this AI-generated voice for scams constitutes misuse of AI leading to harm (fraud and deception) to people, which fits the definition of an AI Incident. The harm is realized or ongoing as scammers are actively using these accounts to deceive the public. Therefore, this event qualifies as an AI Incident due to direct harm caused by AI misuse.

The event involves the use of an AI system (voice synthesis technology) to create fake videos impersonating a public figure, which are used by scammers. While the article does not report actual harm realized yet, the plausible risk of fraud and deception caused by these AI-generated deepfakes constitutes a potential harm. Therefore, this situation qualifies as an AI Hazard because the AI system's use could plausibly lead to an AI Incident involving harm to people through scams.

The event explicitly mentions the use of AI to manipulate images and videos to deceive people, which fits the definition of an AI system's use leading to potential harm. However, since no victims or financial losses have been identified yet, the harm is plausible but not realized. The police investigation and content removal efforts indicate recognition of the risk. Therefore, this situation is an AI Hazard because the AI-generated content could plausibly lead to harm such as fraud or reputational damage if left unchecked.

The event explicitly mentions AI-generated video content used to deceive people and damage the image of a royal institution, which constitutes harm to communities and a violation of rights. The AI system's use in generating and spreading false information directly leads to potential fraud and reputational harm. Although no financial loss has been reported yet, the nature of the content and its intent to deceive qualifies this as an AI Incident under the framework, as harm is occurring or imminent due to the AI system's misuse.

An AI system is explicitly involved as the misuse of AI technology to generate a deceptive video is reported. The harm includes fraud (attempt to deceive the public) and reputational damage to the royal institution, which qualifies as harm to communities and a violation of rights. Since the harm is occurring (fraudulent content being distributed and investigated), this constitutes an AI Incident rather than a hazard or complementary information.