Google Gemini's Hyperrealistic Avatar Feature Raises Deepfake Concerns

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

Google has launched a Gemini AI feature allowing users to create hyperrealistic video avatars of themselves using the Omni model. Despite safeguards like SynthID watermarks and age restrictions, experts and users express concerns about potential misuse for deepfakes, misinformation, and identity abuse. No actual harm has been reported yet.[AI generated]

Why's our monitor labelling this an incident or hazard?

The article focuses on the introduction and wider availability of an AI system capable of generating realistic digital avatars. Although it acknowledges potential misuse risks, no direct or indirect harm has been reported or occurred yet. The safeguards indicate awareness of these risks but do not eliminate the plausible future harm. Since the event describes a new AI capability with potential for harm but no realized harm, it fits the definition of an AI Hazard rather than an Incident or Complementary Information.[AI generated]
AI principles
Privacy & data governanceSafety

Industries
Media, social platforms, and marketingDigital security

Affected stakeholders
ConsumersGeneral public

Harm types
ReputationalHuman or fundamental rightsPublic interest

Business function:
Other

AI system task:
Content generation


Articles about this incident or hazard