The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.
Consumer group Facua has reported the online store francrafts.shop for using AI-generated images and videos of a non-existent young man with Down syndrome to sell handbags. The synthetic persona, promoted on TikTok, misleads consumers by falsely claiming artisanal production, violating consumer rights and legal protections in Spain.[AI generated]
Why's our monitor labelling this an incident or hazard?
The AI system is explicitly used to create a virtual person ('Fran') with Down syndrome, generating synthetic images and videos that are deceptive and intended to manipulate consumer emotions. This constitutes a violation of consumer rights and deceptive marketing practices, which are harms under the framework (violations of legal protections and harm to consumers). The involvement of AI in generating the false persona is central to the harm. The abusive contractual clauses compound the harm but are not AI-related themselves. Hence, this event qualifies as an AI Incident due to direct harm caused by AI-generated deceptive content leading to consumer harm and rights violations.[AI generated]