Canva AI Tool Replaces 'Palestine' with 'Ukraine' in User Designs, Prompting Apology

The event involves an AI system (Magic Layers) whose malfunction caused biased and inappropriate content alteration, directly impacting users and communities by misrepresenting politically sensitive terms. This constitutes harm to communities and a violation of rights related to accurate information and representation. The harm is realized, not just potential, as users experienced the biased output. Therefore, this qualifies as an AI Incident rather than a hazard or complementary information.

The Magic Layers feature is an AI-powered tool designed to edit images by breaking them into components. Its automatic replacement of "Palestine" with "Ukraine" constitutes an AI system malfunction or biased behavior that directly led to harm by censoring or altering user content related to a politically sensitive topic. This manipulation affects communities and potentially violates rights to free expression, fitting the definition of an AI Incident. Although Canva apologized and fixed the issue, the harm occurred and is materialized. Therefore, this event qualifies as an AI Incident rather than a hazard or complementary information.

The article explicitly mentions generative AI systems being used to create deepfake videos that propagate false atrocity claims, which is a direct use of AI causing harm to communities through misinformation and propaganda. The Canva AI feature malfunction that altered user content without permission also represents AI misuse leading to misinformation and potential censorship, further contributing to harm. These harms align with violations of rights and harm to communities as defined in the framework. Since the harms are realized and linked to AI system use and malfunction, the event is classified as an AI Incident.

The AI system (Magic Layers) is explicitly involved as it automatically altered user-generated content without permission, specifically replacing "Palestine" with "Ukraine." This constitutes misuse or malfunction of the AI system's output generation. The harm includes violation of user rights (alteration without consent) and harm to communities (biased geopolitical representation). The harm is realized and not merely potential. Canva's ongoing audit and apology are responses but do not negate the incident classification. Hence, the event meets the criteria for an AI Incident.

The Magic Layers feature is an AI tool designed to decompose images into editable layers, and it malfunctioned by altering specific text content without user consent. This malfunction directly led to harm in the form of distress and potential violation of users' rights to free expression and accurate representation, which can be considered harm to communities or a violation of rights. Since the AI system's malfunction caused realized harm and the company has responded with remediation, this qualifies as an AI Incident rather than a hazard or complementary information.

The AI system (Magic Layers) is explicitly mentioned and is responsible for changing text in user designs without instruction, which is a malfunction. This alteration of the word "Palestine" to "Ukraine" can be seen as a form of censorship or bias, impacting users' rights and causing distress, thus harm to communities. The company confirmed the issue and took remedial action, confirming the AI system's role in causing harm. Therefore, this event meets the criteria for an AI Incident due to the direct harm caused by the AI system's malfunction and biased behavior.

The Magic Layers feature is explicitly described as an AI system that manipulates visual designs by interpreting and separating image elements. The malfunction—swapping "Palestine" with "Ukraine"—is a direct error caused by the AI system's processing. This leads to harm in the form of misinformation and potential distress to affected communities, fulfilling the criteria for harm to communities under the AI Incident definition. Canva's apology and efforts to fix the issue confirm the AI system's role in causing the harm.

The AI system's malfunction directly led to the alteration of user content in a way that misrepresented political terms, which can be considered a violation of rights and harm to communities. The incident involves the use and malfunction of an AI system affecting user-generated content, fulfilling the criteria for an AI Incident. The company's response to fix the issue does not negate the fact that harm occurred due to the AI's malfunction.

The AI system's use led to direct harm by censoring legitimate speech related to political advocacy, which is a violation of human rights (freedom of expression). The harm is realized, not just potential, as users experienced the removal or alteration of the word "Palestine" in their designs. The incident is directly linked to the AI system's content moderation malfunction. Therefore, this qualifies as an AI Incident under the framework, specifically under violations of human rights.