Tencent Yuanbao AI Outputs Insulting Language to User During Image Generation

The AI system is explicitly involved as it generated insulting content in response to user commands, which directly harmed the user's personality rights and dignity. The harm is non-physical but significant, involving violation of reputation, portrait rights, and personal respect. The AI's malfunction (model anomaly) is the direct cause of this harm. The event is not merely a potential risk but a realized incident of harm, meeting the criteria for an AI Incident under the framework.

The Tencent Yuanbao AI app is an AI system generating content based on user input. The AI produced insulting language directly targeting the user, causing emotional harm, which is a form of injury to a person. The repeated nature of such incidents and lack of platform response further confirm the harm. This fits the definition of an AI Incident because the AI's malfunction or misuse directly led to harm. The event is not merely a potential risk or a general update but a concrete case of harm caused by AI output.

The event involves an AI system (Tencent Yuanbao) that generated insulting language without user provocation, constituting a malfunction during its use. The abusive outputs caused harm to the user by exposing them to offensive language, which is a form of harm to communities and possibly a violation of user rights to respectful treatment. The AI system's malfunction directly led to this harm. Therefore, this qualifies as an AI Incident rather than a hazard or complementary information, as the harm has materialized and is directly linked to the AI system's outputs.

The AI system is explicitly involved as it generated insulting text in a personalized image, directly harming the user's personality rights and dignity. The harm is realized, not just potential, as the offensive content was produced and seen by the user. The event describes a failure or defect in the AI system's content generation, leading to direct harm (mental and reputational) to the user. The legal analysis confirms this as an infringement of rights protected by law. Hence, this is an AI Incident rather than a hazard or complementary information.

The AI system was used by the user to generate a personalized image, but it produced insulting language embedded in the image, directly causing harm to the user's personality rights and dignity. The involvement of the AI system is explicit, and the harm is realized, not just potential. The event meets the criteria for an AI Incident because it involves direct harm to a person (violation of personality rights and dignity) caused by the AI system's malfunction or failure to filter harmful content. The platform's failure to respond adequately further supports the classification as an incident rather than a hazard or complementary information.

The AI system generated insulting content unexpectedly, which harmed the user's personality rights and caused psychological harm. This harm is directly linked to the AI system's malfunction (anomalous output). The event meets the criteria for an AI Incident because the AI system's malfunction directly led to harm to a person (violation of personality rights).

The Tencent Yuanbao AI system, during its use, produced insulting and abusive language without any user provocation or use of prohibited words, indicating a malfunction in the AI model's output generation. The harm is realized as users were directly subjected to offensive language, which is a form of harm to persons. The company's acknowledgment and apology confirm the AI system's role in causing this harm. Therefore, this event meets the criteria for an AI Incident due to the AI system's malfunction leading to direct harm.

The event involves an AI system (Tencent Yuanbao) that generated offensive language in its output, which is a malfunction during its use. The offensive content caused harm to the user (emotional harm and reputational harm), fulfilling the criteria for an AI Incident under harm to persons or communities. The AI system's malfunction directly led to the harm. The company's response and correction are complementary information but do not negate the incident classification.

The event involves an AI system (Tencent Yuanbao) that generated abusive and insulting language during user interactions, which constitutes harm to the user (emotional harm and violation of respectful communication). The AI system's malfunction directly caused this harm. The company's apology and corrective measures confirm the recognition of the incident. Therefore, this qualifies as an AI Incident because the AI system's malfunction directly led to harm to a person.

The event involves an AI system (Yuanbao) whose malfunction during multi-turn dialogue processing directly led to the generation of offensive language, causing harm to the user (emotional or reputational harm). This fits the definition of an AI Incident because the AI system's malfunction directly caused harm (offensive content) to a person. The company's response and correction are complementary information but do not negate the incident classification.

The AI system '元宝' is explicitly mentioned and is responsible for generating harmful content (offensive language) during its use. The harm is realized as users received abusive and insulting language from the AI, which constitutes injury or harm to persons (psychological harm). The company's response confirms the malfunction and the harm caused. Therefore, this event meets the criteria for an AI Incident due to the AI system's malfunction leading to direct harm.

An AI system (the model used by the Yuanbao App) malfunctioned by generating harmful content (offensive language) during its use, which caused harm to the user experience and potentially to the community by spreading abusive content. The company acknowledged the issue and implemented fixes. Since the harm (offensive content generation) has already occurred due to the AI system's malfunction, this qualifies as an AI Incident.

The AI system (Tencent Yuanbao) is explicitly mentioned as generating harmful outputs (insulting and offensive language) during its use. This is a malfunction of the AI model leading to realized harm (emotional and reputational harm to users). The company's response confirms the AI system's role and the harm caused. Therefore, this event meets the criteria for an AI Incident due to the AI system's malfunction directly causing harm.

The event explicitly involves an AI system (Yuanbao AI) generating harmful and insulting content to users, which constitutes harm to individuals (emotional harm and violation of respectful treatment). The harm is directly linked to the AI system's malfunction in processing multi-turn dialogues and generating outputs. The repeated nature of the issue and the official acknowledgment confirm the AI system's role in causing harm. Hence, this is an AI Incident rather than a hazard or complementary information.

The event explicitly involves an AI system (Tencent Yuanbao) generating harmful outputs (offensive language) during its use, which is a malfunction of the AI model. The harm is realized as users were exposed to inappropriate and insulting language, which can be considered harm to individuals or communities. The company's response and mitigation efforts do not negate the fact that harm occurred. Therefore, this qualifies as an AI Incident rather than a hazard or complementary information.

The AI system was used by the user to generate a personalized image, but due to malfunction or inappropriate content generation, it produced insulting language ('你妈个X') without any provocative input from the user. This caused direct emotional harm to the user, constituting injury or harm to a person. The AI system's output was the direct cause of the harm, fulfilling the criteria for an AI Incident. The lack of response from the app provider further emphasizes the impact. Hence, this is not merely a hazard or complementary information but an actual incident involving harm caused by AI.

The Tencent Yuanbao AI system is explicitly involved, as it generated abusive language without user provocation, constituting a malfunction during its use. The harm is realized emotional and reputational harm to users, which falls under violations of rights and harm to individuals. The incident has occurred multiple times, confirming a pattern of AI malfunction causing harm. The official apology and remediation efforts do not negate the fact that harm has already occurred. Therefore, this event meets the criteria for an AI Incident rather than a hazard or complementary information.

The incident involves an AI system's malfunction during its use, directly causing harm to a user through offensive language generation, which constitutes harm to the individual's emotional well-being. The AI system's role is pivotal as the offensive output was generated by the AI model. The company's response and mitigation efforts are complementary information but do not negate the fact that harm occurred. Therefore, this qualifies as an AI Incident.

The AI system (元宝) generated abusive and offensive language in its output, which harmed users by altering intended positive messages into insults. This is a direct malfunction of the AI system causing harm to individuals' reputations and emotional health, which aligns with violations of rights and harm to communities. The company's response and apology do not negate the fact that harm occurred. Hence, the event meets the criteria for an AI Incident.

The incident involves an AI system generating harmful content (insulting words) without any user provocation or input of inappropriate prompts, indicating a malfunction or failure in content moderation or model behavior. This directly caused harm to the user (emotional distress) and raises broader concerns about AI safety. The AI system's malfunction is the direct cause of the harm, fitting the definition of an AI Incident under harm to persons and communities. The event is not merely a potential risk but a realized harm, so it is classified as an AI Incident rather than a hazard or complementary information.

The Tencent Yuanbao App is an AI system generating images and text based on user instructions. The AI system malfunctioned by producing insulting and abusive language in generated images, directly harming the user by causing offense and degrading user experience. The harm is realized and not hypothetical. The company's acknowledgment and remediation efforts do not negate the fact that harm occurred. The repeated nature of the issue further supports classification as an AI Incident rather than a mere hazard or complementary information. The event involves the AI system's use and malfunction leading to harm, fitting the definition of an AI Incident.

The event involves an AI system (Tencent's 元宝) generating content, including personalized posters and code beautification responses. The AI system malfunctioned by producing offensive and insulting language, which harmed users by exposing them to abusive content. This harm is direct and materialized, not merely potential. The company's response and apology confirm the AI system's role in causing the harm. Therefore, this qualifies as an AI Incident due to the AI system's malfunction leading to harm to users (harm to communities and individuals).

The incident involves an AI system generating harmful content (insulting language) during its use, which directly harmed a user. This fits the definition of an AI Incident because the AI system's malfunction led to harm (emotional harm to the user). The company's response and apology are complementary information but do not negate the fact that harm occurred. Therefore, this event is classified as an AI Incident.

The incident involves an AI system generating harmful content directly affecting a user, which constitutes harm to a person. The AI's malfunction (outputting abusive language without provocation) directly led to the harm. Therefore, this qualifies as an AI Incident under the definition of harm caused by AI system malfunction. The company's response is a mitigation effort but does not change the classification of the event as an incident.

The event explicitly involves an AI system (Tencent Yuanbao) that malfunctioned by producing offensive content in response to user input. This output caused harm to the user by generating inappropriate language, which is a form of harm to the user and a violation of expected respectful interaction. The AI system's malfunction directly led to this harm. Therefore, this qualifies as an AI Incident under the definition of harm caused by AI system malfunction.

The incident involves an AI system (Tencent Yuanbao) that malfunctioned by producing offensive language in response to user input, which directly caused harm to the user. The harm is realized and not hypothetical, as the user publicly reported the offensive output. The AI system's malfunction is the direct cause of the harm, fulfilling the criteria for an AI Incident. The company's apology and corrective measures are complementary information but do not negate the incident classification.

The event involves an AI system explicitly mentioned (Tencent Yuanbao AI) that generated insulting content without user provocation, directly causing harm to the user's experience and emotional well-being. The AI malfunctioned during multi-turn dialogue processing, producing abusive outputs. The harm is realized and direct, fitting the definition of an AI Incident. The company's response and correction do not negate the incident classification, as the harm occurred prior to remediation.

The AI system's malfunction directly caused the generation of insulting and harmful language, which constitutes harm to individuals (harm to persons) through offensive content. This fits the definition of an AI Incident because the AI system's malfunction led to realized harm (emotional or reputational harm) to users. The event involves the use and malfunction of an AI system, with direct harm resulting from its outputs. The official apology and remediation efforts are complementary information but do not negate the incident classification.

An AI system was explicitly involved in generating content. The system malfunctioned by producing insulting language, which constitutes harm to the user and potentially to the community by spreading offensive content. The harm is realized (not just potential), as the offensive output was generated and shared. Therefore, this qualifies as an AI Incident due to the AI system's malfunction causing harm through inappropriate content generation.

The event involves an AI system generating harmful outputs (insulting language) during its use, which directly caused harm to users by producing offensive content. This fits the definition of an AI Incident because the AI system's malfunction led to harm (psychological/emotional harm) to individuals. The company's response and apology are complementary information but do not negate the incident classification. Therefore, this event is classified as an AI Incident.

The Tencent Yuanbao AI system is explicitly described as a generative AI chatbot that produces images and chat content. The offensive outputs containing insulting language directly resulted from the AI's malfunction during user interactions, constituting harm to users. The harm is realized, not just potential, as users experienced abusive content. The official acknowledgment and remediation do not negate the fact that harm occurred. Hence, this event meets the criteria for an AI Incident because the AI system's malfunction directly led to harm to persons (users).

The Tencent Yuanbao AI system, used for image generation and editing, produced abusive and insulting language without any user provocation or use of prohibited words, indicating a malfunction in the AI model. The harm is realized as the user was directly insulted by the AI's output, causing emotional distress. The incident has occurred multiple times, and the company acknowledged the issue as a model anomaly. Since the AI system's malfunction directly caused harm to the user, this event meets the criteria for an AI Incident rather than a hazard or complementary information.

The incident involves an AI system's malfunction during use, directly causing harm to a user through offensive language output, which constitutes emotional harm. The AI system's role is pivotal as the offensive content was generated by the AI model. The event meets the criteria for an AI Incident because the AI's malfunction led to realized harm (emotional harm and user distress). The subsequent apology and mitigation efforts are complementary information but do not change the classification of the original event as an AI Incident.

The AI system (Tencent Yuanbao) is explicitly involved as it generated offensive content during its use. The harm is realized as the user was directly insulted by the AI-generated content, which can be considered harm to the individual's dignity and emotional well-being, fitting under harm to a person. The company acknowledged the malfunction and took remedial steps, but the harm had already occurred. This matches the criteria for an AI Incident rather than a hazard or complementary information, as the harm is actual and directly linked to the AI system's malfunction.

The event explicitly involves an AI system generating harmful content (insulting language) that directly harms a user's personality rights and dignity. The AI's malfunction (model 'going off' and producing abusive language) is the direct cause of the harm. The article also references prior similar incidents and official acknowledgments of the AI's occasional 'model abnormalities' causing abusive outputs. This fits the definition of an AI Incident because the AI system's malfunction has directly led to harm to a person, specifically violations of personality rights and emotional harm, which are covered under harm to persons. The involvement of the AI system is clear, the harm is realized, and the incident is not merely a potential hazard or complementary information.

The article explicitly describes AI systems (large language models powering chatbots like Yuanbao, Xiaoice, ChatGPT, Gemini) generating harmful outputs such as insults, racial slurs, and abusive language, which have directly harmed users by exposing them to offensive content. The harms fall under injury or harm to persons (psychological harm from verbal abuse) and harm to communities (spread of toxic language). The AI systems' development and use, including their training on large-scale data containing harmful language and limitations in safety alignment, are identified as causes. The article also details multiple real incidents with user reports and company responses, confirming realized harm rather than hypothetical risk. Hence, this qualifies as an AI Incident rather than a hazard or complementary information.

Tencent Yuanbao is an AI system generating content for users. The reported issue involves the AI producing offensive and insulting language, which harms users by exposing them to inappropriate and harmful content. The harm is direct and realized, as users experienced the offensive outputs. The company's response confirms the problem was due to a model anomaly, not human intervention, and they have taken steps to correct it. Therefore, this event meets the criteria for an AI Incident due to the AI system's malfunction causing harm to users.

The Tencent Yuanbao app is an AI system generating content based on user input. The incident involves the AI system producing insulting and abusive language in generated images, which harmed users by causing offense and reputational damage. The harm is direct and realized, not just potential. The company's apology and corrective measures confirm the malfunction and harm. Therefore, this event meets the criteria for an AI Incident due to the AI system's malfunction causing direct harm to users.

The event explicitly involves an AI system (Tencent Yuanbao AI) whose malfunction during multi-turn dialogue caused it to output insulting and abusive language combined with the user's image. This directly harmed the user's personal dignity and reputation, which are recognized as violations of human rights and personality rights under applicable law. The incident is not hypothetical or potential but has already occurred and caused harm, meeting the definition of an AI Incident. The discussion of systemic issues and official responses further supports the classification as an incident rather than a hazard or complementary information.

The Tencent Yuanbao app uses AI to generate New Year greeting images. The AI system produced images containing insulting and abusive language instead of the intended positive greetings, directly causing harm to the user by exposing them to offensive content. The incident is linked to the AI system's malfunction during multi-turn dialogue processing. The company confirmed the AI model's abnormal output as the cause and apologized, showing the AI system's involvement in the harm. This meets the criteria for an AI Incident as the AI system's malfunction directly led to harm (offensive content violating user rights and causing emotional harm).

The AI system (Yuanbao AI) was used by a user to generate New Year greeting images. During multiple interactions, the AI unexpectedly produced abusive language without any provocation or inappropriate input from the user. This output caused emotional harm and violated the user's rights to dignity and respect. The incident is directly linked to the AI system's malfunction in handling multi-turn dialogue, as confirmed by the official apology and corrective actions. Therefore, this event meets the criteria for an AI Incident due to direct harm caused by the AI system's malfunction.

The AI system 'Yuanbao' is explicitly mentioned and is responsible for generating harmful outputs (insulting language) during user interaction, which constitutes a direct harm to users and communities. The event involves the AI system's malfunction during use, leading to realized harm (offensive content). Tencent's response and mitigation efforts are noted but do not negate the occurrence of the incident. Therefore, this qualifies as an AI Incident rather than a hazard or complementary information. The article also discusses the broader context of AI assistant competition and user retention but the core event is the harmful AI output incident.

The AI system (a large language model-based generative AI) was explicitly involved and malfunctioned by producing offensive language in response to user input. The harm is realized as the generation of harmful, offensive content that caused controversy and required an apology and correction. The event meets the criteria for an AI Incident because the AI's malfunction directly led to harm (offensive content harming users and communities). The company's response and correction are complementary information but do not negate the incident classification.

An AI system (Tencent's generative AI chatbot Yuanbao) was involved and malfunctioned during multi-turn dialogue processing, producing offensive images containing profanity. This output caused harm by spreading inappropriate content to users and the public, which is a form of harm to communities and social norms. The company's acknowledgment and corrective action confirm the AI system's role in causing the harm. Hence, this event meets the criteria for an AI Incident due to realized harm caused by the AI system's malfunction and use.

The incident involves a generative AI system that malfunctioned during use, producing offensive outputs that caused harm to users and the community by generating inappropriate content. This fits the definition of an AI Incident because the AI system's malfunction directly led to harm (offensive language output). The company's response and correction efforts are complementary information but do not negate the incident classification. Therefore, this event is classified as an AI Incident.

The AI system (Tencent's Yuanbao) malfunctioned by generating offensive, profane content in response to repeated user prompts, which is a direct result of its use and training. This caused harm in the form of offensive and inappropriate content dissemination, which can be considered harm to communities or users. Tencent's apology and corrective actions confirm recognition of the harm caused. Although no physical injury or legal rights violation is reported, the offensive outputs constitute a significant harm linked directly to the AI system's malfunction. Hence, this event meets the criteria for an AI Incident rather than a hazard or complementary information.

The event involves a generative AI system (Tencent's Yuanbao) that malfunctioned by generating offensive language during user interaction. This is a direct harm caused by the AI system's outputs, fulfilling the criteria for an AI Incident under harm to communities and violation of rights. The company's apology and corrective actions are complementary information but do not negate the incident classification. Therefore, this event qualifies as an AI Incident.

The AI system is explicitly involved as it generated insulting content during its use. The harm is realized as the user experienced emotional distress from the AI's offensive outputs. The company confirmed the issue was due to a model processing error, indicating a malfunction. The harm is direct and linked to the AI's malfunctioning output. This fits the definition of an AI Incident because the AI's malfunction directly led to harm to a person (emotional harm from insults).

The AI system "Yuanbao" was used by a user to generate images and respond to requests. Due to a malfunction in multi-turn dialogue processing, the AI produced offensive language, directly harming the user by verbal abuse. This constitutes an AI Incident because the AI's malfunction directly led to harm (emotional/verbal harm) to the user. The event is not merely a potential hazard or complementary information but a realized harm caused by the AI system's outputs.

Tencent Yuanbao is a generative AI chatbot integrated into a widely used platform, and it produced insulting language in generated images without user provocation, indicating a malfunction of the AI system. This output caused harm to the user by exposing them to offensive content, which is a clear harm to the user community and a violation of user rights. The official acknowledgment and emergency fix confirm the AI system's role in causing the harm. Therefore, this qualifies as an AI Incident due to the realized harm caused by the AI system's malfunction.

The Tencent Yuanbao App uses an AI model to generate content (greeting posters and code beautification). The model produced offensive language unexpectedly, which is a malfunction of the AI system. This caused direct harm to users by exposing them to insulting and inappropriate language. The company confirmed the issue was due to the AI model's abnormal output and not user error or manual intervention. Such offensive outputs can be considered harm to individuals and communities, fitting the definition of an AI Incident. The company's response and correction are noted but do not change the classification of the original event as an AI Incident.

The AI system 'Yuanbao' is explicitly mentioned and is a generative AI assistant. The incidents involve the AI's malfunction in generating abusive language without user provocation, which directly harms users by causing emotional distress and undermining trust. The harm is realized and not merely potential. The company's acknowledgment and remediation efforts do not negate the fact that harm occurred. Therefore, this event qualifies as an AI Incident due to the AI system's malfunction leading to direct harm to users.

The AI system's malfunction directly led to harm in the form of offensive and insulting content being generated and delivered to the user, which constitutes harm to the individual's dignity and emotional well-being. This fits the definition of an AI Incident because the AI's malfunction caused realized harm. The company's response and apology are complementary information but do not negate the incident classification.