Meta's Galactica AI Pulled After Spreading Misinformation and Harmful Content

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

Meta launched Galactica, an AI language model designed to generate scientific content, but quickly withdrew it after users found it produced false, biased, and harmful outputs, including racist remarks and dangerous misinformation. The system's inability to distinguish fact from fiction led to ethical criticism and reputational harm.[AI generated]

Why's our monitor labelling this an incident or hazard?

The event involves the use and deployment of an AI system (Galactica, a large language model) whose outputs have directly led to harms such as dissemination of biased, inaccurate, and potentially harmful scientific information. This constitutes harm to communities and a violation of the obligation to provide truthful information, fitting the definition of an AI Incident. The article describes realized harms from the AI system's use, not just potential risks, and thus it is classified as an AI Incident rather than a hazard or complementary information.[AI generated]
AI principles
AccountabilityFairnessRespect of human rightsSafetyTransparency & explainabilityRobustness & digital security

Industries
Media, social platforms, and marketingEducation and training

Affected stakeholders
ConsumersBusiness

Harm types
ReputationalHuman or fundamental rightsPsychologicalPublic interest

Severity
AI incident

Business function:
Research and development

AI system task:
Content generationReasoning with knowledge structures/planning

In other databases

Articles about this incident or hazard

Thumbnail Image

Trust Large Language Models at Your Own Risk - ExBulletin

2022-11-23
ExBulletin
Why's our monitor labelling this an incident or hazard?
The event involves the use and deployment of an AI system (Galactica, a large language model) whose outputs have directly led to harms such as dissemination of biased, inaccurate, and potentially harmful scientific information. This constitutes harm to communities and a violation of the obligation to provide truthful information, fitting the definition of an AI Incident. The article describes realized harms from the AI system's use, not just potential risks, and thus it is classified as an AI Incident rather than a hazard or complementary information.
Thumbnail Image

Meta Takes Down The Popular Galactica AI Bot Following The Fake Research Complaints

2022-11-24
Mashable India
Why's our monitor labelling this an incident or hazard?
Galactica is an AI system designed to generate scientific knowledge summaries and assist researchers. Its deployment led to the generation and dissemination of false and misleading information, including racist statements and fabricated studies, which harmed individuals' reputations and misinformed users. The harm is direct and materialized, as users reported and denounced the misleading content, and Meta suspended the bot in response. This fits the definition of an AI Incident because the AI system's use directly led to harm to communities and violations of rights through misinformation and reputational damage.
Thumbnail Image

Bigshot chief scientist of major corporation can't handle criticism of the work he hypes.

2022-11-23
Statistical Modeling, Causal Inference, and Social Science
Why's our monitor labelling this an incident or hazard?
Galactica is a large language model AI system explicitly mentioned. Its use led to the generation of biased, incorrect, and fabricated scientific content, which harmed the scientific community and users by spreading misinformation. This constitutes harm to communities and intellectual property rights (due to fabricated papers attributed to real authors). The event involves the use and malfunction (inability to distinguish fact from fiction) of the AI system leading directly to harm. The public criticism and removal of the demo confirm the harm was realized, not just potential. Hence, this is an AI Incident.
Thumbnail Image

Comment on Bigshot chief scientist of major corporation can't handle criticism of the work he hypes. by Thomas [not Basbøll]

2022-11-23
Statistical Modeling, Causal Inference, and Social Science
Why's our monitor labelling this an incident or hazard?
The event involves a large language model AI system (Galactica) that was publicly released and then taken down after it produced biased, incorrect, and fabricated scientific content. This misinformation constitutes harm to communities (scientific and general public) and violates the expectation of accurate scientific communication. The AI system's malfunction and deployment directly caused this harm. The article also discusses the response of Meta's chief scientist, but the primary focus is on the AI system's failure and its consequences. Hence, this is an AI Incident as per the definitions provided.
Thumbnail Image

Artificial Intelligence bot goes rogue and is 'most dangerous thing Meta's made'

2022-11-22
The US Sun
Why's our monitor labelling this an incident or hazard?
The AI system (Galactica) was explicitly mentioned and described as generating fake and racist scientific research, which constitutes harm to communities through misinformation. The harm is realized as the bot was live and actively posting such content, leading to its removal. The incident involves the AI system's use and malfunction (producing false and harmful outputs). This fits the definition of an AI Incident because the AI system's outputs directly led to harm (misinformation and racist content dissemination).
Thumbnail Image

Critics reject Meta-trained AI made to make scientific research easier

2022-11-24
The News International
Why's our monitor labelling this an incident or hazard?
Galactica is an AI system explicitly described as using machine learning to organize scientific knowledge and generate research summaries. Its malfunction and erroneous outputs caused misinformation dissemination, a form of harm to communities and scientific integrity. The harm is realized as users received false or misleading scientific information, which can negatively impact understanding and decision-making. The removal of the demo reflects recognition of this harm. Therefore, this event qualifies as an AI Incident due to the direct harm caused by the AI system's malfunction and use.
Thumbnail Image

Meta's New AI Galactica Model Is Gone - It Makes Stuff Up

2022-11-22
www.independentsentinel.com
Why's our monitor labelling this an incident or hazard?
The article explicitly describes an AI system (Meta's Galactica LLM) that was publicly used and generated harmful outputs such as racist content, false scientific information, and dangerous instructions. These outputs represent realized harm to communities and potentially violate ethical and societal norms. The AI system's malfunction or limitations directly caused these harms, meeting the criteria for an AI Incident rather than a hazard or complementary information.
Thumbnail Image

Meta AI Bot Contributed to Fake Research and Nonsense Before Being Pulled Offline

2022-11-22
Gizmodo AU
Why's our monitor labelling this an incident or hazard?
Galactica is an AI system (a large language model trained on scientific texts) that was publicly released and generated false and harmful content, including misinformation and racist statements. This directly led to harm by spreading misinformation and potentially damaging reputations, fulfilling the criteria for an AI Incident. The harm is realized, not just potential, as users reported and shared harmful outputs. The event is not merely a product launch or research announcement but involves actual harm caused by the AI system's outputs, thus qualifying as an AI Incident.
Thumbnail Image

Meta Halts Academic Paper Generator AI Demo After 3 Days

2022-11-23
Voicebot.ai
Why's our monitor labelling this an incident or hazard?
Galactica is an AI system (a large language model) designed to generate academic content. Its outputs included false scientific claims and instructions for bomb-making, which represent misinformation and potential harm to public safety. These outputs demonstrate direct harm to communities through misinformation and potential physical harm if instructions are followed. The event describes the AI's use and its problematic outputs leading to these harms, fulfilling the criteria for an AI Incident. The retraction of the demo further supports the recognition of harm caused. Hence, the classification as AI Incident is appropriate.
Thumbnail Image

Meta AI Bot Pulled Offline After Releasing a Wave of Misinformation

2022-11-25
https://socialbarrel.com
Why's our monitor labelling this an incident or hazard?
Galactica is an AI system designed to summarize scientific data and generate content. Its deployment led to the direct dissemination of misinformation and racist content, which constitutes harm to communities and individuals' reputations. The harm is realized, not just potential, as users reported and experienced misleading and harmful outputs. Therefore, this qualifies as an AI Incident because the AI system's use directly led to harm.
Thumbnail Image

Galactica, Meta's racist artificial intelligence that spreads fake news and only lasted 72 hours

2022-11-25
Bullfrag
Why's our monitor labelling this an incident or hazard?
Galactica is an AI system designed to generate scientific text. Its deployment resulted in the creation and dissemination of false and misleading scientific information, as well as racist and offensive remarks. These outputs represent harm to communities through misinformation and violation of ethical norms, fulfilling the criteria for an AI Incident. The harm is direct and realized, as the AI's outputs were publicly accessible and caused ethical criticism and reputational damage, leading to the system's withdrawal.
Thumbnail Image

Türkiye'nin Yapay Zeka Küresel Ortaklığı üyelik başvurusu onaylandı

2022-11-23
Yeni Çağ Gazetesi
Why's our monitor labelling this an incident or hazard?
The article does not describe any AI system involved in an incident or hazard. It is about a country's membership approval to an international AI governance and cooperation platform. There is no mention of any harm caused or potential harm from AI systems. The content is about governance, collaboration, and ethical AI promotion, which fits the definition of Complementary Information as it provides context and updates on AI ecosystem developments and governance responses.
Thumbnail Image

Türkiye'nin yapay zeka küresel ortaklığı üyelik başvurusu onaylandı - Ankara Haberleri

2022-11-23
HABERTURK.COM
Why's our monitor labelling this an incident or hazard?
The article does not mention any specific AI system, nor does it describe any event involving AI development, use, malfunction, or harm. It is about Turkey joining an international AI governance partnership to promote ethical and responsible AI. This is a governance and cooperation update, providing context to the AI ecosystem but not reporting any incident or hazard. Therefore, it fits the definition of Complementary Information.
Thumbnail Image

Meta, Galactica Yapay Zekasını Saçmaladığı İçin Kapattı

2022-11-20
Webtekno
Why's our monitor labelling this an incident or hazard?
The event involves the use of an AI system (a large language model) whose outputs directly led to harm in the form of dissemination of misleading, biased, and racist content, which can be considered harm to communities and a violation of ethical standards. The AI's malfunction in generating false scientific references and biased content caused reputational and ethical harm, justifying classification as an AI Incident.
Thumbnail Image

Türkiye, Yapay Zeka Küresel Ortaklığı'na üye oldu - Bloomberg HT

2022-11-24
BloombergHT
Why's our monitor labelling this an incident or hazard?
The article focuses on Turkey joining an international AI governance partnership and highlights the strategic importance of AI for economic development and responsible use. It does not mention any AI system involved in an incident or hazard, nor does it describe any realized or potential harm caused by AI. Therefore, it is best classified as Complementary Information, providing context and updates on AI governance and policy developments rather than reporting an AI Incident or AI Hazard.
Thumbnail Image

Türkiye'den GPAI üyeliği hakkında açıklama

2022-11-23
Ak�am
Why's our monitor labelling this an incident or hazard?
The article does not mention any specific AI system causing harm or malfunction, nor does it describe any incident or plausible future harm related to AI. Instead, it reports on a governance and cooperation development in the AI ecosystem, which fits the definition of Complementary Information as it provides context and updates on AI governance and international collaboration without describing an AI Incident or Hazard.
Thumbnail Image

Bakan Varank Meclis'te açıkladı... Türkiye, Yapay Zeka Küresel Ortaklığı'na üye oldu

2022-11-24
Ak�am
Why's our monitor labelling this an incident or hazard?
The event describes a country's membership in an international AI partnership focused on research, responsible AI development, and collaboration. There is no mention of any AI system causing harm, nor any plausible future harm from this membership itself. The article is about a governance and cooperation development in the AI ecosystem, which fits the definition of Complementary Information as it provides context and updates on AI governance and international collaboration without describing an AI Incident or AI Hazard.
Thumbnail Image

Bakan Varank, Türkiye'nin Yapay Zeka Küresel Ortaklığı'na üye olduğunu açıkladı

2022-11-24
Bursada Bugün
Why's our monitor labelling this an incident or hazard?
The event involves AI in the context of international cooperation and governance but does not describe any AI system's development, use, malfunction, or any realized or potential harm. It is a policy and strategic update about AI collaboration, which fits the definition of Complementary Information as it provides context and background on AI ecosystem developments without reporting an incident or hazard.
Thumbnail Image

La nueva inteligencia artificial de Facebook: racista y difunde 'fake News'

2022-11-21
MARCA
Why's our monitor labelling this an incident or hazard?
The article explicitly mentions that the AI system (Galactica, a large language model) was tested and found to produce racist and misleading outputs, leading to its withdrawal. The harms described (racism and dissemination of fake news) are realized harms caused by the AI system's outputs during its use. Therefore, this qualifies as an AI Incident because the AI system's use directly led to harm to communities and societal harm through misinformation and offensive content.
Thumbnail Image

Meta retira su modelo de IA para ayudar a la ciencia: difundía bulos y era racista

2022-11-23
LaVanguardia
Why's our monitor labelling this an incident or hazard?
Galactica is an AI system (a large language model) explicitly described as generating scientific text and other content. Its use led directly to the production and dissemination of false, racist, and harmful information, which is a form of harm to communities and individuals. The harm is realized, as users observed and shared these outputs, leading Meta to withdraw the demo. The event involves the AI system's use and malfunction (producing erroneous and harmful outputs). Therefore, it meets the criteria for an AI Incident rather than a hazard or complementary information.
Thumbnail Image

Galactica: la IA de Meta que solo duró tres días por ser racista y difundir Fake News

2022-11-22
El Comercio Perú
Why's our monitor labelling this an incident or hazard?
Galactica is an AI language model designed to generate scientific text but produced falsehoods and racially biased content. The AI system's outputs directly caused harm by disseminating misinformation and offensive content, which aligns with harm to communities and violations of ethical norms. The event involves the use and malfunction of an AI system leading to realized harm, meeting the criteria for an AI Incident rather than a hazard or complementary information.
Thumbnail Image

Lo que le faltaba a Facebook: su nueva inteligencia artificial es racista y difunde Fake News

2022-11-21
ComputerHoy.com
Why's our monitor labelling this an incident or hazard?
Galactica is a large language model AI system explicitly mentioned. Its use led directly to the generation and dissemination of racially biased and false scientific information, which can harm communities by spreading misinformation and offensive content. The harm is realized as users were able to generate and access such content before the demo was withdrawn. This fits the definition of an AI Incident due to harm to communities and ethical violations caused by the AI system's outputs.
Thumbnail Image

Meta retira IA Galactica por difundir información falsa y racista | Periódico Zócalo | Noticias de Saltillo, Torreón, Piedras Negras, Monclova, Acuña

2022-11-23
Zócalo Saltillo
Why's our monitor labelling this an incident or hazard?
The event involves an AI system (Galactica, a large language model) whose use directly led to harms including the spread of false scientific information and racially biased content, which can be considered harm to communities and violation of ethical standards. The harms are realized and documented, not merely potential. Therefore, this qualifies as an AI Incident under the framework because the AI system's outputs caused significant misinformation and discriminatory content dissemination.
Thumbnail Image

Poco attendibile, Meta ritira il motore di ricerca IA - Hi-tech

2022-11-23
ANSA.it
Why's our monitor labelling this an incident or hazard?
An AI system (Galactica) was explicitly involved and used publicly. Its outputs included false and misleading information, which can harm communities by spreading misinformation and eroding trust in scientific knowledge. The event describes realized harm in the form of dissemination of false information, even if the project was paused shortly after launch. This fits the definition of an AI Incident because the AI system's use directly led to harm (misinformation dissemination).
Thumbnail Image

Meta ritira il motore di ricerca d'intelligenza artificiale Galactica

2022-11-23
Prima Comunicazione
Why's our monitor labelling this an incident or hazard?
Galactica is an AI system designed to provide encyclopedic search and answers based on scientific data. Its outputs included false and misleading information, which directly relates to harm to communities through misinformation and potential violation of intellectual property rights by fabricating citations. Meta's decision to pause the project due to these issues indicates that the AI system's use led to realized harms. Therefore, this qualifies as an AI Incident because the AI system's use directly led to harm through misinformation dissemination and potential intellectual property violations.
Thumbnail Image

Meta "sospende" Galactica, fa flop il motore di ricerca AI - CorCom

2022-11-23
Cor.Com
Why's our monitor labelling this an incident or hazard?
Galactica is an AI language model designed to provide scientific information and reasoning. Its deployment led to the generation and dissemination of false and misleading scientific content, including fabricated studies and contradictory answers to factual questions. This misinformation can harm communities by spreading disinformation and undermining trust in science, fitting the definition of harm to communities. The AI system's use directly caused this harm, qualifying the event as an AI Incident rather than a hazard or complementary information. The suspension of the system acknowledges the realized harm caused by its outputs.
Thumbnail Image

Meta ritira il motore di ricerca IA: "Poco attendibile"

2022-11-23
L'opinione delle Libertà
Why's our monitor labelling this an incident or hazard?
Galactica is an AI system designed to provide encyclopedic search and answers based on a large dataset of scientific and textual information. Its malfunction—producing false, misleading, and fabricated content—directly led to the spread of misinformation, which constitutes harm to communities and the environment of knowledge. The event involves the use and malfunction of an AI system that caused realized harm through misinformation dissemination. Therefore, this qualifies as an AI Incident under the framework, as the AI system's malfunction directly led to harm (misinformation and potential public harm).
Thumbnail Image

Galactica: il big flop di Mark Zuckerberg

2022-11-26
Millionaire
Why's our monitor labelling this an incident or hazard?
Galactica is explicitly described as an AI system designed to provide encyclopedic knowledge. Its malfunction led to the dissemination of false and misleading information, including fake news and fabricated scientific citations. This constitutes harm to communities through misinformation and potential violation of rights to accurate information. The harm is realized and directly linked to the AI system's outputs. Therefore, this event qualifies as an AI Incident.