Microsoft Develops Tools to Detect Backdoors in AI Language Models

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

Microsoft researchers have identified significant risks from 'sleeper agent' backdoors embedded in large language models, which can be triggered to produce malicious outputs. To address this AI hazard, Microsoft has developed new scanning tools and detection methods to identify poisoned models before deployment, aiming to prevent potential harm.[AI generated]

Why's our monitor labelling this an incident or hazard?

The article clearly involves AI systems, specifically AI language models, and discusses the development and use of AI models that can be poisoned. However, it does not describe any actual harm or incident resulting from such poisoning. Instead, it presents research findings about the risks and detection of model poisoning, which could plausibly lead to AI incidents if exploited maliciously. Therefore, the event is best classified as an AI Hazard, as it concerns a credible risk of harm from AI model poisoning and efforts to detect and mitigate this risk before harm occurs.[AI generated]
AI principles
Robustness & digital securitySafety

Industries
Digital security

Business function:
Research and development

AI system task:
Event/anomaly detection


Articles about this incident or hazard