AI Malfunction Causes Golden Horde Swarming Munitions to Miss Intended Targets in USAF Test

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

During a December test of the US Air Force's Golden Horde program, AI-enabled swarming munitions failed to update flight profiles due to software issues, causing them to strike a failsafe target instead of higher-priority targets. This malfunction highlights operational risks in autonomous military AI systems.[AI generated]

Why's our monitor labelling this an incident or hazard?

The Golden Horde program involves AI systems enabling autonomous swarming bombs. The described software malfunction caused the bombs to hit a failsafe target, preventing harm. No actual injury, property damage, or rights violations occurred. Therefore, this is not an AI Incident. However, the development and testing of autonomous lethal AI weapons with known software issues plausibly pose future risks of harm, qualifying this as an AI Hazard rather than an Incident or Complementary Information. The article focuses on the program's progress and testing challenges, indicating potential future harm rather than realized harm.[AI generated]
AI principles
AccountabilityRobustness & digital securitySafety

Industries
Government, security, and defence

Affected stakeholders
Government

Harm types
Economic/PropertyReputational

Business function:
Research and development

AI system task:
Goal-driven organisationReasoning with knowledge structures/planning


Articles about this incident or hazard