Shield AI and Kratos Integrate Autonomous AI Pilot in Military Drones, Raising Future Risk Concerns

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

Shield AI and Kratos are partnering to integrate an autonomous AI pilot into the XQ-58 Valkyrie military drone, enabling it to operate without GPS or communications. While no harm has occurred yet, the deployment of AI-controlled combat drones poses credible future risks of escalation and unintended harm in conflict scenarios.[AI generated]

Why's our monitor labelling this an incident or hazard?

The event involves the use of an AI system (an AI pilot) controlling an armed military drone. The AI system's development and intended use in combat situations could plausibly lead to harms such as injury or death, disruption of military operations, or other significant harms. Since the article does not report any actual harm or incident caused by the AI system yet, but highlights the potential for such harm, this qualifies as an AI Hazard rather than an AI Incident. The autonomous armed drone with AI pilot technology represents a credible future risk consistent with the definition of an AI Hazard.[AI generated]
AI principles
AccountabilitySafetyRobustness & digital securityRespect of human rightsTransparency & explainabilityDemocracy & human autonomy

Industries
Government, security, and defenceRobots, sensors, and IT hardwareMobility and autonomous vehiclesDigital security

Affected stakeholders
General public

Harm types
Physical (death)Physical (injury)Human or fundamental rightsPublic interestPsychological

Business function:
Research and development

AI system task:
Goal-driven organisationReasoning with knowledge structures/planningRecognition/object detection


Articles about this incident or hazard