Concerns Over AI-Generated Police Reports

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

US police departments are considering using Axon's AI tool, "Draft One," to generate police reports from body camera audio using GPT-4. Experts warn that AI's tendency to "hallucinate" or produce errors could lead to wrongful imprisonment, raising concerns about human rights violations and legal obligations.[AI generated]

Why's our monitor labelling this an incident or hazard?

The piece centers on the imminent deployment of an AI system in law enforcement and the plausible dangers of its known failure mode (hallucination) without describing an actual incident of harm. As such, it describes a scenario where AI use could reasonably lead to significant future harm, fitting the definition of an AI Hazard.[AI generated]
AI principles
AccountabilityFairnessPrivacy & data governanceRespect of human rightsRobustness & digital securitySafetyTransparency & explainability

Industries
Government, security, and defence

Affected stakeholders
General public

Harm types
Human or fundamental rightsPublic interestReputationalPsychological

Business function:
Compliance and justice

AI system task:
Content generation


Articles about this incident or hazard