Tesla Autopilot Under Investigation After Crashes and Safety Concerns Involving AI System

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

US regulators are investigating Tesla's Autopilot AI system after multiple crashes, including incidents where vehicles struck emergency vehicles and failed to avoid child-sized mannequins. The NHTSA is scrutinizing the effectiveness of Tesla's driver monitoring and warning systems, raising concerns about the AI's ability to ensure road safety.[AI generated]

Why's our monitor labelling this an incident or hazard?

Tesla's Full Self-Driving system is an AI system responsible for autonomous vehicle navigation and pedestrian detection. The reported failure to detect a child-sized mannequin and stop the vehicle represents a malfunction or use-related issue that directly risks injury to children, fulfilling the criteria for harm to persons. The mention of a government recall further supports the seriousness of the issue. Although some defenses and contesting views are presented, the core event involves realized harm risk from AI malfunction. Therefore, this qualifies as an AI Incident.[AI generated]
AI principles
AccountabilityRobustness & digital securitySafetyTransparency & explainabilityDemocracy & human autonomy

Industries
Mobility and autonomous vehiclesGovernment, security, and defence

Affected stakeholders
General public

Harm types
Physical (injury)Economic/PropertyReputationalPublic interest

Business function:
Monitoring and quality control

AI system task:
Recognition/object detectionReasoning with knowledge structures/planningGoal-driven organisation


Articles about this incident or hazard