AI-Guided Hike Leads to Rescue on Mount Shasta

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

Three novice hikers in California relied on Google's Gemini AI to plan their Mount Shasta ascent, receiving inadequate advice on supplies and route. This led to them becoming stranded, one suffering an injury, and requiring a rescue operation by local authorities. The incident highlights harm caused by overreliance on AI-generated guidance.[AI generated]

Why's our monitor labelling this an incident or hazard?

The AI system (Google's Gemini AI) was used in the planning phase and provided recommendations that underestimated necessary provisions, which indirectly led to the hikers being stranded and requiring rescue. The event involves realized harm to persons (health and safety risk) caused indirectly by reliance on the AI system's outputs. Therefore, this qualifies as an AI Incident under the framework, as the AI system's use directly or indirectly led to harm to persons.[AI generated]
AI principles
SafetyTransparency & explainability

Industries
Travel, leisure, and hospitality

Affected stakeholders
Consumers

Harm types
Physical (injury)

AI system task:
Content generationReasoning with knowledge structures/planning


Articles about this incident or hazard