AI Agents Fail at Mathematical and Logical Tasks, Causing User Harm

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

A recent study found that advanced AI agents, promoted by tech giants for automating tasks like bill payments and travel planning, consistently fail at basic logic and mathematics. These failures, due to error accumulation in multi-step tasks, have led to financial losses and incomplete user requests, highlighting significant reliability risks.[AI generated]

Why's our monitor labelling this an incident or hazard?

The article explicitly discusses AI agents (AI systems) and their malfunction in performing mathematical and logical tasks, which can directly lead to harm such as financial loss or failure to secure accommodations. The harm is realized or at least highly plausible given the described failures. Therefore, this qualifies as an AI Incident because the AI system's malfunction has directly or indirectly led to harm to persons (financial and practical harm).[AI generated]
AI principles
SafetyRobustness & digital security

Industries
Financial and insurance servicesTravel, leisure, and hospitality

Affected stakeholders
Consumers

Harm types
Economic/Property

Business function:
Citizen/customer service

AI system task:
Interaction support/chatbotsGoal-driven organisation


Articles about this incident or hazard