AI Essay Detector Triggers Lawsuit Over Student's Grade in Palo Alto

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

In Palo Alto, California, a high school student's essay was flagged as AI-generated by detection software, resulting in a lower grade and academic dishonesty record. The student's family sued the school district, alleging unfair treatment and due process violations, highlighting the risks of AI-driven academic assessments.[AI generated]

Why's our monitor labelling this an incident or hazard?

An AI system (Turnitin's AI-detection software) was used in the evaluation of a student's essay, leading to a significant negative academic consequence (grade reduction) and legal claims alleging discrimination and denial of due process. The AI system's output directly influenced the harm experienced by the student and family. The involvement of AI in causing this harm meets the criteria for an AI Incident, as it led to violations of rights and harm to the student. The other dispute about report card comments does not involve AI and is unrelated to AI harms.[AI generated]
AI principles
FairnessTransparency & explainability

Industries
Education and training

Affected stakeholders
Children

Harm types
ReputationalPsychologicalHuman or fundamental rights

AI system task:
Event/anomaly detection


Articles about this incident or hazard