AI-Generated Legal Pleadings Mislead Madhya Pradesh High Court

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

Madhya Pradesh government lawyers used AI tools to draft legal documents, resulting in the citation of a fabricated Supreme Court judgment. The High Court criticized the state for blindly relying on AI, quashed the police officer's dismissal, and warned against irresponsible use of AI in legal proceedings.[AI generated]

Why's our monitor labelling this an incident or hazard?

The AI system was used in drafting legal pleadings, resulting in fabricated and incorrect legal citations being presented to the court. This misuse of AI led to misleading the judiciary and wasting judicial time, which is a clear harm to the legal process and the rights of the individuals involved. The harm is realized and directly linked to the AI system's malfunction or misuse. Hence, this qualifies as an AI Incident under the category of violations of legal obligations and harm to rights.[AI generated]
AI principles
AccountabilityTransparency & explainability

Industries
Government, security, and defence

Affected stakeholders
GovernmentGeneral public

Harm types
ReputationalPublic interest

Business function:
Compliance and justice

AI system task:
Content generation


Articles about this incident or hazard