AI Legal Advice Leads to Lawsuits and Court Sanctions

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

Japan Life Insurance sued OpenAI in the US after ChatGPT provided unauthorized legal advice, causing financial loss and legal disruption. Separately, two lawyers in Singapore were sanctioned for submitting AI-generated fake legal cases to court, highlighting risks of AI misuse in legal proceedings.[AI generated]

Why's our monitor labelling this an incident or hazard?

The event explicitly involves the use of AI tools to generate fabricated legal cases that were submitted to the court, which is a misuse of AI in a legal context. The AI system's outputs directly caused harm by misleading the court, wasting judicial resources, and threatening judicial fairness, which are violations of legal and professional rights and obligations. The court's sanctions and the description of harm confirm that the AI system's involvement led to realized harm, meeting the criteria for an AI Incident rather than a hazard or complementary information.[AI generated]
AI principles
AccountabilityTransparency & explainability

Industries
Financial and insurance servicesGovernment, security, and defence

Affected stakeholders
BusinessGovernment

Harm types
Economic/PropertyPublic interest

Business function:
Compliance and justice

AI system task:
Content generationInteraction support/chatbots


Articles about this incident or hazard