The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.
A South African legal team used the AI tool Legal Genius to generate court submissions, resulting in fabricated case citations being presented in a licensing dispute. The court discovered the false references, prompting a referral to the Legal Practice Council for professional misconduct investigation and highlighting risks of unverified AI use in legal proceedings.[AI generated]
Why's our monitor labelling this an incident or hazard?
The AI system was used in legal research and produced fabricated case citations ('hallucinations'), which were included in legal arguments before a court. This misuse or malfunction of the AI system directly led to misinformation presented in a legal proceeding, undermining the administration of justice and potentially violating legal standards. The judge's concern and referral for investigation further underscore the seriousness of the harm. Hence, the event meets the criteria for an AI Incident due to indirect harm to the justice system and legal rights.[AI generated]