Brazilian Superior Court Investigates AI Prompt Injection Fraud in Legal Petitions

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

The Brazilian Superior Tribunal de Justiça (STJ) launched investigations after detecting prompt injection attacks in legal petitions, where hidden AI commands were used to bypass filters in the court's AI system, STJ Logos. The suspected fraud, allegedly by lawyers, threatens judicial process integrity. Technical safeguards are in place, and sanctions are being considered.[AI generated]

Why's our monitor labelling this an incident or hazard?

An AI system (STJ Logos) is explicitly involved, and the event concerns attempts to misuse it via prompt injection to manipulate judicial AI outputs. Although the system's security measures prevented harm, the attempts represent a credible risk of future harm to the integrity of judicial processes if successful. Therefore, this event is best classified as an AI Hazard, as it plausibly could lead to an AI Incident if the manipulations were effective.[AI generated]
AI principles
Robustness & digital security

Industries
Government, security, and defence

Affected stakeholders
General publicGovernment

Harm types
Public interestReputational

Severity
AI hazard

Business function:
Compliance and justice

AI system task:
Reasoning with knowledge structures/planning


Articles about this incident or hazard