Plaintiff Sanctioned for Attempted AI Prompt Injection in Connecticut Court

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

A Connecticut plaintiff, Matthew Elliott, was sanctioned after embedding hidden AI prompt injections in court filings, aiming to manipulate any AI system reviewing the documents. Although the court does not use AI, the incident highlights risks of AI misuse in legal processes and sets a concerning precedent for future cases.[AI generated]

Why's our monitor labelling this an incident or hazard?

An AI system is explicitly referenced as the target of the hidden prompt injections, which are designed to influence AI outputs in favor of one party. Although no actual harm occurred because the court does not use AI to review filings, the event demonstrates a credible risk that such misuse could lead to biased judicial decisions and harm public trust if AI systems were used. This fits the definition of an AI Hazard, as the development and use of prompt injection techniques could plausibly lead to an AI Incident involving violations of legal fairness and harm to communities' trust in institutions. Since no realized harm occurred, it is not an AI Incident. The event is not merely complementary information or unrelated, as it directly concerns AI misuse with potential harm.[AI generated]
AI principles
Robustness & digital security

Industries
Government, security, and defence


Articles about this incident or hazard