Brazilian Superior Court Investigates AI Prompt Injection Fraud in Legal Petitions

An AI system (STJ Logos) is explicitly involved, and the event concerns attempts to misuse it via prompt injection to manipulate judicial AI outputs. Although the system's security measures prevented harm, the attempts represent a credible risk of future harm to the integrity of judicial processes if successful. Therefore, this event is best classified as an AI Hazard, as it plausibly could lead to an AI Incident if the manipulations were effective.

The article explicitly involves an AI system (STJ Logos) used for automated legal document triage. The fraudulent use of prompt injection to manipulate this AI system directly led to attempts to bypass legal admissibility filters, which is a violation of legal rights and process integrity (harm to the judicial system and potentially to parties involved). Although the attacks were contained, the fraudulent attempts and their detection have triggered criminal investigations and administrative sanctions, indicating realized harm and legal violations. Therefore, this event meets the criteria for an AI Incident due to the direct involvement of AI misuse causing harm and legal consequences.

An AI system is explicitly involved: the court's electronic petition system uses AI models with filters. The event involves misuse of the AI system (prompt injection) to attempt fraud. Although safeguards currently prevent execution of these manipulations, the attempts to bypass AI filters could plausibly lead to harm such as legal fraud or disruption of judicial processes. Since no actual harm has been reported yet, but there is a credible risk, this qualifies as an AI Hazard rather than an AI Incident. The investigation and safeguards indicate recognition of this plausible future harm.

An AI system (STJ Logos) is explicitly involved, and the event concerns the use (misuse) of this AI system via prompt injection to manipulate legal processes. However, the AI system's protections successfully neutralized these attempts, preventing any realized harm such as violation of legal process integrity or rights. Since no harm has occurred but there is a credible risk of harm if such attempts were successful, this qualifies as an AI Hazard rather than an AI Incident. The article focuses on the potential risk and the system's defense rather than an actual harm event.

The article explicitly mentions the use of a generative AI system (STJ Logos) and describes attempts to manipulate it via prompt injection, a known AI misuse technique. While the system has safeguards, the ongoing attempts and investigation indicate a credible risk of harm to the judicial process and legal rights. Since no confirmed harm has yet occurred but plausible future harm is evident, the event fits the definition of an AI Hazard rather than an AI Incident or Complementary Information.

The article explicitly involves an AI system (STJ Logos, a generative AI used in judicial processes) and discusses attempts to manipulate it via prompt injection, which is a misuse of the AI system. Although the AI system currently can prevent these attempts, the investigation and legal actions indicate a credible risk that such misuse could lead to harm, such as procedural fraud and violation of legal rights. Since no actual harm has been reported yet, but plausible future harm is recognized and being addressed, this fits the definition of an AI Hazard rather than an AI Incident. The event is not merely complementary information because it focuses on the investigation of a credible risk rather than a response to a past incident.

The article explicitly involves an AI system (Galileu) used in judicial decision-making. The event concerns the use and attempted misuse of this AI system through prompt injection attacks to manipulate legal outcomes, which is a direct violation of legal rights and judicial integrity. The harm is realized in the form of attempted fraud and undermining of the judicial process, which falls under violations of human rights and breach of legal obligations. The investigation and sanctions confirm the materialization of harm. Hence, this is an AI Incident rather than a hazard or complementary information.

The event explicitly involves an AI system (STJ Logos, a generative AI system used by the court) and describes attempts to manipulate it via prompt injection, a known AI-specific attack vector. These attempts have been detected and neutralized, but their existence and investigation indicate direct misuse of the AI system that could lead to harm in judicial processes, including violations of legal rights and procedural fairness. The event describes realized attempts (not just potential) to interfere with the AI's outputs, which is a direct or indirect cause of harm to legal rights and the justice system's integrity. Hence, it meets the criteria for an AI Incident rather than a hazard or complementary information.

The article explicitly mentions the use of prompt injection, a known AI attack technique, targeting the STJ Logos AI system. The attempts are real and have been detected in legal petitions, indicating direct malicious use of an AI system. Although the system's defenses have prevented execution of harmful commands, the presence of these attempts and the initiation of investigations and sanctions show that harm to legal process integrity and rights is a real concern. The AI system's malfunction is not the cause, but the malicious use of the AI system's input processing is leading to an incident involving potential harm to legal rights and judicial process integrity. Therefore, this event qualifies as an AI Incident rather than a mere hazard or complementary information.

The article explicitly mentions the use of prompt injection to deceive AI models managing electronic legal processes, indicating the involvement of AI systems. The fraudulent use of AI to manipulate legal outcomes constitutes a violation of legal rights and obligations, fitting the definition of an AI Incident. The investigation confirms that harm or attempted harm has occurred or is ongoing, rather than being a mere potential risk.

The event explicitly involves an AI system (STJ Logos) used in judicial decision analysis. The use of 'prompt injection' is a known technique to manipulate AI outputs, which could plausibly lead to harm such as fraudulent judicial decisions, violating legal rights and undermining trust in the justice system. The article reports ongoing investigations and detection of such attempts but does not confirm that harm has already occurred. Thus, the event represents a credible risk of harm stemming from AI misuse, fitting the definition of an AI Hazard rather than an AI Incident. The event is not merely complementary information because it focuses on the investigation of a specific potential misuse with plausible harm, nor is it unrelated as it directly concerns AI system manipulation and its implications.

The event involves the use and attempted misuse of an AI system (the court's AI-based petition system) to commit fraud by bypassing admissibility filters through prompt injection. This misuse directly threatens the legal process and the rights of parties involved, constituting a violation of legal obligations and potentially harming the justice system's integrity. Therefore, this qualifies as an AI Incident because the AI system's misuse has directly led to a breach of legal procedural rights and an attempt to commit fraud within the judicial system.

The event explicitly involves an AI system (the tribunal's electronic petition system using AI models with filters) and describes attempts to misuse it via prompt injection to bypass legal filters, which is a misuse of the AI system. This misuse directly relates to a breach of legal obligations and judicial process integrity, which falls under violations of applicable law protecting fundamental rights. The investigation is in response to actual attempts, indicating realized harm or at least direct attempts at harm. Therefore, this is classified as an AI Incident.

The use of AI prompts to fraudulently manipulate the court's electronic petition system directly relates to the misuse of an AI system leading to a breach of legal obligations and potentially undermining the justice system. Although the investigation is just starting, the described attempts at fraud have already occurred, indicating realized harm related to violations of applicable law and rights. Therefore, this qualifies as an AI Incident.

An AI system is explicitly involved as the court uses AI models to assist in triaging and processing judicial documents. The event stems from the attempted misuse of this AI system via prompt injection to fraudulently influence legal document acceptance, which could have led to harm to the integrity of judicial processes (a form of harm to the operation of critical infrastructure, here the justice system). Although no actual harm occurred, the investigation and the nature of the attempts indicate a direct link to potential harm. Since the harm was not realized but the risk was credible and the system was targeted for manipulation, this qualifies as an AI Hazard rather than an AI Incident. The article focuses on the investigation and security measures rather than reporting actual harm caused by the AI system's malfunction or misuse.

The event involves the use and misuse of an AI system (the court's AI analyzing petitions) through prompt injection, which is a known AI-specific attack method. The misuse has already occurred as fraudulent petitions were identified, indicating realized harm in terms of legal process integrity and potential violation of judicial rights or obligations. Therefore, this is an AI Incident due to the direct involvement of AI misuse leading to harm in the judicial process.

An AI system is explicitly involved as the court's electronic petition system uses AI to process submissions. The event concerns the misuse of AI prompts (prompt injection) to bypass system filters, which is a misuse of the AI system. This misuse directly threatens the integrity of legal processes and could lead to violations of legal rights and procedural fairness, qualifying as harm under the framework. Since the investigation is in response to detected attempts (not just theoretical risks), and these attempts constitute a breach of legal obligations and rights, this qualifies as an AI Incident rather than a mere hazard or complementary information.

An AI system (STJ Logos) is explicitly involved, and its use is being manipulated via prompt injection to commit fraud in legal processes. This constitutes misuse of an AI system that directly threatens the integrity of judicial procedures, which can be considered a violation of legal and procedural rights. The investigation and mapping of these attempts indicate that harm has occurred or is occurring through fraudulent actions enabled by AI misuse. Therefore, this event qualifies as an AI Incident due to the realized harm related to violations of legal process integrity and potential breaches of rights.

An AI system (STJ Logos) is explicitly involved and is being targeted by prompt injection attacks, a form of malicious use of AI inputs. The event concerns the use and attempted misuse of the AI system in judicial processes, which could plausibly lead to harm such as violations of legal rights or undermining judicial integrity. Although no direct harm has yet been reported, the investigation and security measures indicate a credible risk of future harm. Therefore, this event is best classified as an AI Hazard rather than an Incident or Complementary Information.

An AI system is explicitly involved—the court's AI-based petition filtering system. The event concerns the use (misuse) of AI techniques (prompt injection) to manipulate the system to gain unfair advantage in legal processes, which could lead to violations of legal rights and harm to the justice system's integrity. However, the system's safeguards have so far prevented the execution of these manipulative commands, and no actual harm or breach has been reported. Thus, the event is best classified as an AI Hazard, reflecting a credible risk of harm from AI misuse in a critical infrastructure context, but without confirmed realized harm yet.

The event involves the use and attempted misuse of an AI system (STJ Logos) through prompt injection to fraudulently influence judicial decisions, which could lead to violations of legal rights and undermine the justice system. Since the harm is not confirmed as having occurred but the attempts are real and under investigation, this constitutes a plausible risk of harm. Therefore, this event qualifies as an AI Hazard rather than an AI Incident, as the harm is potential but not yet realized.

The event explicitly involves an AI system used in the court's electronic petition processing. The use of prompt injection to manipulate the AI system is a form of malicious use aiming to cause harm by admitting irregular petitions, which would violate legal process integrity (a breach of obligations under applicable law). However, the system's defenses have so far prevented the execution of these malicious commands, and no actual harm has been reported. Thus, the event describes a credible risk of harm due to AI misuse but not a realized harm. According to the definitions, this fits the criteria for an AI Hazard rather than an AI Incident or Complementary Information.

The article explicitly mentions the use and manipulation of an AI system (Logos) in judicial processes, with malicious actors using prompt injection to alter AI behavior and influence case outcomes. This manipulation has led to fraudulent alterations in legal documents and processes, which constitutes a violation of legal rights and undermines the justice system. The harm is realized, not just potential, as investigations are underway for criminal conduct and fraud. Hence, the event meets the criteria for an AI Incident due to direct involvement of AI system misuse causing harm to legal rights and the judicial process.

An AI system is explicitly involved as the court uses AI-based automated mechanisms to analyze legal petitions. The event concerns the misuse of AI (prompt injection) to manipulate these systems, which is a use-related issue. Although no actual harm occurred due to security measures, the attempts aimed to circumvent legal procedural rules, which if successful would constitute a violation of legal rights and harm the judicial process. Since the harm was not realized but the misuse attempts are credible and directly linked to AI system manipulation, this qualifies as an AI Hazard rather than an AI Incident. The article focuses on the investigation and the potential risks rather than reporting actual harm caused by the AI misuse.

The event involves the use and attempted misuse of an AI system (the tribunal's electronic petition system with AI filters). The prompt injection attempts to manipulate the AI system to bypass legal admissibility filters, which could lead to violations of legal process integrity and potentially harm the justice system if successful. However, since the system's safeguards prevented the execution of these commands and no harm has been reported, the event is best classified as an AI Hazard, reflecting a credible risk of harm that has not materialized yet.

An AI system is explicitly involved as the court's electronic petition system uses AI filters to assess admissibility of documents. The misuse of AI via prompt injection to bypass these filters constitutes a misuse of the AI system. Although no harm such as successful fraud or legal violations has been confirmed yet, the investigation indicates that such misuse could plausibly lead to violations of legal process integrity and potentially harm the justice system. Since the harm is not yet realized but plausible, this event qualifies as an AI Hazard rather than an AI Incident. The article focuses on the potential for harm and the response to it, not on a realized harm event.

The article explicitly involves an AI system (STJ Logos) and discusses attempts to manipulate it via prompt injection, which is a misuse of the AI system. However, the system's security measures have prevented these attempts from succeeding, and no harm has been reported as having occurred. The investigation and protective measures indicate a credible risk that such manipulation could lead to procedural fraud or other harms if successful. Since no actual harm has materialized yet, but plausible future harm exists, this event fits the definition of an AI Hazard rather than an AI Incident. It is not merely complementary information because the main focus is on the investigation of potential misuse and risk, not on responses to past incidents or general AI ecosystem updates.