Israeli Cybersecurity Firm Launches AI Agent Security Challenge

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

Israeli cybersecurity company Cato Networks launched the AgentSense Challenge, offering $30,000 to participants who can successfully prompt an autonomous AI agent to deviate from its assigned task. The initiative aims to identify vulnerabilities in AI agents and improve real-time security monitoring, addressing potential risks of malicious prompt injection and unauthorized actions.[AI generated]

Why's our monitor labelling this an incident or hazard?

The event involves an AI system (an AI agent performing email summarization and task execution) and focuses on a security challenge to cause it to deviate from its intended task, which could lead to harm such as incorrect customer communications or data mishandling. No actual harm is reported yet, but the plausible risk of harm is clearly articulated and central to the event. The challenge is designed to uncover vulnerabilities that could lead to incidents if exploited. Hence, it fits the definition of an AI Hazard rather than an Incident or Complementary Information.[AI generated]
AI principles
Robustness & digital securitySafety

Industries
Digital security

Business function:
ICT management and information security

AI system task:
Goal-driven organisation


Articles about this incident or hazard