AI Surveillance and Privacy Concerns: Pegasus Spyware and Apple's Child Safety Features

Thumbnail Image

The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.

AI-powered Pegasus spyware, sold by Israeli firm NSO Group, enabled covert surveillance of mobile users in 50 countries, breaching privacy and data security. Separately, Apple's AI-based child safety features for iCloud photos sparked global criticism over potential privacy violations and government misuse, with thousands of organizations and experts urging Apple to abandon the system.[AI generated]

Why's our monitor labelling this an incident or hazard?

The event involves the use of an AI system explicitly described as scanning images on users' devices to detect illegal CSAM content by matching hashes against known databases. The system's deployment and operation directly impact human rights, specifically privacy rights, and legal obligations to prevent child exploitation. Although the system aims to prevent harm (child sexual abuse), it also raises significant concerns about privacy violations and potential misuse for surveillance, which are violations of fundamental rights. Since the system is actively used and has led to reports to law enforcement, this constitutes an AI Incident due to direct involvement of AI in actions affecting human rights and legal compliance. The concerns about misuse and privacy infringement further support the classification as an incident rather than a mere hazard or complementary information.[AI generated]
AI principles
Privacy & data governanceTransparency & explainabilityAccountabilityRespect of human rightsDemocracy & human autonomy

Industries
Digital securityGovernment, security, and defenceConsumer services

Affected stakeholders
General publicConsumers

Harm types
Human or fundamental rightsPublic interest

AI system task:
OtherRecognition/object detection


Articles about this incident or hazard