Newer AI models missed more payment fraud in Coinbase's benchmark
2026-10-10
CryptoSlate
The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.
Coinbase's benchmark tests revealed that newer versions of three major AI models used for payment fraud detection in its Onramp service identified fewer fraudulent transactions and a smaller share of fraud value compared to previous versions, raising concerns about the effectiveness of AI model upgrades in financial security.[AI generated]
Why's our monitor labelling this an incident or hazard?
The event involves AI systems (fraud detection models including LLMs) used in payment screening, which is a clear AI system involvement. The use of these AI models in screening led to fewer fraud cases being detected, which directly or indirectly causes harm by allowing fraudulent transactions to go through, potentially causing financial losses to customers or the company. This fits the harm category (a) injury or harm to persons or groups, here financial harm to customers, and (d) harm to communities or property (financial property). Although the article notes that actual customer losses were not established in the test, the reduced detection rates in a realistic replay strongly imply harm. Therefore, this is an AI Incident. The event is not merely a potential hazard or complementary information, as the harm is realized in the evaluation context. It is not unrelated or beneficial use, as the AI system's failure to detect fraud is the source of harm.[AI generated]