Initiative overview
The report distinguishes between two broad classes of AI systems: predictive AI (PredAI), which still dominates industrial applications, and generative AI (GenAI), whose adoption in business and consumer contexts has increased rapidly. Real-world failures documented in the report include autonomous vehicles caused to swerve into oncoming lanes and stop signs misclassified as speed limit signs, illustrating the tangible stakes of AML vulnerabilities.
The taxonomy classifies attacks across five dimensions: the type of AI system targeted; the stage of the machine learning life cycle at which the attack occurs (from design and training through to deployment); the attacker's goals in terms of which system properties they seek to violate (availability, integrity, or privacy); the attacker's capabilities and access; and the attacker's knowledge of the learning process. For predictive AI, the taxonomy covers evasion, poisoning, and privacy attacks. For generative AI, it extends to supply chain attacks, direct and indirect prompt injection, misuse violations, and security of AI agents. Each category of attack is paired with corresponding mitigation strategies, along with an assessment of the limitations of those techniques.
The report is the product of an extensive literature review, expert consultation in the field of adversarial machine learning, and original research by its authors, drawn from NIST's Computer Security Division, Northeastern University, the U.S. AI Safety Institute, Cisco, and the U.K. AI Security Institute. It adopts the security, resilience, and robustness concepts from the NIST AI Risk Management Framework, and aligns with the NCSC Machine Learning Principles. While the guidance is voluntary and not intended to supersede existing regulations or law, it is explicitly designed to inform future standards and practice guides. Its primary audience includes those responsible for designing, developing, deploying, evaluating, and governing AI systems.
