Project GuardRail

AI/ML applications have unique security threats. Project GuardRail is a set of security and privacy requirements that AI/ML applications should meet during their design phase that serve as guardrails against these threats. These requirements help scope the threats such applications must be protected against. It consists of a baseline set required for all AI/ML applications, and two additional set of requirements that are specific to continuously learning models and user interacting models respectively. There are four additional questions that are specific to generative AI applications only.

The content of this library comes from a variety of frameworks, lists, and sources, both from academia and industry. We have performed several iterations to refine the library to accurately determine the scope and language of the questions. 

For every application, security and privacy threat models are conducted as usual.

As shown in the diagram below, the "Questionnaire for Manual Threat Modeling" defines the library. The 53 threats (and 4 additional generative AI threats) are divided into three categories as shown.

• All AI/ML applications must meet the 28 baseline requirements.
• If an application is continuously learning, they must meet 6 additional requirements apart from baseline.
• If they EITHER train on user data OR interact with users, they must meet 19 additional requirements apart from baseline.

Generative AI questions are differentiated and put into a separate group under each category if applicable.