Robustness, security and safety (Principle 1.4)
AI systems must function in a robust, secure and safe way throughout their lifetimes, and potential risks should be continually assessed and managed.
- AI systems should be robust, secure and safe throughout their entire lifecycle so that, in conditions of normal use, foreseeable use or misuse, or other adverse conditions, they function appropriately and do not pose unreasonable safety and/or security risks.
- Mechanisms should be in place, as appropriate, to ensure that if AI systems risk causing undue harm or exhibit undesired behaviour, they can be overridden, repaired, and/or decommissioned safely as needed.
- Mechanisms should also, where technically feasible, be in place to bolster information integrity while ensuring respect for freedom of expression.
From the AI Wonk
Rationale for this principle
Addressing the safety and security challenges of complex AI systems is critical to fostering trust in AI. In this context, robustness signifies the ability to withstand or overcome adverse conditions, including digital security risks. This principle further states that AI systems should not pose unreasonable safety risks including to physical security, in conditions of normal or foreseeable use or misuse throughout their lifecycle. Existing laws and regulations in areas such as consumer protection already identify what constitutes unreasonable safety risks. Governments, in consultation with stakeholders, must determine to what extent they apply to AI systems.
AI actors can employ a risk management approach (see below) to identify and protect against foreseeable misuse, as well as against risks associated with use of AI systems for purposes other than those for which they were originally designed. Issues of robustness, security and safety of AI are interlinked. For example, digital security can affect the safety of connected products such as automobiles and home appliances if risks are not appropriately managed.
The Recommendation highlights two ways to maintain robust, safe and secure AI systems:
- traceability and subsequent analysis and inquiry, and
- applying a risk management approach.
like explainability (see 1.3), traceability can help analysis and inquiry into the outcomes of an AI system and is a way to promote accountability. Traceability differs from explainability in that the focus is on maintaining records of data characteristics, such as metadata, data sources and data cleaning, but not necessarily the data themselves. In this, traceability can help to understand outcomes, to prevent future mistakes, and to improve the trustworthiness of the AI system.
Risk management approach: The Recommendation recognises the potential risks that AI systems pose to human rights, bodily integrity, privacy, fairness, equality and robustness. It further recognises the costs of protecting from these risks, including by building transparency, accountability, safety and security into AI systems. It also recognises that different uses of AI present different risks, and some risks require a higher standard of prevention or mitigation than others.
A risk management approach, applied throughout the AI system lifecycle, can help to identify, assess, prioritise and mitigate potential risks that can adversely affect a system’s behaviour and outcomes. Other OECD standards on risk management, for example in the context of digital security risk management and risk-based due diligence under the MNE Guidelines and OECD Due Diligence Guidance for Responsible Business Conduct, may offer useful guidance1. Documenting risk management decisions made at each lifecycle phase can contribute to the implementation of the other principles of transparency (1.3) and accountability (1.5).
Other principles
Inclusive growth, sustainable development and well-being
Human rights and democratic values, including fairness and privacy
Transparency and explainability
Robustness, security and safety
Accountability
Investing in AI research and development
Fostering an inclusive AI-enabling ecosystem
Shaping an enabling interoperable governance and policy environment for AI
Building human capacity and preparing for labour market transition
