State-of-the-art Foundation AI Models Should be Accompanied by Detection Mechanisms as a Condition of Public Release

Foundation models represent a dramatic advance for the state of the art in Artificial Intelligence (AI). In current discussions of AI, a foundation model is defined very generally as an AI model that is trained on large amounts of data, typically using self-supervision, that can be adapted, or ‘fine-tuned’ to a wide range of downstream tasks (see, e.g., Bommasani et al., 2022).1 In this paper, we will argue for a specific regulatory mechanism that governs the release of new state-of-the-art foundation models. For concreteness, our arguments will sometimes make reference to a central ingredient in many current foundation models, namely large language models (LLMs), which have the ability to generate natural language text as output. Many LLMs are foundation models in their own right: for instance, BERT and GPT-3 are LLMs and also canonical examples of foundation models. The discussions in this paper will sometimes refer to LLMs and the text they generate, to give concrete examples of the content that foundation models can produce and the issues that arise for these models. Our broad argument is about foundation models generally, not just about LLMs. But we will begin by introducing LLMs and then show how LLMs can provide the core of foundation models with wider functionality.