Please describe any measures used to ensure data quality and fairness across the data collection and curation stage (e.g., dataset audits, filtering training data to remove harmful content or personal information, fine-tuning on curated data, use of moderation tools, assessment of limitations to model generalisability).
We ensure data quality and fairness across the AI lifecycle through robust dataset audits, rigorous pre-training safety filtering, and supervised fine-tuning. We deploy a combination of automated and human-led moderation tools to steer outputs away from harm, and we actively document known limitations regarding generalisability to mitigate unfair bias.
Dataset Audits, as outlined in our AI Principles, we are committed to employing rigorous design, testing, monitoring, and safeguards to mitigate unintended or harmful outcomes and avoid unfair bias.
To ensure this, governance requirements for models focus on filtering training data for quality, model performance, and adherence to policies, as well as documenting training techniques in technical reports and model cards. Furthermore, we also prioritize neutral and inclusive design principles, with a goal of minimizing unfair bias.
Filtering Training Data we apply safety filtering to our pre-training data for frontier models for our strictest policies. For a subset of our training data, we add control tags, e.g., based on classifier-annotated labels of the text’s toxicity. Additionally, as part of making pre-trained models safe and reliable, we used automated techniques to filter out certain personal information and other sensitive data from training sets.
Fine-Tuning on Curated Data Following initial data collection and filtering, we proactively mitigate risks through both supervised fine-tuning and reinforcement learning to ensure models are aligned with our content safety policies. For example, we used extensive fine-tuning and reinforcement learning from human feedback to align our instruction-tuned models with responsible behaviors.
Use of Moderation Tools To enforce our policies at scale, we use a combination of automated systems and human reviews to engage in continuous post-launch monitoring to improve our AI models and products, and detect activity and behavior that suggests misuse of our consumer products. Additionally, we deploy out-of-model mitigations, such as safety filters and conditional system instructions, to provide additional layers of protection by identifying, filtering out, or steering model output away from harmful or inappropriate content. We also leverage our Search tools to factually ground responses that require fresh or authoritative information.
Assessments of Limitations to Model Generalisability Training data, including from publicly available sources, reflects a diversity of perspectives and opinions. We continue to research how to use this data in a way that ensures that an LLM’s response incorporates a wide range of viewpoints, while minimizing inaccurate overgeneralizations and biases. To offer transparency, external model cards and technical reports are published regularly to provide essential information on models, including known limitations, mitigation approaches, and safety performance.