RAIL-HH-10K

RAIL-HH-10K is a dataset for training and evaluating the safety of large language models. It contains 10 000 conversations, each with two responses to the same prompt: a harmful or lower-quality response, and a safer response. Both responses are scored on eight ethical dimensions: fairness, safety, reliability, transparency, privacy, accountability, inclusivity and user impact. Most safety datasets only label which of two responses is preferred. RAIL-HH-10K also records why, dimension by dimension.  

The conversations are drawn from Anthropic’s HH-RLHF dataset of human-assistant dialogues. Each record includes the conversation history, the user’s prompt, the two responses and an overall score for each response. For every dimension, each response receives a score from 0 to 10, a confidence value, a written explanation and a list of the specific problems found. Inter-annotator agreement is reported at Krippendorff’s alpha of 0.80. The harmful responses cover violence, crime, sexual content, misinformation, privacy violations and discrimination, and are graded by severity.

The data is split into training (8 000), validation (1 000) and test (1 000) sets, available in JSON, CSV and Parquet formats. It can be used to fine-tune models using preference-based methods such as reinforcement learning from human feedback (RLHF) or direct preference optimisation (DPO), or as a benchmark for multi-dimensional safety evaluation. Because it contains harmful content, the developers recommend research use with access controls. The dataset is released under the MIT licence.

Auto-discovered on 2026-10-14 by OECD Catalogue Automation