NeMo Guardrails

NeMo Guardrails is an open-source toolkit developed by NVIDIA for adding programmable safety and control mechanisms to large language model (LLM) based conversational applications. It enables developers to define explicit rules, called "rails," that govern what an LLM powered system can discuss, how it should respond, and what actions it can take, addressing risks such as jailbreaks, prompt injection, hallucination, and off topic or unsafe outputs. The toolkit is model agnostic, working with LLMs such as GPT-4, LLaMA, Falcon, and others, and is released under the Apache 2.0 license with an accompanying peer reviewed paper presented at EMNLP 2023.

The library inserts a guardrails layer between the application and the LLM, intercepting interactions at five points: input (filtering or altering user messages), dialog (steering conversational flow), retrieval (checking retrieved content in RAG pipelines), execution (validating tool or action calls), and output (filtering or altering the model's final response). Rules are authored using a purpose built modeling language called Colang and activated through a YAML configuration file, with a pre built library covering common checks like fact checking, hallucination detection, topic restriction, and jailbreak detection. Developers integrate the toolkit via a Python API or a standalone guardrail server and can validate configurations using a built-in evaluation tool and vulnerability scanning reports that measure how effectively a given setup resists known LLM safety risks.

Auto-discovered on 2026-02-11 by OECD Catalogue Automation