Guardrail

A guardrail is a control placed around a language model to keep its inputs and outputs inside defined bounds. Guardrails sit outside the model rather than inside its weights, which distinguishes them from alignment training. A typical setup checks the incoming prompt for prompt injection, jailbreak patterns, or personal data, then checks the generated response for policy violations, leaked secrets, unsafe instructions, or off-topic content, and blocks, rewrites, or escalates when a rule fires. Implementations range from regular expressions and allowlists to classifier models and a second model asked to judge the first. NVIDIA's NeMo Guardrails, Guardrails AI, and Llama Guard are open options, and the major providers ship moderation endpoints alongside their APIs. For agents, guardrails also constrain actions: which tools may run, what data a query may touch, and when a human must approve. The balance is the hard part. Filters tuned too tightly refuse legitimate requests such as medical questions or security research, while loose ones let harmful output through. Guardrails also fail quietly against novel attacks, so they belong in a layered defence with logging, evaluation, and human review rather than standing alone.