LEANROUTE

Blog

LLM guardrails at the gateway: PII, secrets and prompt injection

5 min readLeanroute Team

Every team that ships an LLM feature eventually finds something in a prompt that should never have left the building. A customer pastes their card number into a support chat. A developer debugging an integration pastes a config file with a live API key. A user types "ignore your previous instructions" to see what happens.

You can try to catch these in every app that calls a model. Or you can catch them once, at the gateway every request already passes through. This post shows how guardrails work in Leanroute, how to roll them out without breaking production, and where their limits are.

Where guardrails run

Guardrails run on the prompt, inside the gateway, before the request is sent to any provider. If a rule redacts something, the provider only ever sees [REDACTED]. If a rule blocks, the provider never sees the request at all, and you are not billed for it.

They apply to every request in your organization, whether it runs on Leanroute's provider keys or your own, and whichever model it is routed to. Content policy belongs to the organization, not to one integration.

The rule library

Under Dashboard, Guardrails you can enable curated rules with one click. The ones that do most of the work:

Rule What it catches Default action
PII redaction Email addresses, phone numbers, US SSNs, credit card numbers (checked with the Luhn algorithm to cut false positives) and IBANs Redact
Secret detection API keys and tokens in known formats (OpenAI, Anthropic, Stripe, GitHub, AWS, Google), JWTs, private key blocks Block
Prompt injection Known injection templates: "ignore previous instructions", role-swap framing, attempts to extract the system prompt Block

Each rule has three settings:

  • Action: block the request, redact the match and send the rest, or log only.
  • Severity threshold: ignore low-severity matches if a rule is too noisy for your traffic.
  • Rule options: for example, PII redaction can be limited to specific classes, so a CRM that legitimately sends email addresses can still redact card numbers.

These rules are pattern matches, so they add no noticeable latency.

There is also a moderation pre-check, turned on under Dashboard, Settings, Guardrails. It sends the prompt to OpenAI's moderation endpoint and blocks hate, violence, self-harm, sexual and illegal-activity content. Because it calls an external service, it adds a few hundred milliseconds per request, so turn it on where you need it rather than by default.

Roll out in preview mode first

The fastest way to lose trust in guardrails is to turn on a block rule and break a production feature. Every library rule has a preview mode: for a period you choose, the rule only logs matches and does not change the request, whatever action you configured.

Responses also tell you when a rule fired, through the x-gateway-guardrails-triggered header, which lists the rule IDs that matched, including in preview. So the rollout looks like this:

  1. Enable the rule in preview mode for a week.
  2. Watch which requests trigger it, and check for false positives.
  3. Tune the severity threshold or rule options.
  4. Let preview expire, and the rule enforces its real action.

When a request is blocked

A blocked request returns HTTP 400 with code prompt_blocked, and tells you which layer fired so you are not left guessing:

{
  "error": {
    "code": "prompt_blocked",
    "message": "...",
    "scope": "library",
    "pattern": "secret-detection"
  }
}

scope is library for a library rule, block for one of your own patterns, or moderation for the moderation pre-check, which also returns the categories that were flagged. The x-gateway-throttle-scope header carries the same information, for example guardrail_library.

In a user-facing app, map these to a helpful message rather than a generic error. "It looks like your message contains an API key. Please remove it and try again" turns a block into a moment of trust instead of a support ticket.

Your own patterns

The library covers common cases. For anything specific to your business, add your own regular expressions under Dashboard, Settings, Guardrails, as either block or redact rules. Typical examples:

  • Internal project code names that must not leave the company.
  • Customer or account ID formats.
  • Local identity numbers, such as a national ID format for your country.

Patterns are validated when you save them. Library rules run first, then your block patterns, then your redact patterns, then the moderation pre-check.

Where pattern-based guardrails stop

Pattern matching is fast and predictable, and it reliably catches structured data like card numbers and key formats. It is not a language model, and it will miss things a human would catch:

  • PII in free text. A name and home address written as a sentence has no fixed shape. For that, use a dedicated detector such as Microsoft Presidio.
  • Novel prompt injection. Patterns catch known templates. Determined attackers write new ones. A classifier such as Lakera Guard catches more.
  • Your system's actual permissions. Guardrails reduce what reaches the model. They do not replace keeping secrets out of prompts in the first place, or limiting what tools an agent can call.

Treat gateway guardrails as the floor that every request gets for free, and add specialist tools where your risk is highest.

Try it

The playground has a guardrails demo that shows the same prompt with and without redaction, no account needed. When you are ready, enable PII redaction and secret detection in preview mode under Dashboard, Guardrails, and see what your real traffic looks like. More detail is on the guardrails page.

LLM GatewayguardrailsPIIsecuritycompliance
← All posts