Configuring LLM Controls - Barndoor Documentation

Overview

Once your LLM Gateway is set up — see Using the LLM Gateway — the Controls group of LLM Management is where admins govern how the people and services in their organization actually consume it: capping spend, throttling busy callers, and restricting which models a given role can use.

The Controls group of LLM Management has three sections:

Per-request usage and cost observability lives separately, under Reporting → LLM Usage Dashboard.

Before You Begin

You’ll need:

• A Barndoor account with admin privileges.
• An LLM Gateway that’s already configured with at least one provider and one route — see Using the LLM Gateway.
• Spending budgets work out of the box — Barndoor costs requests against its managed default pricing catalog automatically. Set your own per-model pricing under LLM Management → Model Pricing only when you want to override those defaults. See Managing Model Pricing for the full guide.

How Limits Compose

When a caller sends a request to the gateway, Barndoor evaluates policies in a fixed order. The first one that denies stops the chain and returns an HTTP error.

• Counters are debited after the upstream call returns, using actual prompt + completion token counts. The pre-check is what enforces the limit; the post-debit is what keeps the counter honest.
• Warn-only budgets never deny. They still record usage and fire alerts at their configured thresholds, but the request goes through.
• Multiple policies can apply at once. If a user is covered by both a user-scoped budget and an org-scoped budget, both are pre-checked and both are debited; the most restrictive one is what denies.
• Target-bound budgets are the exception to this ordering. A budget pinned to a Target is evaluated after model resolution, per route — it can remove an exhausted target and let the request fail over rather than denying outright.

Token Budgets

Budgets cap how much a scope is allowed to consume in a rolling window — daily, weekly, or monthly. They can cap tokens, spending, or both (whichever hits its limit first denies the request), and can optionally apply to just one slice of traffic via a Target.

Once you have more than a handful, the list supports search, pagination, and sorting — by usage (percent of limit consumed, the default so near-exhausted budgets float to the top), period, or name.

Creating a budget

1. Open LLM Management → Budgets and click Create Budget

The dialog opens with sensible defaults; fill in the fields below.

2. Name and scope

   • Name — anything short and descriptive. The name surfaces in the denial message the caller sees, so make it identifiable (for example Engineering monthly or Sales-team daily cap).
   • Scope — pick Org to cover everyone, or Group, Role, or User to target a slice. Picking a scope without choosing a specific entity means “every entity of that type in this org”.

3. Period and action

   • Period — Daily, Weekly, or Monthly. Counters reset at the start of each period.
   • Action when exhausted — choose Block Requests (default) to return 429 once the limit is hit, or Warn Only to keep accepting requests and just fire alerts at the threshold percentages.

4. Target (optional) By default a budget counts all traffic that matches its scope. Set a Target to narrow it to a single routing dimension instead:

   Target	Counts only…
   All targets (default)	Everything in scope — no narrowing.
   LLM Provider	Requests served by one specific provider connection.
   Upstream Model	One specific upstream model name (for example gpt-4o).
   Model Route	One specific client-facing route name (the value callers put in model).

5. Limits

   • Token limit — total prompt + completion tokens allowed in the period.
   • Spending limit (optional) — a dollar cap. Costed from per-model pricing, which Barndoor’s default catalog supplies automatically (override it under Managing Model Pricing).
   • Alert thresholds — percentages at which Barndoor emits an alert (default 80, 90).

6. Save The new budget appears in the tab with a progress bar. Status refreshes roughly every 30 seconds.

How budgets are counted

• Tokens — prompt + completion tokens reported by the upstream provider on every request, summed per (budget, period).
• Spending — for each request, Barndoor multiplies prompt and completion tokens by the input and output prices in LLM Management → Model Pricing for that model.
• Target — a target-bound budget only counts requests that match its Target dimension; an All targets budget counts every request in scope.
• Reset — at the start of the next period (next day at 00:00 UTC for daily, next Monday for weekly, the 1st for monthly).

When a caller hits the limit

A blocking budget returns:

HTTP/1.1 429 Too Many Requests
content-type: application/json

{
  "error": {
    "message": "Token monthly budget exhausted (budget: Engineering monthly) (100% used: 1001234 / 1000000 tokens).",
    "type": "budget_exhausted",
    "code": null
  }
}

Spending budgets use the same shape with "Spending …" in the message.

Rate Limits

Rate limits throttle a scope on a rolling 60-second window. They’re the right tool when you want to smooth bursts and prevent a single noisy caller from drowning out everyone else. (Budgets are the right tool when you want to control total spend over a day, week, or month.)

Creating a rate limit

1. Open LLM Management → Rate Limits and click Create Rate Limit

The dialog appears with the same scope picker pattern as Budgets.

2. Name and scope

   • Name — appears in the denial message the caller sees.
   • Scope — Org, Group, Role, User, API Key, or Model. Selecting Model lets you cap a particular alias across all users.

3. Limits Fill in at least one of:

   • Requests per minute — caps the number of inbound requests, regardless of size.
   • Tokens per minute — caps the sum of prompt + completion tokens over the same window.

You can set both, or only one. The form rejects the policy if both are blank.

4. Save The new policy appears in the table with live RPM and TPM bars. Status refreshes roughly every 10 seconds.

When a caller hits the limit

HTTP/1.1 429 Too Many Requests
retry-after: 60
content-type: application/json

{
  "error": {
    "message": "Rate limit exceeded (policy: Sales-team rate cap). Try again in 60 seconds.",
    "type": "rate_limit_error",
    "code": null
  }
}

Model Access

Model access policies decide whether a caller is allowed to invoke a particular model at all. They support both allowlists (“only let this scope call these models”) and denylists (“this scope must not call these models”).

Creating a policy

1. Open LLM Management → Model Access and click Create Policy

Pick Allowlist for “only these models” or Denylist for “everything except these models”.

2. Name and scope

   • Name — appears in the denial message.
   • Scope — Org, Group, Role, User, or API Key.

3. Add at least one target Each target is one rule about which models the policy covers. You can mix and match any number of these:

   Target kind	Matches when…	Example
   Model route	The caller’s model field matches a route you’ve defined in Model Routes. Supports trailing * wildcard.	claude-*
   Upstream model	The resolved upstream model name (the string the gateway sends to the provider).	gpt-4o
   Provider	Any model served by the selected provider.	openai-prod
   Provider + model	A specific provider/upstream-model pair.	azure-east + gpt-4o-mini

4. Save The new policy appears in the table. Enable / disable any policy from the table without deleting it.

How allowlists and denylists combine

• If any allowlist applies to a caller, at least one of its targets must match — otherwise the call is denied.
• If any denylist applies, none of its targets may match — otherwise the call is denied.
• A caller with both an allowlist and a denylist must satisfy both.
• A caller with neither has unrestricted access (subject to budgets and rate limits).

When a caller is denied

HTTP/1.1 403 Forbidden
content-type: application/json

{
  "error": {
    "message": "Model 'claude-opus-4-8' is blocked by access policy: External-models-only",
    "type": "permission_error",
    "code": null
  }
}

What callers see when something is denied

Troubleshooting

A user keeps getting 429s — which policy is responsible? Look at the error.message in the response — Barndoor names the policy that denied the request ("… (policy: Sales-team rate cap)" or "… (budget: Engineering monthly)"). Open that policy in the relevant tab to inspect the configuration and the live usage bars.

Frequently Asked Questions

When should I use a budget vs a rate limit? Use a budget when you care about total consumption over a long horizon (a team’s monthly spend, a project’s daily token allowance). Use a rate limit when you care about short-term load (smoothing bursts, protecting upstream providers from a runaway loop). They compose naturally: a budget for how much, a rate limit for how fast.