Virtual Keys, RBAC & Per-Team Budgets

Every lesson since Module 3 has used a virtual key without formalizing what it actually models: role-based access control, applied to LLM traffic. This lesson makes that model explicit, teams, roles, and per-role model and budget scopes, and shows how LiteLLM’s team/key hierarchy expresses it directly in config, closing the security gap from Lesson 1 where “some service is sending customer data to a provider nobody approved for that data” had no chokepoint to be caught at.

Why Raw Provider Keys Never Leave the Gateway

The pattern established since Lesson 7 bears restating precisely: real provider credentials (OPENAI_API_KEY, ANTHROPIC_API_KEY) live only in the gateway’s own environment or secrets manager, referenced from config.yaml via os.environ/.... No application, service, or end user ever holds one. What they hold instead is a virtual key, sk-... issued by the gateway itself, that the gateway resolves internally to a real credential only at the adapter stage (Lesson 6), after routing has already decided which provider to call. This single indirection is what makes everything in this lesson possible: a virtual key can be scoped, budgeted, and revoked independently of the real credential it ultimately maps to, and revoking one leaked virtual key never breaks any other caller.

Modeling Teams and Roles

flowchart TD
    Org["Organization: acme-corp"] --> TeamA["Team: fraud-detection"]
    Org --> TeamB["Team: support-bot"]
    TeamA --> RoleEng["Role: internal-eng\nmodels: [claude-sonnet, gpt-4o, self-hosted-llama]\nbudget: $500/mo"]
    TeamA --> RoleContractor["Role: contractor\nmodels: [self-hosted-llama]\nbudget: $20/mo"]
    TeamB --> RoleBot["Role: production-service\nmodels: [claude-sonnet]\nbudget: $2000/mo"]

    style Org fill:#4338CA,stroke:#4338CA,color:#ffffff,rx:8px,ry:8px
    style TeamA fill:#EEF0F7,stroke:#6366F1,color:#0F172A,rx:8px,ry:8px
    style TeamB fill:#EEF0F7,stroke:#6366F1,color:#0F172A,rx:8px,ry:8px
    style RoleEng fill:#F7F8FC,stroke:#0D9488,color:#0F172A,rx:8px,ry:8px
    style RoleContractor fill:#F7F8FC,stroke:#0D9488,color:#0F172A,rx:8px,ry:8px
    style RoleBot fill:#F7F8FC,stroke:#0D9488,color:#0F172A,rx:8px,ry:8px

LiteLLM expresses this with teams as a first-class object, and keys issued under a team inherit that team’s constraints unless further scoped down:

# Create a team with a shared budget ceiling and default allowed models
curl -X POST http://localhost:4000/team/new \
  -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" \
  -d '{
    "team_alias": "fraud-detection",
    "models": ["claude-sonnet", "gpt-4o", "self-hosted-llama"],
    "max_budget": 500
  }'

# A contractor's key, issued under that team but scoped down further:
# restricted to the cheap self-hosted model only, with its own tighter budget
curl -X POST http://localhost:4000/key/generate \
  -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" \
  -d '{
    "team_id": "fraud-detection",
    "models": ["self-hosted-llama"],
    "max_budget": 20,
    "metadata": {"role": "contractor"}
  }'

A key’s effective permissions are the intersection of its own models/max_budget and its team’s: a contractor key can never exceed what the fraud-detection team itself is scoped to, even if someone mistakenly tried to grant it broader access, since the team-level constraint still applies on top.

Enforcement in Practice

from openai import OpenAI

# Contractor's key: scoped to self-hosted-llama only
contractor_client = OpenAI(api_key="sk-contractor-key-...", base_url="http://localhost:4000")

# Succeeds: within this key's allowed models
contractor_client.chat.completions.create(
    model="self-hosted-llama",
    messages=[{"role": "user", "content": "Summarize this log line."}],
)

# Rejected with a 403 before any routing happens: claude-sonnet is outside
# this key's scope, exactly the enforcement point from Lesson 4's auth stage
contractor_client.chat.completions.create(
    model="claude-sonnet",
    messages=[{"role": "user", "content": "Draft a customer-facing apology email."}],
)

This is the concrete mechanism that closes Lesson 1’s fourth failure mode: a policy like “contractors may only use the self-hosted model, never a frontier model with customer data” is not a document someone has to remember and enforce by hand, it is a scope set once at key issuance and enforced automatically on every single request, with no application code needing to know the policy exists at all. Lesson 20 adds a complementary control that works even within an allowed model: catching and redacting PII in the request itself, regardless of which model or team is calling.