Skip to content

Cost Control Rollout — 2026-07-24

Trigger: $46 additional unexpected spend on top of $155.76/3-day burn. Root cause: Unlimited LiteLLM key + GPT-5.6 Terra as default gateway model with no enforced budget, session-size, or model-allowlist guardrails.

Changes Deployed

1. LiteLLM — New constrained team + key

  • Team hermes-normal-ops: $3.33 rolling daily cap + $100 rolling 30-day cap, 30 RPM, 250K TPM, max 3 parallel requests.
  • Key hermes-normal-ops-daily-capped (...I3gQ): Hard $3.33/day, model-restricted to approved list only.
  • Allowed models: claude-sonnet-5, deepseek-v4-pro, deepseek-v4-flash, glm-5.2, MiniMax-M3, qwen3.7-plus.
  • Terra/GPT-5.6/Claude-Opus-4: explicitly excluded from this key — LiteLLM returns HTTP 403.
  • Legacy key sk-...itzA: blocked (blocked=t in DB).

2. Hermes config — Default routing

  • Conductor: claude-sonnet-5 (admin-ai/LiteLLM proxy).
  • Fallbacks: deepseek-v4-pro → deepseek-v4-flash (admin-ai only).
  • Delegation/workers: deepseek-v4-pro, fallback deepseek-v4-flash.
  • No automatic escalation to premium — failure stops, not silently upgrades.

3. Session controls

  • Context length: 128k tokens hard ceiling.
  • Compression: enabled at 50% fill, targets 20% ratio.
  • Max turns: 50 per session (prevents unbounded tool-call marathons).

4. Rate limits (on LiteLLM key)

  • 30 RPM, 250K TPM, max 3 concurrent requests.
  • 429 throttle-backoff confirmed working in live logs.

Verification

  • Sonnet makes calls through new key: HTTP 200.
  • Terra through new key: HTTP 403 (blocked).
  • Old key blocked in DB: blocked = t.
  • Live gateway confirmed routing through admin-ai at https://admin-ai.itpropartner.com/v1/.

What's still behavioral (not enforced)

  • Model compliance is enforced at the proxy key layer. Cost caps are enforced at the key+team layer. Session size is a Hermes config setting — stickiness depends on the runtime respecting it.