diff --git a/docs/infrastructure/model-chain.md b/docs/infrastructure/model-chain.md index db0c901..5e2e94c 100644 --- a/docs/infrastructure/model-chain.md +++ b/docs/infrastructure/model-chain.md @@ -1,6 +1,6 @@ # AI Model Architecture — IT Pro Partner -**Updated:** August 17, 2026 +**Updated:** September 11, 2026 Two separate concepts: **fallback chain** (survival — direct API keys) and **operational chain** (daily toolbox — admin-ai only). The two-key strategy means operational keys run through admin-ai/LiteLLM; fallback keys are direct provider API keys with daily limits. @@ -14,9 +14,9 @@ Survives admin-ai outage. All direct provider keys have daily caps. Fires in ord | Tier | Model | Provider | Key type | Daily cap | |---|---|---|---|---| -| **Primary** | `deepseek-v4-pro` | `admin-ai` | operational | $30/mo budget | +| **Primary** | `deepseek-v4-flash` | `admin-ai` | operational | $700/30d (~$23.33/day) | | **F1** | `deepseek-v4-flash` | `deepseek` (direct) | fallback | $3 | -| **F2** | `gemini-3.6-flash` | `google` (direct) | fallback | $2 | +| **F2** | `gemini-3.8-flash` | `google` (direct) | fallback | $2 | | **F3** | `grok-4.5` | `xai` (direct) | fallback | $2 | | **F4** | `claude-sonnet-5` | `anthropic` (direct) | fallback | $5 | | **F5** | `gpt-5-mini` | `openai` (direct) | fallback | $2 | @@ -31,15 +31,17 @@ All route through admin-ai. Shared budget via `hermes-agent-v5` key. | Role | Model | Provider | Use When | |---|---|---|---| -| **Conductor** | `deepseek-v4-pro` | admin-ai (DeepSeek) | All standard work — orchestration, delegation, coding | -| **Workhorse** | `deepseek-v4-pro` | admin-ai (DeepSeek) | Delegated tasks, scripts, infra code | +| **Conductor** | `deepseek-v4-flash` | admin-ai (DeepSeek) | All standard work — orchestration, delegation, coding | +| **Workhorse** | `deepseek-v4-flash` | admin-ai (DeepSeek) | Delegated tasks, scripts, infra code | | **Batch Workhorse** | `deepseek-v4-flash` | admin-ai (DeepSeek) | Bulk scripts, log parsing, repetitive tasks | | **Lightweight** | `claude-haiku-4-5` | admin-ai (Anthropic) | Email triage, classification, simple tasks | | **Simple Workhorse** | `gpt-5.6-luna` | admin-ai (OpenAI) | Lightweight tasks under 128K context | | **Auditor** | `gpt-5.6-luna` | admin-ai (OpenAI) | Code review, QA (primary auditor) | | **Auditor 2** | `xai/grok-4.5` | admin-ai (xAI) | Second-opinion code review (different provider) | | **Critical** | `claude-sonnet-5` | admin-ai (Anthropic) | Client comms, legal docs, architecture (explicit) | -| **Professional Comms** | `gemini-3.6-flash` | admin-ai (Google) | Client emails, professional messaging | +| **Professional Comms** | `gemini-3.8-flash` | admin-ai (Google) | Client emails, professional messaging | +| **Research** | `sonar-pro` | admin-ai (Perplexity) | Live-search-grounded: competitive/compliance/market research, current events, cited answers | +| **Deep Research** | `sonar-reasoning-pro` | admin-ai (Perplexity) | Multi-step synthesis, reasoning + citations | --- @@ -47,13 +49,13 @@ All route through admin-ai. Shared budget via `hermes-agent-v5` key. ### hermes-agent-v5 (Main — Sho'Nuff) - **Created:** Jul 31, 2026 -- **Budget:** $30/day -- **Spend:** $20.36 (as of Aug 6) +- **Budget:** $23.33/day ($700/30d) +- **Spend:** $184.79 (as of Sep 3) - **Models:** deepseek-v4-pro, deepseek-v4-flash, gemini-flash-latest, claude-sonnet-5, claude-haiku-4-5, gpt-5.6-luna, xai/grok-4.5 (+ claude-sonnet-4-6, claude-opus-4-8, claude-fable-5, gemini-2.5-flash/pro, grok-4.3, gpt-5, gpt-5-mini available) ### Anita's Hermes Key -- **Budget:** $10/day -- **Spend:** $0.11 (as of Aug 6) +- **Budget:** $3.33/day ($100/30d) +- **Spend:** $6.86 (as of Sep 3) - **Models:** deepseek-v4-pro, deepseek-v4-flash, gemini-flash-latest, claude-sonnet-5, claude-haiku-4-5, gpt-5.6-luna, xai/grok-4.5 (+ same expansions as hermes-agent-v5) --- @@ -62,8 +64,8 @@ All route through admin-ai. Shared budget via `hermes-agent-v5` key. | Component | Daily est. | |---|---| -| Conductor + Workhorse (ds-v4-pro) | ~$3.00 | -| Batch Workhorse (ds-v4-flash) | ~$0.50 | +| Conductor + Workhorse (ds-v4-flash) | ~$2.00 | +| Batch Workhorse (ds-v4-flash) | ~$1.50 | | Lightweight (haiku-4-5) | ~$0.30 | | Simple Workhorse (luna) | ~$0.50 | | Auditors (luna + grok-4.5) | ~$0.80 | @@ -80,3 +82,6 @@ All route through admin-ai. Shared budget via `hermes-agent-v5` key. - **Aug 5, 2026:** Admin-ai budget cap hit (~$20). Fallback chain exhausted 4 dead legs, landed on Anthropic direct. Burned $45 in 10 hours on claude-sonnet-5 via direct key. Anthropic key capped until Sep 1. - **Aug 6, 2026:** Root cause of Aug 5 outage: 4 fallback legs dead simultaneously (admin-ai budget, DeepSeek balance $0, grok-4.6 404, Anthropic capped). Implemented two-key strategy (operational vs fallback). Rotated all 5 fallback keys. Added F5 (gpt-5-mini via OpenAI). Added haiku-4-5 and grok-4.5 to operational chain. Fixed grok-4.6 → grok-4.5. Synced Anita profile identically. Budget raised to $30. - **Aug 17, 2026:** Live verification of fallback chain against the LiteLLM model DB (admin-ai). Confirmed present: `deepseek-v4-flash`, `gemini-3.6-flash` (registered as `gemini/gemini-3.6-flash`), `grok-4.5` (`xai/grok-4.5`), `claude-sonnet-5`. Replaced F5 `gpt-4.1-mini` → `gpt-5-mini` — the gpt-4.1 series is no longer present in LiteLLM. Synced `config.yaml` `fallback_providers` and `model.fallbacks`. +- **Sep 3, 2026:** DeepSeek promo rates ended and prices rose again. Live LiteLLM map now bills deepseek-v4-pro at $1.32/$3.96 and deepseek-v4-flash at $0.44/$1.32 (peak; off-peak half). Registered gemini-3.8-flash ($0.75/$3.75 intro) on admin-ai. Swapped F2 and Professional Comms gemini-3.6-flash to gemini-3.8-flash (Core + Anita). Restored delegation default to deepseek-v4-pro (was claude-sonnet-5, ~$13/day burn). Corrected F5 name in operational-models.md. Re-baselined cost tracker thresholds ($3/$4 to $15/$20). Verified actual key budgets: hermes-agent-v5 $700/30d ($23.33/day), Anita $100/30d ($3.33/day). +- **Sep 3, 2026 (sonar + vision):** Seated Perplexity sonar as the Research tier (`sonar-pro` + `sonar-reasoning-pro`, both verified search-grounded via admin-ai with live test calls). Swapped `auxiliary.vision` `claude-sonnet-5` → `claude-haiku-4-5` (still `anthropic` direct), ~3x cheaper vision; verified with a real image call. Conductor unchanged (stays `deepseek-v4-pro`). +- **Sep 11, 2026:** DeepSeek V4.1 Flash released (Sep 10) and surpasses V4 Pro on performance/cost/speed. Flipped primary + delegation from deepseek-v4-pro to deepseek-v4-flash on Core and Anita. V4.1 Flash peak $0.30 input / $1.20 output (off-peak $0.15/$0.60; cache-hit peak $0.006). **CORRECTION (verified against the live DeepSeek pricing page, Sep 11): V4 Pro is NOT retiring.** DeepSeek reversed the Sep 14 sunset and continues V4 Pro unchanged at $1.32/$3.96, so the legacy name does not auto-route to Flash. Cost map: the admin-ai built-in map carried the superseded Flash peak ($0.44 miss / $0.014 cache-hit / $1.32 out); a peak override ($0.30 miss / $0.006 cache-hit / $1.20 out) was applied to the `deepseek-v4-flash` deployment via `/model/update` and verified end-to-end (live call billed 1.71e-05 = 37x3e-07 + 5x1.2e-06). Pro in the map was already correct at $1.32/$0.044/$3.96.