ippadmin/itpp-infrastructure
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: model-chain — key was stale, fixed with hermes-agent-v5 Jul 31

Browse Source
This commit is contained in:
root
2026-07-31 07:14:11 -04:00
parent 100e65c296
commit 7cbef9bd8f
1 changed files with 24 additions and 17 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/model-chain.md
+24 -17
View File
@@ -1,6 +1,6 @@
# AI Model Architecture — IT Pro Partner # AI Model Architecture — IT Pro Partner
**Updated:** July 27, 2026 **Updated:** July 31, 2026
Two separate concepts that must not be conflated: Two separate concepts that must not be conflated:
@@ -12,8 +12,8 @@ Survives admin-ai outage. Uses direct API keys for F1–F4.
| Tier | Model | Provider | Notes | | Tier | Model | Provider | Notes |
|---|---|---|---| |---|---|---|---|
| **Primary** | `deepseek-v4-flash` | `admin-ai` | Fast, cheap conductor via LiteLLM | | **Primary** | `deepseek-v4-pro` | `admin-ai` | Conductor & workhorse via LiteLLM v5 key |
| **F1** | `deepseek-v4-flash` | `deepseek` (direct) | Same model, direct API | | **F1** | `deepseek-v4-flash` | `deepseek` (direct) | Flash demoted -- fallback only, was hallucinating |
| **F2** | `gemini-3.6-flash` | `google` (direct) | Fast, free-tier eligible | | **F2** | `gemini-3.6-flash` | `google` (direct) | Fast, free-tier eligible |
| **F3** | `grok-4.6` | `xai` (direct) | Verified working (NOT grok-4.5) | | **F3** | `grok-4.6` | `xai` (direct) | Verified working (NOT grok-4.5) |
| **F4** | `claude-sonnet-5` | `anthropic` (direct) | Premium survival fallback | | **F4** | `claude-sonnet-5` | `anthropic` (direct) | Premium survival fallback |
@@ -22,32 +22,39 @@ Survives admin-ai outage. Uses direct API keys for F1–F4.
--- ---
## Operational Models *(daily toolbox — admin-ai only)* ## Operational Models *(daily toolbox -- admin-ai only)*
No direct API keys. Shared $5/day budget via hermes-agent-v3 key. All route through admin-ai. Shared $20/day budget via hermes-agent-v3 key.
| Role | Model | When | | Role | Model | When |
|---|---|---| |---|---|---|
| Conductor | `deepseek-v4-flash` | Orchestrating, delegating, quick decisions | | **Conductor** | `deepseek-v4-pro` | ALL standard work -- orchestrating, delegating, infra, coding |
| Workhorse | `deepseek-v4-pro` | Large context, big outputs, infra code | | **Workhorse** | `deepseek-v4-pro` | Same model -- large context, big outputs, infra code |
| Workhorse | `deepseek-reasoner` | Multi-step analysis, debugging, planning | | **Workhorse** | `deepseek-reasoner` | Multi-step analysis, debugging, planning |
| Auditor | `gemini-3.6-flash` | Sanity checks, every output (essentially free) | | **Auditor** | `gemini-3.6-flash` | Sanity checks, every output (essentially free) |
| Auditor | `claude-sonnet-5` | Architecture, security, auth — sparingly ($15/M output) | | **Auditor** | `claude-sonnet-5` | Architecture, security, auth -- sparingly ($15/M output) |
--- ---
## Admin-AI Virtual Key ## Admin-AI Virtual Key
- **Key Name:** `hermes-agent-v3` (generated Jul 27, 2026) - **Key Name:** `hermes-agent-v5` (regenerated Jul 31, 2026)
- **Daily Budget Cap:** $5/day shared across all operational models - **Daily Budget Cap:** $20/day shared across all operational models
- **Expiry:** Jul 27, 2027 - **Expiry:** Jul 31, 2027
- **Master Key:** stored in Hudu - **Master Key:** stored in Hudu
- **Models:** deepseek-v4-pro, deepseek-v4-flash, deepseek-reasoner, gemini-3.6-flash, claude-sonnet-5, xai/grok-4.3, xai/grok-4-1-fast, grok-4.6
## Budget Targets ## Budget Targets
| Component | Daily est. | | Component | Daily est. |
|---|---| |---|---|
| Conductor (ds-v4-flash) | ~$0.30 | | Conductor + Workhorse (ds-v4-pro) | ~$2.50 |
| Workhorses (ds-v4-pro + reasoner) | ~$1.70 | | Workhorse (ds-reasoner) | ~$0.70 |
| Auditors (gemini + claude sparingly) | ~$0.80 | | Auditors (gemini flash + claude sparingly) | ~$0.80 |
| **Total** | **$3–5/day** | | **Total** | **~$4/day** (well under $20 cap) |
## Key Decisions
- **Jul 29, 2026:** DeepSeek V4 Flash demoted from conductor to F1 fallback. Was hallucinating tool calls and producing inconsistent results. V4 Pro promoted to primary conductor and workhorse.
- **Jul 30, 2026:** LiteLLM key regenerated — hermes-agent-v3 created in DB. However, the config.yaml was NOT updated with the new key (doc was written but config file not patched). Hermes silently fell back to DeepSeek direct since Jul 30 19:25 UTC.
- **Jul 31, 2026:** Discovered config.yaml still had stale key (`sk-DuZ..._AMg`, dead since rotation). Generated `hermes-agent-v5` with same model list + xAI voice models. Verified chat OK. Cleaned up stale keys (v3, v4, v4b). Updated model-chain doc to reflect actual state. Daily digest cron updated with HTML email + clean prompt for Telegram delivery.