From 3424b9013bc89aa10beb14638caa8caf7fe5774d Mon Sep 17 00:00:00 2001 From: root Date: Fri, 17 Jul 2026 13:28:05 -0400 Subject: [PATCH] Model fallback chain: GPT-5.5 primary, 5 providers, Ollama removed, DNS updates --- README.md | 28 +++++++++++++++++++++------- backup-plan.md | 1 - project-log-2026-07-17.md | 26 ++++++++++++++++++++++---- 3 files changed, 43 insertions(+), 12 deletions(-) diff --git a/README.md b/README.md index cccca9a..df50444 100644 --- a/README.md +++ b/README.md @@ -13,22 +13,20 @@ - **Provider:** netcup RS 2000 G12 - **Specs:** 8 vCPU EPYC 9645, 15 GB DDR5 ECC, 512 GB NVMe - **OS:** Debian 13 -- **Role:** Hermes + Ollama fallback + Portals +- **Role:** Hermes + Portals - **Key Services:** - Hermes Agent (Telegram + cron, 22 cron jobs) - - Caddy reverse proxy (10 domains, auto-TLS) + - Caddy reverse proxy (12 domains, auto-TLS) - Ops Portal (FastAPI, port 8090) - Prometheus (native, port 9090) + Grafana (native, port 3002) - Uptime Kuma (Docker, port 3001) — 9+ monitors - Vaultwarden (Docker, port 8080) — vault.iamgmb.com - - Super Search MCP v2.0.0 — migrated to app1 - Twenty CRM (Docker) — crm.debtrecoveryexperts.com - DocuSeal (Docker, port 3000) — sign.core.itpropartner.com - SearXNG (Docker, port 8888) - Komodo (Docker, port 9120) - Tailscale, StrongSwan, WireGuard (home CCR tunnel 10.77.0.0/24) - Redis cache - - Ollama (systemd, port 11434) ### App1 Server - **Hostname:** app1 @@ -38,7 +36,6 @@ - **OS:** Debian 13 - **Role:** AI/Service Hub - **Key Services:** - - Ollama (Docker, port 11434) - Open WebUI (Docker, port 3000) — ai.itpropartner.com - n8n + Postgres (Docker, port 5678) — n8n.itpropartner.com - LiteLLM (Docker) + Postgres — admin-ai.itpropartner.com @@ -89,8 +86,25 @@ ### Legacy / Decommissioned - **old-ai:** 178.156.167.181 (Hetzner CPX41) — **decommissioned** (LiteLLM migrated to app1) - **old app1:** 87.99.144.163 (Hetzner CPX11) — **deleted** -- **wphost02:** 5.161.62.38 (Hetzner CPX21) — **migrating to app3** -- **Tony Hermes:** 87.99.159.142 (Hetzner CPX21) — Anthony's personal Hermes +- **wphost02:** 5.161.62.38 (Hetzner CPX21) — **migrated to app3** +- **Ollama:** Removed from Core (systemd) and app1 (Docker) Jul 17 + +--- + +## Model Fallback Chain + +All providers use direct API keys. GPT-5.5 quality survives through admin-ai → OpenRouter, then degrades through DeepSeek → Gemini → Grok. + +| # | Model | Provider | Gateway | +|---|---|---|---| +| Primary | GPT-5.5 | admin-ai | Self-hosted LiteLLM (app1) | +| Fallback 1 | GPT-5.5 | OpenRouter | openrouter.ai | +| Fallback 2 | DeepSeek v4 Pro | DeepSeek | api.deepseek.com | +| Fallback 3 | Gemini 3.5 Flash | Google | generativelanguage.googleapis.com | +| Fallback 4 | Grok 4.5 | xAI | api.x.ai | + +**Credits (Jul 17):** DeepSeek $58, OpenRouter ~$30 remaining, OpenAI/xAI/Google on pay-as-you-go +**Health check:** Daily 8 AM cron (`model-usage-check`) --- diff --git a/backup-plan.md b/backup-plan.md index 176c88e..4ee58c3 100644 --- a/backup-plan.md +++ b/backup-plan.md @@ -52,7 +52,6 @@ | LiteLLM | Postgres pg_dump + config.yaml → S3 (daily) | 24 hr | | Open WebUI | /app/backend/data → S3 (daily) | 24 hr | | n8n | Postgres pg_dump → S3 (daily) | 24 hr | -| Ollama | Models preserved in Docker volume | 24 hr | | MCP servers | Configs in Git + Docker volumes | 24 hr | | Mattermost | Postgres DB + data volume → S3 (daily) | 24 hr | diff --git a/project-log-2026-07-17.md b/project-log-2026-07-17.md index 146f613..6318f4e 100644 --- a/project-log-2026-07-17.md +++ b/project-log-2026-07-17.md @@ -32,7 +32,25 @@ - DNS: Cloudflare, updated A record to 152.53.241.111 - Previously: was resolving to old IP, Cloudflare zone accessible -## Model Credits Reliability -- DeepSeek balance: $59.29 (not exhausted) -- Cron fixed to track deepseek-v4-pro instead of deepseek-chat -- Local Ollama fallback: llama3.2:3b +## Model Fallback Chain (Major Update) +- **Primary:** GPT-5.5 via admin-ai (self-hosted LiteLLM on app1) +- **Fallback 1:** GPT-5.5 via OpenRouter (different gateway, same model) +- **Fallback 2:** DeepSeek v4 Pro direct (DeepSeek API) +- **Fallback 3:** Gemini 3.5 Flash direct (Google API) +- **Fallback 4:** Grok 4.5 direct (xAI API) +- **Funded:** OpenRouter ($280 total, $250 used), DeepSeek ($58) +- **Removed:** Ollama uninstalled from Core (systemd) and app1 (Docker) +- **Health check:** Daily 8 AM cron checks all 5 providers + +## Model Credits (Jul 17) +- DeepSeek: $58.06 remaining +- OpenRouter: $280 total, ~$30 remaining +- GPT-5.5, Gemini, Grok: pay-as-you-go via direct API keys +- OpenAI direct API key exists but not in fallback chain + +## DNS Updates +- noc.itpropartner.com → 152.53.36.131 (Mattermost, app1) +- vault.itpropartner.com → 152.53.36.131 (Vaultwarden, app1) +- status.itpropartner.com → 152.53.192.33 (Status page, Core) +- my.itpropartner.com → moved from CREATE to PRODUCTION +- mainwp.itpropartner.com → moved from DELETE to PRODUCTION