From 02b07b06bc1784d3956dc81d3ec08333b06871d5 Mon Sep 17 00:00:00 2001 From: Sho'Nuff Date: Sun, 9 Aug 2026 02:37:34 -0400 Subject: [PATCH] audit: N1 resolved (auth2 on app3), Core storage 503G, Appendix C counts fixed, model info added --- docs/architecture.md | 2 +- docs/comprehensive-audit-summary-2026-08-09.md | 14 ++++++++------ 2 files changed, 9 insertions(+), 7 deletions(-) diff --git a/docs/architecture.md b/docs/architecture.md index b61dc0d..2290a20 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -10,7 +10,7 @@ | Server | IP | Specs | Provider | Role | |---|---|---|---|---| -| **Core** | 152.53.192.33 | RS 2000 G9.5 (8 vCPU EPYC 9645, 15 GB RAM, 256 GB SSD) | netcup | Hermes agent host, monitoring, Caddy reverse proxy (26 sites) | +| **Core** | 152.53.192.33 | RS 2000 G9.5 (8 vCPU EPYC 9645, 15 GB RAM, 503 GB SSD) | netcup | Hermes agent host, monitoring, Caddy reverse proxy (26 sites) | | **app1** | 152.53.36.131 | RS 4000 G9.5 (12 vCPU EPYC, 32 GB RAM, 1 TB SSD) | netcup | Service hub — AI gateway, CRM, signing, TTS, auth, automation | | **app2** | 152.53.39.202 | RS 4000 G9.5 (12 vCPU EPYC, 32 GB RAM, 1 TB SSD) | netcup | Infrastructure — Gitea, Hudu, Ubiquiti controllers, Traccar, DNS, SIEM | | **app3** | 152.53.241.111 | RS 4000 G9.5 (12 vCPU EPYC, 32 GB RAM, 1 TB SSD) | netcup | Web hosting — CloudPanel CE (WordPress/static/PHP), client sites | diff --git a/docs/comprehensive-audit-summary-2026-08-09.md b/docs/comprehensive-audit-summary-2026-08-09.md index d3dc8b7..2137221 100644 --- a/docs/comprehensive-audit-summary-2026-08-09.md +++ b/docs/comprehensive-audit-summary-2026-08-09.md @@ -3,6 +3,7 @@ **Prepared for:** External Review **Auditor:** Sho'Nuff (Hermes Agent) +**Model:** DeepSeek V4 Pro via admin-ai.itpropartner.com (LiteLLM gateway) — full audit, issue resolution, and follow-up task orchestration **Master tracker:** [org-audit/docs/production-audit.md](https://git.itpropartner.com/ippadmin/org-audit/src/branch/master/docs/production-audit.md) (single source of truth) **Narrative companion:** [itpp-infrastructure/docs/post-audit-report-2026-08-09.md](https://git.itpropartner.com/ippadmin/itpp-infrastructure/src/branch/main/docs/post-audit-report-2026-08-09.md) @@ -24,7 +25,7 @@ | Server | IP | Specs (SSH-verified) | Provider | |---|---|---|---| -| Core | 152.53.192.33 | 8 vCPU EPYC 9645, 15 GB RAM, 256 GB | netcup RS 2000 | +| Core | 152.53.192.33 | 8 vCPU EPYC 9645, 15 GB RAM, 503 GB | netcup RS 2000 | | app1 | 152.53.36.131 | 12 vCPU EPYC, 32 GB RAM, 1 TB | netcup RS 4000 | | app2 | 152.53.39.202 | 12 vCPU EPYC, 32 GB RAM, 1 TB | netcup RS 4000 | | app3 | 152.53.241.111 | 12 vCPU EPYC, 32 GB RAM, 1 TB | netcup RS 4000 | @@ -57,7 +58,7 @@ |---|---|---|---| | R1 | **fleettracker360.com DNS** | Flagged as broken DNS | ✅ **RESOLVED** — false positive. Cloudflare orange-cloud proxy IPs (188.114.x.x) are expected. Site returns HTTP/2 200 through proxy. | | R2 | **itpp-infrastructure stale docs** | `master-apps-services.md` listed defunct servers | ✅ **RESOLVED** — file deleted. `architecture.md` is now authoritative, updated with verified specs. | -| N1 | **Auth API / Stack Auth gap** | Not in audit scope | 🆕 **OPEN** — `auth.itpropartner.com` resolves to Core but nothing listens. No Docker containers for auth/hexclave/stack-auth anywhere. DNS for `stack.itpropartner.com` and `hexclave.itpropartner.com` doesn't exist. `auth.iamgmb.com` decommissioned. **Not deployed.** | +| N1 | **Auth API / Stack Auth / Hexclave** | Not in audit scope, flagged as missing | ✅ **RESOLVED (false alarm, closed 2026-08-09).** `auth2.itpropartner.com` (app3) is live. Hexclave (formerly Stack Auth) Docker containers confirmed: `hexclave-server`, `hexclave-cron`, `hexclave-postgres`, `hexclave-clickhouse`. Daily backups at 3:15 AM and 3:30 AM. **Root cause:** Audit checked wrong domains (`auth.itpropartner.com`, `stack.itpropartner.com`) instead of the known-correct `auth2.itpropartner.com`. Container search was scoped to app1 only, missing app3. Process gap: established facts weren't referenced before fresh discovery scans. | | N2 | **Gitea deployment docs** | No deployment doc existed | ✅ **DOCUMENTED** (565 lines). The service hosting all documentation is now itself documented. | --- @@ -74,7 +75,7 @@ | Gitea repositories | 49 | | Repos WITH deployment docs | 6 of 24 services (critical services complete) | | Repos with CRITICAL issues | 0 (both plaintext-secret repos resolved) | -| Active cron jobs | 39 (38 silent, 1 alert-capable) | +| Active cron jobs | 62 (59 no-agent scripts, 3 LLM-driven — 3 currently with errors) | | Backup frequency | 15-min checkpoints + daily full backups on all 4 app servers | | Pre-commit secret scanner | Deployed on 7 repos | @@ -180,7 +181,7 @@ Warm failover — boots and auto-restores from S3. | 🟡 **HIGH** | **LiteLLM deployment doc** needs fallback chain section + verify `gemini-3.6-flash` availability | Reopened | | 🟡 **STALE** | 14 repos with partial/stale docs | Ongoing | | 🟡 **UNDOCUMENTED** | 15 production services lack deployment guides (DocuSeal, Komodo, RAGFlow, Dawarich, Camofox, Open WebUI, n8n, etc.) | Not started | -| 🟢 **GAP** | Auth API / Stack Auth — not deployed, not documented. Needs decision: build or remove DNS entries. | Open | +| 🟢 **GAP** | **Auth API / Stack Auth** — now confirmed running at `auth2.itpropartner.com` on app3. Needs deployment documentation. | N1 closed. Doc gap remains. | | 🟢 **NOTES** | Homelab: adguard-home VM 100 stopped on vm-host-01. QNAP NFS mounts both pointing to `/ISO` export. | Low-priority | ### Guardrails to prevent recurrence @@ -191,6 +192,7 @@ Warm failover — boots and auto-restores from S3. | **Single master tracker** | `org-audit/docs/production-audit.md` is the one place for finding status. No other audit document tracks status independently. | | **Headline accuracy rule** | Executive summaries must not claim more than the body supports. "Full documentation coverage" was wrong; "critical services complete" is correct. | | **Server specs: SSH-verify, never assume** | "8C/16G/320G" was wrong — no source supported it. Going forward, specs must be verified via `nproc`, `free -m`, `df -BG` directly. | +| **Fact-reference before discovery** | N1 false alarm: audit checked guessed domains (`auth.itpropartner.com`, `stack.itpropartner.com`) instead of the known-correct `auth2.itpropartner.com`. Established memory facts (Hexclave = Stack Auth = auth2.itpropartner.com on app3) were available but not referenced before conducting fresh DNS discovery. Process fix: before any discovery scan, query memory and fact_store for known entities being scanned. | --- @@ -217,8 +219,8 @@ Warm failover — boots and auto-restores from S3. | Status | Count | Repos | |---|---|---| -| ✅ MATCHES | 20 | itpp-infrastructure, ops-portal, ops-reports, backup-restore, hudu, unifi, unms, hermes-skills, hermes-recovery, mcp-servers, mcp-browser, mcp-email, mcp-filesystem, mcp-git, super-search, shonuff-caller, nvr-shield, shark-game, gift-a-roast, dre, fleettracker360, voipsimplicity, voipsimplicity-manual, apextrack, startup-studio | -| ⚠️ PARTIAL/STALE | 14 | disaster-recovery, scripts, model-fallback, homelab, auth, pipeline, research-search-mcp, voice-agent, osint-tool, launchcheck, verdicttank, track-a-flock, transitpin, personal-assistant, content-creation-pipeline, pry, itpropartner-website, forefront-wireless-portal, cartmylist, mooresunnydaze, msp-forms, boxpilot, super-search-business, digital-signage | +| ✅ MATCHES | 23 | itpp-infrastructure, ops-portal, ops-reports, backup-restore, hudu, unifi, unms, hermes-skills, hermes-recovery, mcp-servers, mcp-browser, mcp-email, mcp-filesystem, mcp-git, super-search, shonuff-caller, nvr-shield, dre, fleettracker360, voipsimplicity, voipsimplicity-manual, shark-game, gift-a-roast, apextrack | +| ⚠️ PARTIAL/STALE | 17 | disaster-recovery, scripts, model-fallback, homelab, auth, pipeline, research-search-mcp, voice-agent, osint-tool, content-creation-pipeline, pry, itpropartner-website, forefront-wireless-portal, cartmylist, mooresunnydaze, msp-forms, boxpilot | | ❌ NOT DEPLOYED | 10 | super-search-business, launchcheck, verdicttank, track-a-flock, transitpin, digital-signage, personal-assistant, hotnow.io, IntelSight, SchoolCart | ### D. Key Documents