Files
itpp-infrastructure/docs/architecture.md
T
ShoNuff cac3cde372 docs(infra): app4 + core-bu provisioning, migration plan, verified inventory
- migration-plan-app4-core-bu-2026-09-15.md: 8-phase plan (Nuremberg decision,
  provider-diversity gap, acceptance criteria, rollback, DNS/Caddy checklist)
- core-service-inventory-2026-09-15: verified Core inventory, ~30 customer-facing
  services (the Aug 15 plan listed 5), 3 DocuSeal instances, TimeTrex Postgres,
  dead Caddy routes
- reference-update-matrix-2026-09-15: 52 artifacts that name a host
- fix naming collision: 6 files called the Hetzner box core-bu, the name core-bu
  now claims; app1-bu = 5.161.225.131, core-bu = 159.195.204.203 (netcup Nuremberg)
- correct the false provider-diversity claim (the standby is now netcup too)
- supersede app4-migration-plan.md (wrong region reported, silent on core-bu)
2026-09-15 10:19:46 -04:00

138 lines
6.7 KiB
Markdown

# IT Pro Partner — Live Architecture Reference
**Last Updated:** 2026-08-09
**Maintainer:** Sho'Nuff (Hermes Agent)
**Purpose:** Single source of truth for ITPP server infrastructure. Replaces the archived `master-apps-services.md` (Jul 16, 2026) which listed 10+ defunct servers and stale specs.
---
## Servers
| Server | IP | Specs | Provider | Role |
|---|---|---|---|---|
| **Core** | 152.53.192.33 | RS 2000 G9.5 (8 vCPU EPYC 9645, 15 GB RAM, 503 GB SSD) | netcup | Hermes agent host, monitoring, Caddy reverse proxy (26 sites) |
| **app1** | 152.53.36.131 | RS 4000 G9.5 (12 vCPU EPYC, 32 GB RAM, 1 TB SSD) | netcup | Service hub — AI gateway, CRM, signing, TTS, auth, automation |
| **app2** | 152.53.39.202 | RS 4000 G9.5 (12 vCPU EPYC, 32 GB RAM, 1 TB SSD) | netcup | Infrastructure — Gitea, Hudu, Ubiquiti controllers, Traccar, DNS, SIEM |
| **app3** | 152.53.241.111 | RS 4000 G9.5 (12 vCPU EPYC, 32 GB RAM, 1 TB SSD) | netcup | Web hosting — CloudPanel CE (WordPress/static/PHP), client sites |
| **app1-bu** | 5.161.225.131 | CPX21 (3 vCPU, 4 GB RAM, 80 GB) | Hetzner | Warm standby for Core — **the only non-netcup box**. Auto-restores from S3. |
| **app4** | 159.195.205.80 | RS 4000 G12 (12 vCPU EPYC, 32 GB RAM, 1 TB NVMe) | netcup | **Customer-facing services** (migration target off Core). Nuremberg. Provisioned 2026-09-15. |
| **core-bu** | 159.195.204.203 | RS 2000 G12 (8 vCPU EPYC, 16 GB RAM, 503 GB) | netcup | **Core's warm standby** (new role, Nuremberg). Provisioned 2026-09-15. |
> **Provider diversity (2026-09-15):** the standby pair is now Core (netcup, Manassas) + **core-bu (netcup, Nuremberg)**. That gives
> **regional** separation but **not provider** separation: a netcup-wide incident affects both. `app1-bu` (Hetzner) is the only
> non-netcup box; provider diversity therefore depends on retaining it. Open decision — see
> `docs/infrastructure/migration-plan-app4-core-bu-2026-09-15.md` Q4.
---
## Services by Server
### Core (152.53.192.33)
| Service | Port | Type | Docs |
|---|---|---|---|
| Hermes Agent | — | Systemd | See `hermes-agent` skill |
| Caddy | 80, 443 | Systemd | `/etc/caddy/Caddyfile` |
| Prometheus | 9090 (internal) | Docker | — |
| Grafana | :3002 | Docker | Credential in Vaultwarden |
| Uptime Kuma | :3001 | Docker | — |
| Telegraf | — | Docker | — |
| MikroTik Exporter | :9436 | Docker | — |
| Microbin | — | Docker | — |
| Browserless | — | Docker | — |
| Camofox Browser | :9377 | Docker | — |
| Mealie | 9925 (internal) | Docker | — |
### App1 (152.53.36.131)
| Service | Port | Type | Docs |
|---|---|---|---|
| **LiteLLM** | :4000 | Docker | In progress — `org-audit/docs/services/litellm-deployment.md` |
| **Twenty CRM** | 3000 | Docker (4 containers: server, worker, db, redis) | In progress — `org-audit/docs/services/twenty-crm-deployment.md` |
| DocuSeal | — | Docker | — |
| Kokoro TTS | :8880 | Docker | — |
| n8n | — | Docker | — |
| Open WebUI | — | Docker | — |
| **Vaultwarden** | :8081 | Docker | `org-audit/docs/services/vaultwarden-deployment.md` |
| **Wazuh** | :5601 | Docker (3 containers: dashboard, indexer, manager) | `org-audit/docs/services/wazuh-deployment.md` |
| Komodo | :9120 | Docker | — |
### App2 (152.53.39.202)
| Service | Port | Type | Docs |
|---|---|---|---|
| **Gitea** | :3001 | Docker | In progress — `org-audit/docs/services/gitea-deployment.md` |
| Hudu | :3000 | Docker | — |
| UNMS (UISP) | :6443 | Docker | — |
| UniFi | :8443 | Docker | — |
| Traccar | :8082 | Docker | Verified operational 2026-08-09 |
| **Technitium DNS** | :5380 | Docker | In progress — `org-audit/docs/services/technitium-dns-deployment.md` |
| RAGFlow | — | Docker | — |
| Dawarich | :3002 | Docker | — |
| searxng | 9925 (internal) | Docker | — |
### App3 (152.53.241.111)
| Service | Port | Type | Docs |
|---|---|---|---|
| CloudPanel CE | — | Systemd (nginx + PHP-FPM) | — |
| WordPress sites | 80, 443 | nginx | Per-site in CloudPanel |
| Static HTML sites | 80, 443 | nginx | Per-site in CloudPanel |
---
## DNS & Domains
| Domain | Registrar | DNS Provider | Notes |
|---|---|---|---|
| itpropartner.com | Cloudflare | SiteGround (external) | Cloudflare zone has no authority — records must be set at SiteGround |
| iamgmb.com | Cloudflare | Cloudflare | Grey-cloud only (no proxy). Zone: `f1fb2d357b8ff0fab54c5856130ec9ed` |
| germainebrown.com | Cloudflare | Cloudflare | Personal domain |
| fleettracker360.com | Cloudflare | Cloudflare | Zone: `d1830faef5a6d83365360fa925feeb13`. Orange-cloud proxy → app2 |
| debt... (DRE) | Cloudflare | Cloudflare | Access-protected |
| hotnow.io | Cloudflare | Cloudflare | Registered Aug 2026. No deployment yet. |
---
## Backup Schedule
| Source | Target | Frequency | Script |
|---|---|---|---|
| Core Hermes (live sync) | s3://hermes-vps-backups/live/ | Every 15 min | `hermes-live-sync` |
| Core Hermes (full) | s3://hermes-vps-backups/hermes-full-backup/ | Daily 1:00 AM | `hermes-backup.sh` |
| App1 | S3 | Daily 2:00 AM | Cron on app1 |
| App2 | S3 | Daily 2:30 AM | Cron on app2 |
| App3 | S3 | Daily 3:00 AM | Cron on app3 |
**S3 target:** Wasabi `s3.us-east-1.wasabisys.com`, bucket `hermes-vps-backups`.
---
## Key Repositories
| Repo | Gitea URL | Purpose |
|---|---|---|
| itpp-infrastructure | ippadmin/itpp-infrastructure | Infrastructure docs and scripts |
| disaster-recovery | ippadmin/disaster-recovery | DR plans, runbooks, issue log |
| hermes-skills | ippadmin/hermes-skills | Hermes Agent skills |
| hermes-recovery | ippadmin/hermes-recovery | Hermes recovery bundles |
| org-audit | ippadmin/org-audit | External audit exports and deployment docs |
| homelab | ippadmin/homelab | Home lab documentation |
---
## Security Notes
1. **Plaintext secrets in repos: RESOLVED 2026-08-09.** `hermes-recovery` and `hermes-skills` Git histories were purged of exposed credentials. Both exposed keys (SyncroMSP token, Apex MySQL password) were already stale at purge time.
2. **Vaultwarden** is the sole credential store — deployment docs at `org-audit/docs/services/vaultwarden-deployment.md`.
3. **LiteLLM** routes all AI model traffic — deployment docs at `org-audit/docs/services/litellm-deployment.md`.
4. **Wazuh** is the security monitoring infrastructure — deployment docs at `org-audit/docs/services/wazuh-deployment.md`.
5. **Technitium DNS** is authoritative for internal zones — admin password already changed from default.
---
## Changelog
- **2026-08-09:** Created as replacement for archived `master-apps-services.md`. Corrected server specs, removed defunct servers, added documentation cross-references.
- **2026-07-28:** Original `master-apps-services.md` archived (stale — listed Mattermost, wphost01, standalone hudu, incorrect specs).