Initial resurrection kit — 81 scripts, 62 references, configs, systemd units, crons, Docker compose files, Caddy config, master README

2026-07-15 17:45:36 -04:00
commit ae056eaf83
197 changed files with 26127 additions and 0 deletions
@@ -0,0 +1,839 @@
+# Hetzner Server Inventory Audit
+
+> **Live document.** Comprehensive SSH-based audit of every running server.
+> Last updated: 2026-07-09
+> All servers SSH'd via `root@<ip>` with `itpp-infra` key.
+
+---
+
+## Quick Summary (10 Servers)
+
+| # | Server | Hostname | IP | Specs | OS | Docker? | Key Apps |
+|---|--------|----------|----|-------|----|---------|----------|
+| 1 | **app1-bu** | app1-bu.itpropartner.com | 5.161.114.8 | 2C/2G/40G | Ubuntu 26.04 | ❌ | Standby Hermes, StrongSwan VPN, L2TP |
+| 2 | **ai.itpropartner.com** | ai | 178.156.167.181 | 8C/16G/240G | Ubuntu 24.04 | ✅ 6 containers | LiteLLM, Open WebUI, Ollama, Qdrant, Caddy |
+| 3 | **hudu.itpropartner.com** | hudu | 178.156.130.130 | 3C/4G/80G | Ubuntu 24.04 | ✅ 5 containers | Hudu Docs, SWAG (Let's Encrypt), MariaDB, Postfix |
+| 4 | **unifi** | unifi | 178.156.131.57 | 3C/4G/80G | Ubuntu 24.04 | ❌ | UniFi Controller, MongoDB, MariaDB |
+| 5 | **unms.forefrontwireless.com** | unms | 5.161.225.131 | 3C/4G/80G | Ubuntu 24.04 | ✅ 9 containers | UNMS/UISP, UCRM, Nginx, RabbitMQ, PostgreSQL |
+| 6 | **wphost02** | wphost02 | 5.161.62.38 | 3C/4G/80G | Ubuntu 24.04 | ❌ | RunCloud (Nginx, Apache, MariaDB, WordPress) |
+| 7 | **app1.itpropartner.com (n8n)** | n8n | 87.99.144.163 | 2C/2G/40G | Ubuntu 24.04 | ✅ 2 containers | n8n, PostgreSQL, Nginx, MariaDB |
+| 8 | **docker** | docker | 178.156.168.35 | 2C/2G/40G | Ubuntu 24.04 | ✅ 7 containers | NPM, RustDesk, Uptime-Kuma, StrongSwan, RunCloud |
+| 9 | **fleettracker360** | fleettrack360 | 178.156.149.32 | 2C/2G/40G | Ubuntu 24.04 | ❌ | Traccar, RunCloud (Apache/Nginx), MariaDB |
+| 10 | **app1 (netcup - live Hermes)** | core | 152.53.192.33 | 8C/15G/512G | Debian 13 | ✅ 6 containers | Hermes, Twenty CRM, Vaultwarden, Docuseal, SearXNG, Caddy |
+
+---
+
+## 1. 🔵 app1-bu — Hermes Standby
+
+| Field | Value |
+|-------|-------|
+| **IP** | 5.161.114.8 |
+| **Hostname** | app1-bu.itpropartner.com |
+| **Plan** | CPX11 |
+| **Specs** | 2 vCPU AMD EPYC-Rome · 1.9 GB RAM · 38 GB disk |
+| **OS** | Ubuntu 26.04 LTS (Resolute Raccoon) |
+| **Kernel** | 7.0.0-27-generic |
+| **Uptime** | 3 days 13:46 |
+| **Load** | 0.01 / 0.03 / 0.01 |
+| **Disk** | 54% used (20G of 38G) |
+
+### Docker
+❌ **No Docker installed**
+
+### Running Systemd Services (custom)
+- `hermes-gateway.service` — Hermes Gateway (disabled/dormant)
+- `hermes-standby.service` — Standby restore service
+- `home-router-keepalive.service` — Router keepalive
+- `home-router-vpn.service` — Router VPN tunnel
+- `strongswan-starter.service` — IPsec IKEv1/IKEv2
+- `xl2tpd.service` — L2TP tunnel daemon
+
+### Listening Ports
+| Port | Service |
+|------|---------|
+| 22 | SSH |
+| 53 | systemd-resolved (DNS) |
+| 500 | charon (IPsec IKE) |
+| 4500 | charon (IPsec NAT-T) |
+| 1701 | xl2tpd (L2TP) |
+| 323 | chronyd (NTP) |
+
+### Web Servers
+❌ None
+
+### Databases
+❌ None
+
+### Users
+- `root` (only user)
+
+### Cron (root)
+```cron
+*/5 * * * * /root/.hermes/scripts/hermes-standby-watchdog.sh
+*/10 * * * * /root/.hermes/scripts/hermes-standby-sync.sh
+```
+
+### Custom Scripts
+- `/root/triage_continue.sh`
+- `/root/triage_run.sh`
+- `/root/triage_verify_summary.py`
+
+---
+
+## 2. 🔵 ai.itpropartner.com — AI Proxy
+
+| Field | Value |
+|-------|-------|
+| **IP** | 178.156.167.181 |
+| **Hostname** | ai |
+| **Plan** | CPX41 |
+| **Specs** | 8 vCPU AMD EPYC-Rome · 15 GB RAM · 226 GB disk |
+| **OS** | Ubuntu 24.04.4 LTS (Noble Numbat) |
+| **Kernel** | 6.8.0-134-generic |
+| **Uptime** | 3 days 19:59 |
+| **Load** | 0.02 / 0.06 / 0.08 |
+| **Disk** | ⚠️ **92% used** (199G of 226G) |
+
+### Docker Containers (6 running)
+| Name | Image | Status | Ports |
+|------|-------|--------|-------|
+| `litellm` | ghcr.io/berriai/litellm:v1.84.0 | Up 3 days (unhealthy) | 4000 |
+| `litellm_postgres` | postgres:16 | Up 3 days (healthy) | 5432 |
+| `openwebui` | ghcr.io/open-webui/open-webui:0.9.6 | Up 3 days (healthy) | 8080 |
+| `ollama` | ollama/ollama:latest | Up 3 days | 11434 |
+| `qdrant` | qdrant/qdrant:latest | Up 3 days | 6333-6334 |
+| `caddy` | caddy:latest | Up 3 days | 80, 443 |
+
+### Docker Images
+| Repository | Tag | Size |
+|------------|-----|------|
+| ollama/ollama | latest | 10.1 GB |
+| ghcr.io/open-webui/open-webui | 0.9.6 | 6.73 GB |
+| ghcr.io/open-webui/open-webui | main | 6.7 GB |
+| ghcr.io/berriai/litellm | v1.84.0 | 2.58 GB |
+| postgres | 16 | 641 MB |
+| qdrant/qdrant | latest | 285 MB |
+| caddy | latest | 87.9 MB |
+| curlimages/curl | latest | 36.1 MB |
+
+### Docker Compose
+- `/opt/ai/docker-compose.yml`
+- `/opt/ai/docker-compose-last.yml`
+
+### Running Systemd Services (non-standard)
+- `containerd.service`, `docker.service`
+
+### Listening Ports (public)
+| Port | Service |
+|------|---------|
+| 22 | SSH |
+| 80 | HTTP → Caddy (Docker) |
+| 443 | HTTPS → Caddy (Docker) |
+| 4000 | LiteLLM |
+| 6333-6334 | Qdrant |
+| 11434 | Ollama |
+
+### Web Servers
+❌ Native — Caddy runs in Docker
+
+### Databases
+❌ None native (PostgreSQL runs in Docker)
+
+### Users
+- `root`, `ippadmin`
+
+### Cron
+No crontab for root
+
+### Custom Scripts
+❌ None
+
+---
+
+## 3. 🔵 hudu.itpropartner.com — Documentation
+
+| Field | Value |
+|-------|-------|
+| **IP** | 178.156.130.130 |
+| **Hostname** | hudu |
+| **Plan** | CPX21 |
+| **Specs** | 3 vCPU AMD EPYC-Rome · 3.7 GB RAM · 75 GB disk |
+| **OS** | Ubuntu 24.04.4 LTS (Noble Numbat) |
+| **Kernel** | 6.8.0-134-generic |
+| **Uptime** | 3 days 20:44 |
+| **Load** | 0.00 / 0.02 / 0.00 |
+| **Disk** | 23% used (16G of 75G) |
+
+### Docker Containers (5 running)
+| Name | Image | Status | Ports |
+|------|-------|--------|-------|
+| `hudu2-app-1` | hududocker/hudu:latest | Up 3 days | 3000 |
+| `hudu2-worker-1` | hududocker/hudu:latest | Up 3 days | 3000 |
+| `hudu2-db-1` | postgres:16.2 | Up 3 days | 5432 |
+| `hudu2-redis-1` | redis:latest | Up 3 days | 6379 |
+| `letsencrypt` | lscr.io/linuxserver/swag | Up 3 days | 80, 443 |
+
+### Docker Images
+| Repository | Tag | Size |
+|------------|-----|------|
+| hududocker/hudu | latest | 3.64 GB |
+| lscr.io/linuxserver/swag | latest | 602 MB |
+| postgres | 16.2 | 611 MB |
+| redis | latest | 202 MB |
+| alpine | latest | 12.8 MB |
+| hello-world | latest | 20.3 kB |
+
+### Running Systemd Services (non-standard)
+- `containerd.service`, `docker.service`
+- `fail2ban.service`, `firewalld.service`
+- `mariadb.service` — **MariaDB 11.4.12** (native, not Docker)
+- `email@redacted` — Postfix mail server
+- `runcloud-agent.service` — RunCloud agent
+- `supervisor.service`
+
+### Custom Systemd Services
+- `apache2-rc.service`, `nginx-rc.service`
+- `php81rc-fpm.service` through `php85rc-fpm.service`
+- `runcloud-admin.service`, `runcloud-agent.service`, `runcloud-upgrade.service`
+
+### Listening Ports (public)
+| Port | Service |
+|------|---------|
+| 22 | SSH |
+| 25 | Postfix (SMTP) |
+| 80 | HTTP → SWAG (Docker) |
+| 443 | HTTPS → SWAG (Docker) |
+| 3306 | MariaDB |
+| 34210 | RunCloud |
+
+### Databases
+- **MariaDB** 11.4.12 (native, active)
+- **MySQL** CLI (mysql/mysqld)
+- **Redis** (native, active)
+
+### Web Servers
+❌ Native — SWAG (Nginx) runs in Docker for Let's Encrypt
+- RunCloud custom Apache/Nginx installed but not running
+
+### Users
+- `root`, `runcloud`, `itpropartner`, `ippadmin`
+
+### Cron
+No crontab for root
+
+### Custom Scripts
+❌ None
+
+---
+
+## 4. 🔵 unifi — Network Controller
+
+| Field | Value |
+|-------|-------|
+| **IP** | 178.156.131.57 |
+| **Hostname** | unifi |
+| **Plan** | CPX21 |
+| **Specs** | 3 vCPU AMD EPYC-Rome · 3.7 GB RAM · 75 GB disk |
+| **OS** | Ubuntu 24.04.3 LTS (Noble Numbat) |
+| **Kernel** | 6.8.0-134-generic |
+| **Uptime** | 3 days 20:44 |
+| **Load** | 0.06 / 0.02 / 0.00 |
+| **Disk** | 14% used (9.6G of 75G) |
+
+### Docker
+❌ **No Docker installed**
+
+### Running Systemd Services (non-standard)
+- `unifi.service` — **UniFi Network Controller** (Java)
+- `snapd.service` — Snap daemon (certbot)
+- `fail2ban.service`, `firewalld.service`
+- `mariadb.service` — MariaDB 11.4.12
+- `email@redacted` — Postfix mail server
+- `runcloud-agent.service`
+- `supervisor.service`
+
+### Custom Systemd Services
+- `apache2-rc.service`, `nginx-rc.service`
+- `php81rc-fpm.service` through `php85rc-fpm.service`
+- `runcloud-admin.service`, `runcloud-agent.service`, `runcloud-upgrade.service`
+- `snap.certbot.renew.service`
+
+### Listening Ports (public)
+| Port | Service |
+|------|---------|
+| 22 | SSH |
+| 25 | Postfix (SMTP) |
+| 3306 | MariaDB |
+| 8080 | UniFi Controller (HTTP) |
+| 8443 | UniFi Controller (HTTPS) |
+| 8880 | UniFi (HTTP portal redirect) |
+| 8843 | UniFi (HTTPS portal redirect) |
+| 6789 | UniFi (Speed Test / throughput) |
+| 27117 | MongoDB (local) |
+| 34210 | RunCloud |
+| 3478 | STUN (UDP) |
+| 5514 | Syslog (UDP) |
+| 10001 | UBNT discovery (UDP) |
+
+### Databases
+- **MariaDB** 11.4.12 (native, active)
+- **MongoDB** (native, running on 127.0.0.1:27117)
+- **Redis** (installed)
+- MySQL CLI
+
+### Web Servers
+❌ Native — UniFi has built-in web server
+- RunCloud custom Apache/Nginx installed but not running
+
+### Users
+- `root`, `runcloud`, `itpropartner`
+
+### Cron
+No crontab for root
+
+### Custom Scripts
+❌ None
+
+---
+
+## 5. 🔵 unms.forefrontwireless.com — WISP Management
+
+| Field | Value |
+|-------|-------|
+| **IP** | 5.161.225.131 |
+| **Hostname** | unms |
+| **Plan** | CPX21 |
+| **Specs** | 3 vCPU AMD EPYC-Rome · 3.7 GB RAM · 75 GB disk |
+| **OS** | Ubuntu 24.04.4 LTS (Noble Numbat) |
+| **Kernel** | 6.8.0-134-generic |
+| **Uptime** | 3 days 20:44 |
+| **Load** | 0.61 / 0.55 / 0.38 |
+| **Disk** | 67% used (48G of 75G) |
+
+### Docker Containers (9 running)
+| Name | Image | Status | Ports |
+|------|-------|--------|-------|
+| `unms-nginx` | ubnt/unms-nginx:3.0.140 | Up 3 days | 80, 81, 443, 8089 |
+| `unms-api` | ubnt/unms:3.0.140 | Up 3 days (healthy) | — |
+| `unms-device-ws-1` | ubnt/unms:3.0.140 | Up 3 days (healthy) | — |
+| `unms-device-ws-2` | ubnt/unms:3.0.140 | Up 3 days (healthy) | — |
+| `unms-postgres` | ubnt/unms-postgres:3.0.140 | Up 3 days | 5432 |
+| `unms-siridb` | ubnt/unms-siridb:3.0.140 | Up 3 days (healthy) | 9000, 9010 |
+| `unms-rabbitmq` | rabbitmq:3.7.28-alpine | Up 3 days | 4369, 5671-5672, 25672 |
+| `unms-fluentd` | ubnt/unms-fluentd:3.0.140 | Up 3 days | 5140, 24224 |
+| `unms-netflow` | ubnt/unms-netflow:3.0.140 | Up 3 days | 2055/udp |
+| `ucrm` | ubnt/unms-crm:4.5.31 | Up 3 days | 80-81, 443, 9000, 2055/udp |
+
+### Docker Images
+| Repository | Tag | Size |
+|------------|-----|------|
+| ubnt/unms | 3.0.140 | 1.07 GB |
+| ubnt/unms-crm | 4.5.31 | 918 MB |
+| ubnt/unms-postgres | 3.0.140 | 833 MB |
+| ubnt/unms-nginx | 3.0.140 | 227 MB |
+| ubnt/unms-netflow | 3.0.140 | 213 MB |
+| ubnt/unms-siridb | 3.0.140 | 75.4 MB |
+| ubnt/unms-fluentd | 3.0.140 | 39.5 MB |
+| ubnt/ucrm-conntrack | latest | 6.29 MB |
+| rabbitmq | 3.7.28-alpine | 102 MB |
+| hello-world | latest | 10.1 kB |
+
+### Running Systemd Services (non-standard)
+- `containerd.service`, `docker.service`
+- `fail2ban.service`
+- `mariadb.service` — MariaDB 11.4.12 (native)
+- `email@redacted`
+- `runcloud-agent.service`
+- `snapd.service`
+- `supervisor.service`
+
+### Custom Systemd Services
+- `apache2-rc.service`, `nginx-rc.service`
+- `php81rc-fpm.service` through `php85rc-fpm.service`
+- `runcloud-admin.service`, `runcloud-agent.service`, `runcloud-upgrade.service`
+
+### Listening Ports (public)
+| Port | Service |
+|------|---------|
+| 22 | SSH |
+| 25 | Postfix (SMTP) |
+| 80, 81 | HTTP → UNMS Nginx (Docker) |
+| 443 | HTTPS → UNMS Nginx (Docker) |
+| 3306 | MariaDB (native) |
+| 8089 | UNMS Nginx (alt) |
+| 2055/udp | UNMS Netflow |
+| 24224 | Fluentd (local) |
+| 34210 | RunCloud |
+
+### Databases
+- **MariaDB** 11.4.12 (native, active)
+- **Redis** (native, active)
+- MySQL CLI
+
+### Web Servers
+❌ Native — Nginx/Apache via RunCloud installed but not primary
+- UNMS Nginx runs in Docker as main frontend
+
+### Users
+- `root`, `runcloud`, `unms`, `fwadmin`
+
+### Cron
+No crontab for root
+
+### Custom Scripts
+❌ None
+
+---
+
+## 6. 🔵 wphost02 — WordPress Host
+
+| Field | Value |
+|-------|-------|
+| **IP** | 5.161.62.38 |
+| **Hostname** | wphost02 |
+| **Plan** | CPX21 |
+| **Specs** | 3 vCPU AMD EPYC-Rome · 3.7 GB RAM · 75 GB disk |
+| **OS** | Ubuntu 24.04 LTS (Noble Numbat) |
+| **Kernel** | 6.8.0-134-generic |
+| **Uptime** | 3 days 20:45 |
+| **Load** | 0.12 / 0.10 / 0.07 |
+| **Disk** | 19% used (14G of 75G) |
+
+### Docker
+❌ **No Docker installed**
+
+### Running Systemd Services (non-standard)
+- `apache2-rc.service` — **Apache HTTP Server** (RunCloud)
+- `nginx-rc.service` — **Nginx reverse proxy** (RunCloud)
+- `php81rc-fpm.service` — PHP-FPM 8.1
+- `mariadb.service` — MariaDB 11.1.6
+- `email@redacted`
+- `fail2ban.service`, `firewalld.service`
+- `runcloud-agent.service`
+- `supervisor.service`
+
+### Custom Systemd Services
+- `apache2-rc.service`, `nginx-rc.service`
+- `php81rc-fpm.service` through `php85rc-fpm.service`
+- `runcloud-admin.service`, `runcloud-agent.service`, `runcloud-upgrade.service`
+
+### Listening Ports (public)
+| Port | Service |
+|------|---------|
+| 22 | SSH |
+| 25 | Postfix (SMTP) |
+| 80 | HTTP → Nginx (RunCloud) |
+| 443 | HTTPS → Nginx (RunCloud) |
+| 81 | Apache (local proxy target) |
+| 3306 | MariaDB |
+| 34210 | RunCloud |
+
+### Web Servers
+- **nginx-rc** (Nginx w/ RunCloud — active frontend)
+- **apache2-rc** (Apache w/ RunCloud — backend on port 81)
+
+### Databases
+- **MariaDB** 11.1.6 (native, active)
+- **Redis** (native, active)
+- MySQL CLI
+
+### Users
+- `root`, `runcloud`, `ippadmin`
+
+### Cron (root)
+```cron
+0 3 * * * /usr/local/bin/lynis-scan.sh
+```
+
+### Custom Scripts
+- `/usr/local/bin/lynis-scan.sh` (security scan, runs nightly at 3 AM)
+
+---
+
+## 7. 🔵 app1.itpropartner.com (n8n) — Workflow Automation
+
+| Field | Value |
+|-------|-------|
+| **IP** | 87.99.144.163 |
+| **Hostname** | n8n |
+| **Plan** | CPX11 |
+| **Specs** | 2 vCPU AMD EPYC-Rome · 1.9 GB RAM · 38 GB disk |
+| **OS** | Ubuntu 24.04.4 LTS (Noble Numbat) |
+| **Kernel** | 6.8.0-134-generic |
+| **Uptime** | 3 days 14:35 |
+| **Load** | 0.04 / 0.01 / 0.00 |
+| **Disk** | 23% used (8.2G of 38G) |
+
+### Docker Containers (2 running)
+| Name | Image | Status | Ports |
+|------|-------|--------|-------|
+| `n8n` | n8nio/n8n:latest | Up 3 days | 127.0.0.1:5678 |
+| `n8n-postgres` | postgres:16-alpine | Up 3 days (healthy) | 5432 |
+
+### Docker Images
+| Repository | Tag | Size |
+|------------|-----|------|
+| n8nio/n8n | latest | 2.28 GB |
+| docker.n8n.io/n8nio/n8n | latest | 2.28 GB |
+| postgres | 16-alpine | 395 MB |
+| hello-world | latest | 25.9 kB |
+
+### Docker Compose
+- `/opt/n8n/docker-compose.yml`
+
+### Running Systemd Services (non-standard)
+- `containerd.service`, `docker.service`
+- `nginx.service` — **Nginx** (native, not RunCloud)
+- `mariadb.service` — MariaDB 11.4.12
+- `email@redacted`
+- `fail2ban.service`, `firewalld.service`
+- `supervisor.service`
+
+### Custom Systemd Services
+- `apache2-rc.service` (installed but not running)
+- `php81rc-fpm.service` through `php84rc-fpm.service` (installed but not running)
+
+### Listening Ports (public)
+| Port | Service |
+|------|---------|
+| 22 | SSH |
+| 25 | Postfix (SMTP) |
+| 80 | HTTP → Nginx |
+| 443 | HTTPS → Nginx |
+| 3306 | MariaDB |
+| 5678 | n8n (localhost only) |
+
+### Web Servers
+- **Nginx** (native — proxying to n8n on 5678)
+- Sites enabled: `app1.itpropartner.com`
+
+### Databases
+- **MariaDB** 11.4.12 (native, active)
+- **Redis** (native, active)
+- MySQL CLI
+
+### Users
+- `root`, `runcloud`, `ippadmin`
+
+### Cron
+No crontab for root
+
+### Custom Scripts
+❌ None
+
+---
+
+## 8. 🔵 docker — Docker Host / NPM / RustDesk
+
+| Field | Value |
+|-------|-------|
+| **IP** | 178.156.168.35 |
+| **Hostname** | docker |
+| **Plan** | CPX11 |
+| **Specs** | 2 vCPU AMD EPYC-Rome · 1.9 GB RAM · 38 GB disk |
+| **OS** | Ubuntu 24.04.3 LTS (Noble Numbat) |
+| **Kernel** | 6.8.0-134-generic |
+| **Uptime** | 3 days 20:45 |
+| **Load** | 0.25 / 0.47 / 0.38 |
+| **Disk** | ⚠️ **73% used** (26G of 38G) |
+
+### Docker Containers (7 running)
+| Name | Image | Status | Ports |
+|------|-------|--------|-------|
+| `nginx-proxy-manager` | jc21/nginx-proxy-manager:latest | Up 3 days | 80, 81, 443 |
+| `uptime-kuma` | louislam/uptime-kuma:latest | Up 3 days (healthy) | 3001 |
+| `hbbr` | rustdesk/rustdesk-server | Up 3 days | 21117, 21119 |
+| `hbbs` | rustdesk/rustdesk-server | Up 3 days | 21114, 21116, 21118, 21116/udp |
+| `strongswan` | dagavi/strongswan | Up 3 days | — |
+| `runcloud_mariadb.*` | runcloud/mariadb:11.4 | Up 3 days (healthy) | — |
+| `runcloud_php56rc.*` | runcloud/php56rc:latest | Up 3 days (healthy) | — |
+| `runcloud_php81rc.*` | runcloud/php81rc:latest | Up 3 days (healthy) | — |
+
+### Docker Images
+| Repository | Tag | Size |
+|------------|-----|------|
+| jc21/nginx-proxy-manager | latest | 1.14 GB |
+| runcloud/php56rc | latest | 521 MB |
+| runcloud/php81rc | latest | 519 MB |
+| louislam/uptime-kuma | latest | 448 MB |
+| runcloud/mariadb | 11.4 | 419 MB |
+| nginx | alpine | 53.7 MB |
+| dagavi/strongswan | latest | 34.3 MB |
+| rustdesk/rustdesk-server | latest | 12.8 MB |
+
+### Docker Compose
+- `/opt/litellm/docker-compose.yml`
+- `/opt/openwebui/docker-compose.yml`
+- `/opt/rustdesk/docker-compose.yml`
+- `/opt/strongswan/docker-compose.yml`
+
+### Running Systemd Services (non-standard)
+- `containerd.service`, `docker.service`
+- `fail2ban.service`
+- `email@redacted`
+- `supervisor.service`
+
+### Listening Ports (public)
+| Port | Service |
+|------|---------|
+| 22 | SSH |
+| 25 | Postfix (SMTP) |
+| 80, 81 | HTTP → NPM |
+| 443 | HTTPS → NPM |
+| 3001 | Uptime-Kuma |
+| 3306 | MySQL (native) |
+| 21114 | RustDesk (relay) |
+| 21116 | RustDesk (TCP+UDP) |
+| 21117 | RustDesk (relay) |
+| 21118 | RustDesk |
+| 21119 | RustDesk |
+| 500, 4500 | StrongSwan IPsec |
+| 2377, 7946 | Docker swarm |
+
+### Databases
+- **MySQL** (native, active on 3306 — likely RunCloud MariaDB)
+
+### Web Servers
+❌ Native — Nginx Proxy Manager runs in Docker
+- RunCloud Docker containers for PHP
+
+### Users
+- `root`, `runcloud`, `ippadmin`
+
+### Cron
+No crontab for root
+
+### Custom Scripts
+❌ None
+
+---
+
+## 9. 🔵 fleettracker360 — GPS Fleet Tracking
+
+| Field | Value |
+|-------|-------|
+| **IP** | 178.156.149.32 |
+| **Hostname** | fleettrack360 |
+| **Plan** | CPX11 |
+| **Specs** | 2 vCPU AMD EPYC-Rome · 1.9 GB RAM · 38 GB disk |
+| **OS** | Ubuntu 24.04.4 LTS (Noble Numbat) |
+| **Kernel** | 6.8.0-134-generic |
+| **Uptime** | 3 days 20:45 |
+| **Load** | 0.00 / 0.00 / 0.00 |
+| **Disk** | 25% used (8.9G of 38G) |
+
+### Docker
+❌ **No Docker installed**
+
+### Running Systemd Services (non-standard)
+- `traccar.service` — **Traccar GPS tracking** (Java)
+- `apache2-rc.service` — Apache (RunCloud)
+- `nginx-rc.service` — Nginx (RunCloud)
+- `php81rc-fpm.service` — PHP-FPM 8.1
+- `mariadb.service` — MariaDB 11.4.12
+- `email@redacted`
+- `fail2ban.service`, `firewalld.service`
+- `runcloud-agent.service`
+- `supervisor.service`
+
+### Custom Systemd Services
+- `apache2-rc.service`, `nginx-rc.service`
+- `php81rc-fpm.service` through `php85rc-fpm.service`
+- `runcloud-admin.service`, `runcloud-agent.service`, `runcloud-upgrade.service`
+- `traccar.service`
+
+### Listening Ports (very busy — Traccar uses many ports)
+| Port | Service |
+|------|---------|
+| 22 | SSH |
+| 25 | Postfix (SMTP) |
+| 80 | HTTP → Nginx (RunCloud) |
+| 443 | HTTPS → Nginx (RunCloud) |
+| 81 | Apache (local proxy) |
+| 3306 | MariaDB |
+| 5001-5020 | Traccar (various tracking protocols) |
+| 8082 | Traccar (web UI) |
+| 34210 | RunCloud |
+
+**UDP also: 5001-5032 — Traccar tracking protocols**
+
+### Web Servers
+- **nginx-rc** (Nginx w/ RunCloud — active frontend)
+- **apache2-rc** (Apache w/ RunCloud — backend)
+
+### Databases
+- **MariaDB** 11.4.12 (native, active)
+- **Redis** (native, active)
+- MySQL CLI
+
+### Users
+- `root`, `runcloud`, `ippadmin`
+
+### Cron
+No crontab for root
+
+### Custom Scripts
+❌ None
+
+---
+
+## 10. 🔵 app1 (netcup) — Live Hermes (core)
+
+| Field | Value |
+|-------|-------|
+| **IP** | 152.53.192.33 |
+| **Hostname** | core |
+| **Plan** | RS 2000 G12 (netcup) |
+| **Specs** | 8 vCPU AMD EPYC 9645 · 15 GB RAM · 503 GB disk |
+| **OS** | Debian 13 (trixie) |
+| **Kernel** | 6.12.94+deb13-amd64 |
+| **Uptime** | 3 days (from context) |
+| **Disk** | 6% used (26G of 503G) |
+
+### Docker Containers (7 running)
+| Name | Image | Status | Ports |
+|------|-------|--------|-------|
+| `twenty-server-1` | twentycrm/twenty:latest | Up 34 hours (healthy) | 3001→3000 |
+| `twenty-worker-1` | twentycrm/twenty:latest | Up 34 hours | — |
+| `twenty-db-1` | postgres:16 | Up 34 hours (healthy) | 5432 |
+| `twenty-redis-1` | redis | Up 34 hours (healthy) | 6379 |
+| `docuseal` | docuseal/docuseal:latest | Up 2 days | 127.0.0.1:3000 |
+| `searxng` | searxng/searxng:latest | Up 2 days | 127.0.0.1:8888 |
+| `vaultwarden` | vaultwarden/server:1.33.2 | Up 3 days (healthy) | 8080→80 |
+
+### Docker Images
+| Repository | Tag | Size |
+|------------|-----|------|
+| twentycrm/twenty | latest | 1.12 GB |
+| docuseal/docuseal | latest | 476 MB |
+| postgres | 16 | 451 MB |
+| searxng/searxng | latest | 258 MB |
+| vaultwarden/server | 1.33.2 | 237 MB |
+| redis | latest | 143 MB |
+
+### Docker Compose
+- `/opt/theharvester/docker-compose.yml`
+
+### Running Systemd Services (non-standard)
+| Service | Status | Purpose |
+|---------|--------|---------|
+| `hermes.service` | ✅ active | Hermes Agent |
+| `hermes-browser.service` | ✅ active | Headless Chromium (CDP) |
+| `caddy.service` | ✅ active | Caddy web server |
+| `ollama.service` | ✅ active | Local LLM service |
+| `mysql-tunnel.service` | ✅ active | AutoSSH tunnel → wphost02 MySQL |
+| `shark-game.service` | ✅ active | Shark Attack Fantasy Game |
+| `strongswan-starter.service` | ✅ active | IPsec VPN |
+| `xl2tpd.service` | ✅ active | L2TP tunnel |
+| `tailscaled.service` | ✅ active | Tailscale |
+
+### Listening Ports (public)
+| Port | Service |
+|------|---------|
+| 22 | SSH |
+| 80 | Caddy (HTTP) |
+| 443 | Caddy (HTTPS) |
+| 8080 | Vaultwarden |
+| 8081 | Hermes assistant |
+| 8083 | Shark Game |
+| 3001 | Twenty CRM |
+| — | Internal services (localhost) |
+| 3000 | Docuseal (localhost) |
+| 8888 | SearXNG (localhost) |
+| 9222 | Chrome DevTools (localhost) |
+| 11434 | Ollama (localhost) |
+| 33060 | MySQL tunnel (localhost) |
+
+**UDP:** 500, 4500 (IPsec), 1701 (L2TP), 41641 (Tailscale), 51820 (WireGuard)
+
+### Web Servers
+- **Caddy** 2.11.4 (native — main reverse proxy with auto HTTPS)
+
+### Caddy Config
+- `sign.itpropartner.com` → proxy to Docuseal (127.0.0.1:3000), serves `/vehicles.json`
+- `core.itpropartner.com` → serves static files, `/health`, `/capabilities`
+
+### Databases
+- MySQL CLI installed
+- SQLite3 installed
+- **No native MariaDB/PostgreSQL** — databases run in Docker
+
+### Users
+- `root`, `ippadmin`
+
+### Cron (root)
+```cron
+* * * * * /root/.hermes/scripts/dre-mail-poller.py
+0 * * * * /root/.hermes/scripts/boys-mail-monitor.py collect 2>&1 | logger -t boys-mail-collect
+0 19 * * * /root/.hermes/scripts/boys-mail-monitor.py summary 2>&1 | logger -t boys-mail-summary
+0 8 * * * /root/shark-game/scraper/run.sh
+*/15 * * * * /root/.hermes/scripts/shark-draft-reminder.sh 2>&1 | logger -t shark-draft-reminder
+```
+
+### Custom Scripts
+- `/root/explore_crm.py`, `explore_crm2.py`, `explore_crm3.py`
+- `/root/fetch_zabbix_docs.py`
+- `/root/kw_autocomplete.py`, `kw_paa_scrape.py`, `kw_research.py`
+- `/root/parse_docs.py`, `parse_docs2.py`
+- `/root/process_inbox.py`
+- `/root/research_savannah.py`, `research_savannah2.py`
+- `/root/test-feed.py`
+- `/usr/local/bin/home-router-watchdog.sh`
+
+---
+
+## Cross-Cutting Summary
+
+### Docker Hosts (5 of 10)
+| Server | Containers | Notable |
+|--------|-----------|---------|
+| ai.itpropartner.com | 6 | LiteLLM, Open WebUI, Ollama, Qdrant |
+| hudu.itpropartner.com | 5 | Hudu, SWAG, PostgreSQL, Redis |
+| unms.forefrontwireless.com | 10 | UNMS, UCRM, RabbitMQ, PostgreSQL, Netflow |
+| app1.itpropartner.com (n8n) | 2 | n8n, PostgreSQL |
+| docker | 7+ | NPM, RustDesk, Uptime-Kuma, StrongSwan, RunCloud |
+| app1 (netcup) | 7 | Twenty CRM, Vaultwarden, Docuseal, SearXNG |
+
+### Non-Docker Hosts (4 of 10)
+| Server | Native Apps |
+|--------|------------|
+| app1-bu | StrongSwan, L2TP, standby scripts |
+| unifi | UniFi Controller, MongoDB, MariaDB |
+| wphost02 | RunCloud (Nginx+Apache+PHP), MariaDB, WordPress |
+| fleettracker360 | Traccar, RunCloud (Nginx+Apache+PHP), MariaDB |
+
+### RunCloud Hosts (5 of 10)
+hudu, unifi, unms, wphost02, fleettracker360 — all share RunCloud agent, custom Apache/Nginx/php*-fpm services
+
+### Database Engines (across fleet)
+| Engine | Servers |
+|--------|---------|
+| MariaDB/MySQL | hudu, unifi, unms, wphost02, n8n, fleettracker360, docker |
+| PostgreSQL (Docker) | ai, hudu, unms, n8n, core (Twenty CRM) |
+| MongoDB | unifi (UniFi Controller) |
+| Redis | hudu, core, unms, wphost02, fleettracker360 |
+| Qdrant | ai (vector DB) |
+| SQLite | core |
+
+### VPN / Networking
+| Type | Server |
+|------|--------|
+| IPsec StrongSwan | app1-bu, docker, core |
+| L2TP | app1-bu, core |
+| Tailscale | core |
+| WireGuard | core |
+
+### Monitoring
+| Tool | Server |
+|------|--------|
+| Uptime-Kuma | docker |
+| Lynis security scan | wphost02 (cron) |
+| Fail2ban | hudu, unifi, unms, wphost02, n8n, docker, fleettracker360 |
+
+### Disk Warnings
+- ⚠️ **ai.itpropartner.com**: 92% disk (199G of 226G) — critical
+- ⚠️ **docker**: 73% disk (26G of 38G) — getting full
+
+---
+
+> **Audited:** 2026-07-09 via itpp-infra SSH key across all 10 servers.