ippadmin/itpp-infrastructure
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
d70665039fbe13c2b2e9a68e99cacbf946531a0b
itpp-infrastructure/README.md
T

9.1 KiB
Raw Blame History

IT Pro Partner — Infrastructure Documentation

Last Updated: July 16, 2026 Maintainer: Sho'Nuff

Server Inventory

Core Server

  • Hostname: Core
  • IP: 152.53.192.33
  • Provider: netcup RS 2000 G12
  • Specs: 8 vCPU EPYC 9645, 15 GB DDR5 ECC, 512 GB NVMe
  • OS: Debian 13
  • Role: Hermes + LiteLLM + Ollama fallback + Portals
  • Key Services:
    • Hermes Agent (Telegram + cron, 22 cron jobs)
    • Caddy reverse proxy (11 domains, auto-TLS)
    • Ops Portal (FastAPI, port 8090)
    • Prometheus (native, port 9090) + Grafana (native, port 3002)
    • Uptime Kuma (Docker, port 3001) — 9+ monitors
    • Vaultwarden (Docker, port 8080) — vault.iamgmb.com
    • Super Search MCP v2.0.0 (FastMCP, port 8899)
    • Twenty CRM (Docker) — crm.debtrecoveryexperts.com
    • DocuSeal (Docker, port 3000) — sign.itpropartner.com
    • SearXNG (Docker, port 8888)
    • Komodo (Docker, port 9120)
    • Tailscale, StrongSwan, WireGuard (home CCR tunnel)
    • Redis cache

App1 Server

  • Hostname: app1
  • IP: 152.53.36.131
  • Provider: netcup RS 4000 G12
  • Specs: 12 vCPU EPYC 9645, 32 GB DDR5 ECC, 1 TB NVMe
  • OS: Debian 13
  • Role: AI/Service Hub
  • Key Services:
    • Ollama (Docker, port 11434)
    • Open WebUI (Docker, port 3000) — ai.itpropartner.com
    • n8n + Postgres (Docker, port 5678) — n8n.itpropartner.com
    • Caddy (systemd, 80/443)
    • 5 MCP servers: Super Search (:8899), Browser (:8901), Filesystem (:8900), Email (:8902), Git (:8903)
  • Planned: LiteLLM + Postgres (migrating from old-ai 178.156.167.181)

App2 Server

  • Hostname: app2
  • IP: 152.53.39.202
  • Provider: netcup RS 4000 G12
  • Specs: 12 vCPU EPYC 9645, 32 GB DDR5 ECC, 1 TB NVMe
  • OS: Debian 13
  • Role: Infrastructure Server
  • Key Services:
    • Traccar GPS (Docker, port 8082 + 5000-5150) — fleettracker360.com
    • UniFi Controller (Docker, port 8443) — unifi.itpropartner.com
    • UNMS/UISP (10 Docker containers) — unms.forefrontwireless.com
    • Hudu (Docker) — hudu.itpropartner.com
    • Caddy (4 domains)

App3 Server

  • Hostname: app3
  • IP: 152.53.241.111
  • Provider: netcup RS 4000 G12
  • Specs: 12 vCPU EPYC 9645, 32 GB DDR5 ECC, 1 TB NVMe
  • OS: Debian 13
  • Role: Web Hosting
  • Key Services:
    • CloudPanel CE — panel.itpropartner.com
    • Nginx (80/443) + Percona MySQL 8.4 + PHP 8.3
    • voipsimplicity.com (WordPress, migrated Jul 16)
    • my.voipsimplicity.com (PHP customer portal)

Core-BU (Warm Standby)

  • Hostname: core-bu
  • IP: 5.161.114.8
  • Provider: Hetzner CPX11 (planned upgrade: CPX31)
  • Specs: 2 vCPU, 2 GB RAM, 40 GB SSD
  • Role: Warm standby — auto-failover if Core down
  • Watchdog: 5-min check, 4-cycle confirmation, S3 sync every 10 min

Other Active Servers

  • old-ai: 178.156.167.181 (Hetzner CPX41) — LiteLLM + Open WebUI + Ollama — deprecated, migrating to app1
  • wphost02: 5.161.62.38 (Hetzner CPX21) — WordPress host — migrating to app3
  • Tony Hermes: 87.99.159.142 (Hetzner CPX21) — Anthony's personal Hermes

Domain / DNS Map

⚠️ itpropartner.com — SiteGround Nameservers Only

itpropartner.com uses SiteGround nameservers exclusively. A Cloudflare zone exists (0dc20632…) but is NOT authoritative — records created there silently fail. All *.itpropartner.com changes must be manual through SiteGround panel.

Domain IP Server Service
core.itpropartner.com 152.53.192.33 Core Static API docs
ops.itpropartner.com 152.53.192.33 Core Ops dashboard
sign.itpropartner.com 152.53.192.33 Core DocuSeal
uptimekuma.itpropartner.com 152.53.192.33 Core Monitoring
hudu.itpropartner.com 152.53.39.202 app2 IT documentation
unifi.itpropartner.com 152.53.39.202 app2 UniFi controller
panel.itpropartner.com 152.53.241.111 app3 CloudPanel CE
ai.itpropartner.com 152.53.36.131 app1 Open WebUI
n8n.itpropartner.com 152.53.36.131 app1 n8n automation
admin-ai.itpropartner.com 152.53.36.131 app1 LiteLLM (currently → old-ai)

Cloudflare-Managed Domains

Domain IP Server Service
fleettracker360.com 152.53.39.202 app2 Fleet tracking
gps.fleettracker360.com 152.53.39.202 app2 GPS devices
voipsimplicity.com 152.53.241.111 app3 VoIP site
support.voipsimplicity.com 152.53.241.111 app3 Support portal
portal.debtrecoveryexperts.com 152.53.192.33 Core DRE portal
crm.debtrecoveryexperts.com Cloudflare Access — DRE CRM

Backup Pipeline

Backup Schedule Target Purpose
hermes-live-sync Every 15 min s3://hermes-vps-backups/live/ Live state sync
hermes-full-backup Daily 1 AM s3://hermes-vps-backups/hermes-full-backup/ Full Hermes backup
home-router-backup Daily 6 AM s3://mikrotik-ccr-backups/ CCR config
root-essentials-backup Daily 3 AM S3 /root essentials
docker-volume-sync Daily 3 AM S3 Docker volumes
system-config-sync Daily 4 AM S3 System configs
unms-backup-sync Daily 6 AM s3://hermes-vps-backups/unms-backups/ UNMS data
unifi-backup-sync Daily 2 AM s3://hermes-vps-backups/unifi-backups/ UniFi configs
hudu-backup Daily 3 AM s3://hermes-vps-backups/hudu-backups/ Hudu (30-day ret)
wphost02-mariadb-dump Daily 2:30 AM S3 WordPress DB
warm-standby-sync Every 10 min core-bu ← S3 DR readiness

Caddy Configuration

Core (152.53.192.33)

sign.itpropartner.com    → :3000 (DocuSeal)
core.itpropartner.com    → static files
crm.debtrecoveryexperts.com → :3001 (Twenty)
portal.debtrecoveryexperts.com → static
internal.debtrecoveryexperts.com → static
app.itpropartner.com     → static + :8082
shark.itpropartner.com   → static + :8083
shark.iamgmb.com         → static + :8083
ops.itpropartner.com     → :8090 + static
uptimekuma.itpropartner.com → :3001
vault.iamgmb.com         → :8080

App1 (152.53.36.131)

ai.itpropartner.com      → :3000 (Open WebUI)
n8n.itpropartner.com     → :5678 (n8n)

App2 (152.53.39.202)

hudu.itpropartner.com    → Hudu internal
gps.fleettracker360.com  → :8082 (Traccar)
unms.forefrontwireless.com → UNMS Nginx
unifi.itpropartner.com   → :8443 (UniFi)

Service Access

Service URL Location Auth
Open WebUI https://ai.itpropartner.com app1 Chat UI
Open WebUI Admin https://admin-ai.itpropartner.com/ui app1 user: admin, pw: LITELLM_MASTER_KEY
Ops Portal https://ops.itpropartner.com Core Internal dashboard
Uptime Kuma https://uptimekuma.itpropartner.com Core Service monitoring
Vaultwarden https://vault.iamgmb.com Core Password vault
CloudPanel https://panel.itpropartner.com app3 SQLite DB at /home/clp/htdocs/app/data/db.sq3
Traccar https://gps.fleettracker360.com app2 GPS fleet tracking
UniFi https://unifi.itpropartner.com app2 Network controller
UNMS https://unms.forefrontwireless.com app2 WISP management
Hudu https://hudu.itpropartner.com app2 IT documentation
n8n https://n8n.itpropartner.com app1 Automation
CRM (DRE) https://crm.debtrecoveryexperts.com Cloudflare Access TwentyCRM

MCP Access (from Open WebUI)

All 5 MCP servers run on app1 and are wired into Open WebUI at https://ai.itpropartner.com:

MCP Port Tools
Super Search :8899 web_search, web_extract, web_search_premium
Browser :8901 browser_navigate, browser_snapshot, browser_click, browser_type, browser_console
Filesystem :8900 read_file, write_file, search_files, list_dir, file_info
Email :8902 search_emails, send_email, get_email
Git/Gitea :8903 clone, commit, push, pull

SSH Access

  • Key: itpp-infra (deployed to all servers)
  • User: ippadmin (sudo privileges)
  • Root SSH: Enabled on app1, app2, app3 (key-only exception per provisioning standard)
  • Core SSH: ssh -i /root/.ssh/itpp-infra root@152.53.192.33

Firewall

UFW is enabled on all servers. Standard rules:

  • Core: 22, 80, 443, 3000, 3001, 8080, 8082, 8443, 9090, 3002
  • app1: 22, 80, 443, 3000, 5678, 8899, 8900, 8901, 8902, 8903
  • app2: 22, 80, 443, 3000, 8080, 8082, 8089, 8443, 8843, 3478, 10001, 5000:5150
  • app3: 22, 80, 443, 8443

SSL

All SSL certificates issued via Let's Encrypt through Caddy. All certs auto-renew. No manual management needed.

Notes

  • DNS trap: itpropartner.com uses SiteGround nameservers. Cloudflare zone is NOT authoritative. Always verify with dig NS domain.com before creating records.
  • SiteGround: No API access. All DNS changes are manual through SiteGround panel.
  • Provider diversity: core-bu stays at Hetzner specifically so a netcup outage can't kill both Core and standby simultaneously.
  • app3 MySQL: Root password at MOQMINFQIhklM0AF, accessible via 127.0.0.1:3306.
  • CloudPanel: SQLite DB at /home/clp/htdocs/app/data/db.sq3 — users live here, not in MySQL.
Reference in New Issue View Git Blame Copy Permalink