341 lines
14 KiB
Markdown
341 lines
14 KiB
Markdown
# IT Pro Partner — Master Apps & Services Inventory
|
||
|
||
> **Last Updated:** July 11, 2026
|
||
> **Maintainer:** Sho'Nuff
|
||
> **Purpose:** Complete registry of every installed application, subscription, API key, FOSS tool, and the server/service it lives on.
|
||
|
||
---
|
||
|
||
## Infrastructure Overview
|
||
|
||
| Server | IP | Specs | Provider | Role |
|
||
|---|---|---|---|---|
|
||
| **Core** | 152.53.192.33 | RS 2000 (8C/15G/512GB) | netcup | Ops hub |
|
||
| **app1** | 152.53.36.131 | RS 4000 (12C/32G/1TB) | netcup | AI/ML stack |
|
||
| **app2** | 152.53.39.202 | RS 4000 (12C/32G/1TB) | netcup | UISP/Infrastructure |
|
||
| **app3** | 152.53.241.111 | RS 4000 (12C/32G/1TB) | netcup | Web apps (spare) |
|
||
| **app1-bu** | 5.161.114.8 | CPX11 (2C/2G/40G) | Hetzner | Warm standby |
|
||
| **old-ai** | 178.156.167.181 | CPX41 (226G disk) | Hetzner | **Decommissioning** — admin-ai pending migration to app1 |
|
||
| **wphost02** | 5.161.62.38 | — | Hetzner | WordPress host (migrating to app3) |
|
||
| **Tony Hermes** | 87.99.159.142 | — | — | Anthony's personal Hermes |
|
||
|
||
**SSH Key:** `itpp-infra` (deployed to all servers, root SSH key-only)
|
||
**Base OS:** Debian 13 (all netcup boxes), Debian 12 (Hetzner boxes)
|
||
|
||
### Active Projects
|
||
|
||
| Project | Status | Server | Description |
|
||
|---|---|---|---|
|
||
| **Super Search** | ✅ Live v1.1.0 | Core | Multi-provider web search (SearXNG → Exa → Firecrawl) + extraction via FastMCP |
|
||
| **DRE / Debt Recovery Experts** | Active | Core + Cloudflare | Collections platform, CRM, approval workflows, letter templates |
|
||
| **Apex Track Experience** | Active | wphost02 (→ app3) | NASA Top Speed Event, WPForms, SMTP, PayPal |
|
||
| **Shark Attack Fantasy League** | Active | Core | shark.iamgmb.com, Discord bridge, changelog |
|
||
| **VoIPSimplicity** | Planning | — | RingLogix CPaaS, Yealink rentals |
|
||
| **Ops Portal** | Active | Core | ops.itpropartner.com, admin dashboards, customer portal |
|
||
| **admin-ai Migration** | Phase 0 Pending | old-ai → app1 | Move LiteLLM from Hetzner to netcup |
|
||
|
||
---
|
||
|
||
## Server-by-Server Inventory
|
||
|
||
### Core (netcup RS 2000 — 152.53.192.33)
|
||
|
||
#### System Services
|
||
|
||
| Service | Type | Purpose |
|
||
|---|---|---|
|
||
| **Hermes Agent** | systemd | AI assistant gateway (Telegram + cron) |
|
||
| **Caddy** | systemd | Reverse proxy (port 80/443) with Let's Encrypt |
|
||
| **Ops Portal** | systemd (uvicorn) | Internal admin dashboard at ops.itpropartner.com |
|
||
| **OSINT API** | systemd (uvicorn) | OSINT search API on port 8100 |
|
||
| **Hermes Assistant (PWA)** | systemd | PWA backend on port 8082 |
|
||
| **Shark Game** | systemd | Shark Attack Fantasy League backend on port 8083 |
|
||
| **Ollama** | systemd | Local LLM fallback (llama3.2:3b) |
|
||
| **Tailscale** | systemd | Mesh VPN for infra access |
|
||
| **StrongSwan + xl2tpd** | systemd | L2TP/IPsec VPN server |
|
||
| **WireGuard** | systemd | Tunnel to home router (10.77.0.0/24) |
|
||
| **MySQL Tunnel** | systemd | SSH tunnel to wphost02 MySQL (port 33060) |
|
||
| **Redis** | systemd | Cache layer |
|
||
| **Prometheus Node Exporter** | systemd | System metrics collection |
|
||
| **Hermes Browser** | systemd | Chromium headless for browser automation (9222) |
|
||
|
||
#### Docker Containers
|
||
|
||
| Container | Image | Port | Purpose |
|
||
|---|---|---|---|
|
||
| **Komodo Core** | ghcr.io/moghtech/komodo-core:2 | 9120 | Docker management UI |
|
||
| **Komodo Mongo** | mongo | — | Komodo database |
|
||
| **Prometheus** | prom/prometheus | 9090 (loopback) | Metrics aggregation |
|
||
| **Grafana** | grafana/grafana | 3002 (loopback) | Monitoring dashboards |
|
||
| **Twenty CRM** | twentycrm/twenty:latest | 3001 | DRE CRM (crm.debtrecoveryexperts.com) |
|
||
| **Twenty Worker** | twentycrm/twenty:latest | — | Twenty async worker |
|
||
| **Twenty DB** | postgres:16 | — | Twenty PostgreSQL |
|
||
| **Twenty Redis** | redis | — | Twenty cache |
|
||
| **DocuSeal** | docuseal/docuseal:latest | 3000 (loopback) | Document signing |
|
||
| **SearXNG** | searxng/searxng:latest | 8888 (loopback) | Private search engine |
|
||
| **Vaultwarden** | vaultwarden/server:1.33.2 | 8080 | Password manager (tailnet only) |
|
||
|
||
#### Caddy Sites
|
||
|
||
| Domain | Backend | Purpose |
|
||
|---|---|---|
|
||
| sign.itpropartner.com | → :3000 (DocuSeal) | Document signing |
|
||
| core.itpropartner.com | static files | JSON data, capabilities, portal |
|
||
| crm.debtrecoveryexperts.com | → :3001 (Twenty) | DRE customer CRM |
|
||
| portal.debtrecoveryexperts.com | static | DRE customer portal |
|
||
| internal.debtrecoveryexperts.com | static | DRE internal dashboard |
|
||
| app.itpropartner.com | static + → :8082 | Hermes Assistant PWA |
|
||
| shark.itpropartner.com | static + → :8083 | Shark game |
|
||
| shark.iamgmb.com | static + → :8083 | Shark game alias |
|
||
| ops.itpropartner.com | → :8090 + static | Ops admin portal |
|
||
|
||
#### Cron Jobs (Hermes-managed, 22 total)
|
||
|
||
| Job | Schedule | Purpose |
|
||
|---|---|---|
|
||
| hermes-live-sync | Every 15m | S3 backup sync |
|
||
| service-health-check | Every 5m | Service monitoring |
|
||
| Home-Router-Watchdog | Every 5m | CCR watchdog |
|
||
| vps-threshold-check | Every 15m | Disk/mem alerts |
|
||
| hermes-memory-consolidate | Every 10m | Memory pruning |
|
||
| IMAP-email-triage | Every 60m | Email classification |
|
||
| IMAP-triage-watchdog | Every 10m | IMAP health check |
|
||
| shonuff-inbox-agent | Every 15m | Inbox monitoring |
|
||
| daily-tech-digest | Daily 11 AM | Tech news summary |
|
||
| exotic-vehicle-scout | Daily 9 AM | New car alerts |
|
||
| shark-scraper | Every 60m | Shark sighting data |
|
||
| dre-mail-poller | Every 1m | DRE inbox polling |
|
||
| dre-pending-approval-reminder | Hourly | DRE approval reminders |
|
||
| boxpilot-triage | Every 10m | BoxPilot email triage |
|
||
| bounce-check | Every 60m | Email bounce detection |
|
||
| apex-mail-watchdog | Every 5m | Apex SMTP health |
|
||
| ops-data-collector | Every 5m | Ops portal data |
|
||
| lynis-scan | Daily 3 AM | Security audit |
|
||
| firecrawl-usage-check | Daily 9 AM | Web search credits |
|
||
| model-usage-check | Daily 8 AM | LLM availability |
|
||
| hetzner-weekly-snapshots | Weekly Mon 5 AM | Server snapshots |
|
||
| home-router-daily-backup | Daily 6 AM | CCR config backup |
|
||
|
||
---
|
||
|
||
### app1 (netcup RS 4000 — 152.53.36.131) — AI/ML Stack
|
||
|
||
#### Docker Containers
|
||
|
||
| Container | Image | Port | Purpose |
|
||
|---|---|---|---|
|
||
| **Ollama** | ollama/ollama:latest | 11434 (loopback) | Local LLM inference |
|
||
| **Open WebUI** | ghcr.io/open-webui/open-webui:latest | 3000 (loopback) | LLM chat UI (ai.itpropartner.com) |
|
||
| **n8n** | n8nio/n8n:latest | 5678 (loopback) | Workflow automation (n8n.itpropartner.com) |
|
||
| **n8n-postgres** | postgres:16-alpine | — | n8n database |
|
||
|
||
#### Caddy Sites
|
||
|
||
| Domain | Backend | Purpose |
|
||
|---|---|---|
|
||
| ai.itpropartner.com | → :3000 (Open WebUI) | LLM chat interface |
|
||
| n8n.itpropartner.com | → :5678 (n8n) | Automation workflows |
|
||
|
||
#### System Services
|
||
|
||
- Caddy, Docker, fail2ban, unattended-upgrades
|
||
|
||
#### Plans
|
||
|
||
- LiteLLM + Postgres (migrating from old-ai)
|
||
- Designated admin-ai.itpropartner.com target
|
||
|
||
---
|
||
|
||
### app2 (netcup RS 4000 — 152.53.39.202) — Infrastructure
|
||
|
||
#### Docker Containers
|
||
|
||
| Container | Image | Port | Purpose |
|
||
|---|---|---|---|
|
||
| **UNMS API** | ubnt/unms:latest | — | UISP core API |
|
||
| **UCRM** | ubnt/unms-crm:4.5.33 | 80/443 | Customer CRM |
|
||
| **UNMS Nginx** | ubnt/unms-nginx:latest | 80/443/8089 | UISP web proxy |
|
||
| **UNMS Postgres** | ubnt/unms-postgres:latest | 5432 | UISP database |
|
||
| **UNMS SiriDB** | ubnt/unms-siridb:latest | — | UISP time-series DB |
|
||
| **UNMS RabbitMQ** | rabbitmq:3.7.28-alpine | 5672 | UISP message queue |
|
||
| **UNMS Netflow** | ubnt/unms-netflow:latest | 2055/udp | Network flow data |
|
||
| **UNMS Fluentd** | ubnt/unms-fluentd:latest | 24224 | Log aggregation |
|
||
|
||
#### System Services
|
||
|
||
Docker, fail2ban, unattended-upgrades
|
||
|
||
---
|
||
|
||
### app3 (netcup RS 4000 — 152.53.241.111) — Web Apps
|
||
|
||
**Status:** Fresh server, Docker installed, no containers yet. Designated for WordPress migrations from wphost02/SiteGround.
|
||
|
||
---
|
||
|
||
### app1-bu (Hetzner CPX11 — 5.161.114.8) — Warm Standby
|
||
|
||
#### System Services
|
||
|
||
| Service | Purpose |
|
||
|---|---|
|
||
| **Hermes** | Gateway dormant — auto-activates if Core fails (5 min watchdog) |
|
||
| **Ollama** | Local LLM fallback |
|
||
| **StrongSwan** | L2TP/IPsec VPN |
|
||
| **Tailscale** | Mesh VPN |
|
||
| **Node Exporter** | Metrics |
|
||
| **apex-status-receiver** | Apex endpoint |
|
||
|
||
---
|
||
|
||
### wphost02 (Hetzner — 5.161.62.38) — WordPress
|
||
|
||
| Site | Purpose |
|
||
|---|---|
|
||
| apextrackexperience.com | Apex Track Experience (WordPress + WooCommerce + WPForms) |
|
||
| apex-status | Apex status receiver endpoint |
|
||
|
||
**MySQL:** WordPress DB at `apextrackexperience_1781549652` (via Core tunnel on port 33060)
|
||
|
||
---
|
||
|
||
### old-ai (Hetzner CPX41 — 178.156.167.181)
|
||
|
||
**Status: Decommissioning.** All services migrating to app1.
|
||
|
||
| Container | Image | Purpose |
|
||
|---|---|---|
|
||
| LiteLLM | ghcr.io/berriai/litellm:v1.84.0 | LLM proxy (admin-ai) |
|
||
| LiteLLM Postgres | postgres:16 | Model configs, API keys |
|
||
| Caddy | caddy:latest | TLS termination |
|
||
| Qdrant | qdrant/qdrant:latest | Vector DB |
|
||
| Ollama | ollama/ollama:latest | Local models |
|
||
|
||
---
|
||
|
||
## Subscriptions & API Keys
|
||
|
||
### Active Subscriptions
|
||
|
||
| Service | Plan | Cost | Billing | Purpose |
|
||
|---|---|---|---|---|
|
||
| **netcup** | RS 2000 + 4× RS 4000 | ~$156/mo | Monthly | All servers |
|
||
| **Hetzner** | CPX11 + CPX41 + misc | ~$50/mo | Monthly | app1-bu, old-ai (decomm), wphost02, old boxes (decomm) |
|
||
| **MXroute** | Email hosting | ~$15/yr? | Annual | email@redacted, DRE email, boys' email |
|
||
| **SiteGround** | WordPress hosting | ~$10/mo | Monthly | Apex Track Experience (migrating) |
|
||
| **Cloudflare** | Free (DNS + CDN + Access) | Free | — | All domains |
|
||
| **OpenRouter** | Pay-as-you-go | Variable | Prepaid | Model API fallback |
|
||
| **Firecrawl** | Pay-as-you-go | Variable | — | Web scraping for AI |
|
||
| **ScrapingAnt** (or similar) | Pay-as-you-go | Variable | — | Web extraction |
|
||
| **SyncroMSP** | RMM | ~$2/mo per endpoint | Monthly | 26 customers, ITPP endpoints |
|
||
| **Wasabi S3** | Hot cloud storage | ~$7/TB/mo | Monthly | Hermes backups, router configs |
|
||
| **Nous Research** | Hermes Agent license | Usage-based | Prepaid | Hermes platform |
|
||
|
||
### API Keys & Tokens
|
||
|
||
| Key | Location | Service | Status |
|
||
|---|---|---|---|
|
||
| FIRECRAWL_API_KEY | ~/.hermes/.env | Web scraping | ✅ Active |
|
||
| CLOUDFLARE_API_TOKEN | ~/.hermes/.env | DNS management | ✅ Active |
|
||
| NETCUP_API_KEY | ~/.hermes/.env | Server provisioning | ✅ Active |
|
||
| NETCUP_PASSWORD | ~/.hermes/.env | SCP API auth | ✅ Active |
|
||
| SYNCROMSP_API_TOKEN | ~/.hermes/.env | RMM platform | ✅ Active |
|
||
| TELEGRAM_BOT_TOKEN | ~/.hermes/.env | Telegram bot | ✅ Active |
|
||
| LITELLM_MASTER_KEY | ~/.hermes/config.yaml + /opt/ai/.env | admin-ai proxy | ✅ Active |
|
||
| LITELLM_SALT_KEY | /opt/ai/.env | Admin-ai encryption | ✅ Active |
|
||
| OpenRouter API Key | ~/.hermes/config.yaml | Model API + credits | ✅ Active |
|
||
| Hermes Nous Portal Key | — | Hermes updates | ❌ Not logged in |
|
||
| SERVER_PASSWORDS | ~/.hermes/.env | Server root passwords | ✅ Active |
|
||
| DeepSeek API Key | Litellm Postgres (encrypted) | DeepSeek LLM API | ✅ Active |
|
||
| Anthropic API Key | LiteLLM Postgres (encrypted) | Claude models | ✅ Active |
|
||
| Google/Gemini API Key | LiteLLM Postgres (encrypted) | Gemini vision + text | ✅ Active |
|
||
| OpenAI API Key | LiteLLM Postgres (encrypted) | GPT models | ✅ Active |
|
||
| Groq API Key | LiteLLM Postgres (encrypted) | Fast inference | ✅ Active |
|
||
| xAI API Key | LiteLLM Postgres (encrypted) | Grok models | ✅ Active |
|
||
|
||
---
|
||
|
||
## FOSS Software Stack
|
||
|
||
### Core Infrastructure
|
||
|
||
| Software | License | Server | Purpose |
|
||
|---|---|---|---|
|
||
| Debian 13 | GPL | All | Server OS |
|
||
| Docker + Compose | Apache 2.0 | All | Container runtime |
|
||
| Caddy | Apache 2.0 | Core, app1 | Reverse proxy + TLS |
|
||
| Tailscale | BSD | Core, app1-bu | Mesh VPN |
|
||
| WireGuard | GPL | Core | Site-to-site VPN |
|
||
| StrongSwan | GPL | Core, app1-bu | L2TP/IPsec VPN |
|
||
| Prometheus | Apache 2.0 | Core | Metrics |
|
||
| Grafana | AGPL | Core | Dashboards |
|
||
|
||
### Applications
|
||
|
||
| Software | License | Server | Purpose |
|
||
|---|---|---|---|
|
||
| **Hermes Agent** | Proprietary | Core, app1-bu | AI assistant |
|
||
| **Ollama** | MIT | Core, app1, app1-bu | Local LLM |
|
||
| **Open WebUI** | MIT | app1 | LLM chat UI |
|
||
| **n8n** | Sustainable Use License | app1 | Workflow automation |
|
||
| **Twenty CRM** | AGPL | Core | Customer CRM |
|
||
| **DocuSeal** | AGPL | Core | Document signing |
|
||
| **SearXNG** | AGPL | Core | Private search |
|
||
| **Vaultwarden** | GPL | Core | Password manager |
|
||
| **Komodo** | GPL | Core | Docker management |
|
||
| **LiteLLM** | MIT | old-ai (→ app1) | LLM proxy |
|
||
| **Qdrant** | Apache 2.0 | old-ai | Vector DB |
|
||
| **UISP/UNMS** | Proprietary | app2 | ISP management |
|
||
| **UCRM** | Proprietary | app2 | Customer management |
|
||
| **Ollama Models** | Various | Core, app1, old-ai | LLM weights |
|
||
| **Redis** | BSD | Core | Cache |
|
||
| **PostgreSQL** | PostgreSQL | Core, app1, app2 | Databases |
|
||
| **WordPress** | GPL | wphost02 (→ app3) | Web content |
|
||
| **WooCommerce** | GPL | wphost02 | E-commerce |
|
||
| **WPForms** | GPL | wphost02 | Form builder |
|
||
| **BlogWatcher** | — | Core cron | RSS monitoring |
|
||
| **Lynis** | GPL | Core | Security auditing |
|
||
|
||
---
|
||
|
||
## Domains & DNS
|
||
|
||
| Domain | Registrar | DNS | Purpose |
|
||
|---|---|---|---|
|
||
| itpropartner.com | — | Cloudflare | Primary |
|
||
| germainebrown.com | — | MXroute | Email |
|
||
| debtrecoveryexperts.com | — | Cloudflare | DRE |
|
||
| apextrackexperience.com | — | SiteGround/Cloudflare | Apex |
|
||
| iamgmb.com | — | MXroute | Boys' email |
|
||
| voipsimplicity.com | — | Cloudflare | VoIPSimplicity |
|
||
| tailc2f3b0.ts.net | — | Tailscale | Tailnet DNS (internal) |
|
||
|
||
---
|
||
|
||
## Backup & DR
|
||
|
||
| Backup | Schedule | Location | Retention |
|
||
|---|---|---|---|
|
||
| Hermes full backup | Daily 5 AM | s3://hermes-vps-backups/hermes-full-backup/ | — |
|
||
| Hermes live state | Every 15 min | s3://hermes-vps-backups/live/ | Rolling |
|
||
| Router config (home) | Daily 6 AM | s3://mikrotik-ccr-backups/ | — |
|
||
| Root essentials | Daily | S3 | — |
|
||
| Hetzner snapshots | Weekly Mon | Hetzner Cloud | — |
|
||
|
||
**DR Standby:** app1-bu (Hetzner) boots and auto-restores from S3 if Core is down for 2+ minutes.
|
||
|
||
---
|
||
|
||
## Email Accounts
|
||
|
||
| Address | Provider | Server | Purpose |
|
||
|---|---|---|---|
|
||
| email@redacted | MXroute | mail.germainebrown.com | Germaine's primary |
|
||
| email@redacted | MXroute | mail.germainebrown.com | Sho'Nuff's outbox |
|
||
| email@redacted | SiteGround | c1113726.sgvps.net:2525/993 | Apex business |
|
||
| email@redacted | MXroute | — | DRE client-facing |
|
||
| email@redacted | MXroute | — | DRE debtor-facing |
|
||
| email@redacted | MXroute | — | Garrison Brown |
|
||
| email@redacted | MXroute | — | Greyson Brown |
|