# Ops Portal — ops.itpropartner.com

## Architecture
- **Server:** Core (152.53.192.33, netcup RS 2000)
- **Backend:** FastAPI at `/opt/ops-portal/server.py` (port 8090)
- **Proxy:** Caddy → reverse_proxy to 127.0.0.1:8090
- **Static files:** `/opt/ops-portal/static/` — 11 HTML pages, ops.css, app.js, utils.js
- **Auth:** JWT via `POST /api/auth/login`, token in localStorage
- **Data:** `/var/www/ops/data/ops-status.json` (5-min collector refresh)
- **Collector:** `/root/.hermes/scripts/ops-data-collector.py` — Wazuh, Bitdefender, S3, UISP, system health
- **Systemd:** `ops-portal.service`, env from `/root/.hermes/.env`
- **Credentials:** ippadmin (password → Vaultwarden / `~/.hermes/.env`)

## Pages (11 total)
| Page | Path | Description |
|------|------|-------------|
| Dashboard | / | System health, widgets, audit log |
| Services | /services.html | Systemd service control, audit log, server column |
| Servers | /servers.html | 5 servers with ping health |
| Network | /network.html | UISP data (44 sites, 90 devices), DNS zones |
| Backups | /backups.html | S3 bucket status (6 buckets) |
| FleetTracker | /fleettracker360.html | Traccar device tracking |
| Cron Jobs | /cron.html | Hermes cron jobs with expandable scripts |
| Config | /config.html | Active configs, /root/.hermes/scripts/ listing |
| Logs | /logs.html | Aggregated log viewer |
| Audit | /audit.html | Full audit trail |
| Costs | /cost.html | API cost tracking by model |

## API Endpoints
| Method | Path | Auth | Description |
|--------|------|------|-------------|
| POST | /api/auth/login | No | JWT authentication |
| GET | /api/health | No | Health check, DB status |
| GET | /api/status | JWT | Full dashboard data (17 sections) |
| GET | /api/servers | JWT | Server list with IPs |
| GET | /api/servers/health | JWT | Ping health (all 7 LIVE) |
| GET | /api/audit-log?limit=N | JWT | Audit trail entries |
| GET | /api/ft360/status | JWT | FleetTracker device data |

## Critical Services (API restart blocked)
hermes, hermes-assistant, hermes-browser, caddy, ops-portal, mysql-tunnel

## Dashboard Widgets
- System Health — Core metrics (jobs, disk, memory, S3, APIs)
- Wazuh Security — agent count, alerts
- Bitdefender GravityZone — 9 managed endpoints
- Alerts and Notifications — DR issues, backup failures, cron errors
- Quick Actions — Restart Ops Portal

## Recovery
```
systemctl restart ops-portal
systemctl reload caddy
python3 /root/.hermes/scripts/ops-data-collector.py
```