docs: ops portal and backup-restore documentation (2026-07-20)
This commit is contained in:
root
@@ -0,0 +1,30 @@
|
||||
# Backup-Restore Changelog
|
||||
|
||||
## 2026-07-20 — Restore logging and manual backup
|
||||
|
||||
### Added
|
||||
- **Backup Now button:** Green "+ Backup Now" button on backup page
|
||||
- **Domain dropdown:** Select element with all 9 hosted domains
|
||||
- **Note field:** Optional "why" note saved as note.txt in snapshot
|
||||
- **Restore History section:** Auto-expanded table at bottom — Domain, Snapshot, Date/Time, Status
|
||||
- **Restore logging:** Every restore writes to `/opt/backup-restore/logs/restore.log`
|
||||
- **Status formatting:** Green OK / red FAILED with centered status column
|
||||
|
||||
### Fixed
|
||||
- **Restore timing out:** Caddy flush_interval added + 300s transport timeouts
|
||||
- **Route ordering:** `/api/restore` decorator was stacked on backup function → reconnected to restore function
|
||||
- **API routing:** `/api/restore`, `/api/backup`, `/api/log` not proxied → added to Caddy config
|
||||
- **Mobile toggle:** Inline `display:none` on site tables removed → CSS class toggle now works
|
||||
- **Mobile touch:** role="button", tabindex="0", Enter key support added to card headers
|
||||
- **Auto-expand first domain:** Removed — all domains now start collapsed
|
||||
- **Restore History auto-expanded:** tbl-log has class="show", arrow is ▼
|
||||
|
||||
### Changed
|
||||
- "Backup Log History" → "Restore History"
|
||||
- Config page scripts directory now shows content when clicked
|
||||
|
||||
## 2026-07-17 — Initial deployment
|
||||
- Flask app deployed on app3 as systemd service
|
||||
- Snapshot script scheduled (1 AM, 1 PM)
|
||||
- 9 WordPress sites configured for backup
|
||||
- Caddy proxy from Core via my.itpropartner.com
|
||||
@@ -0,0 +1,50 @@
|
||||
# Backup-Restore — my.itpropartner.com/backups/
|
||||
|
||||
## Architecture
|
||||
- **Server:** app3 (152.53.241.111, netcup RS 4000)
|
||||
- **Backend:** Flask Python app at `/opt/backup-restore/app/app.py` (port 8090)
|
||||
- **Proxy:** Caddy on Core → reverse_proxy to 152.53.241.111:8090 with 300s timeouts
|
||||
- **Snapshots:** `/opt/backup-restore/snapshots/<domain>/<timestamp>/`
|
||||
- **Scheduled:** `0 1,13 * * * /opt/backup-restore/snapshot.sh` — 1 AM and 1 PM daily
|
||||
- **Systemd:** `backup-restore.service`
|
||||
- **Retention:** 30 days (auto-cleanup)
|
||||
|
||||
## Sites Backed Up (9 domains)
|
||||
apextrackexperience.com, boxpilotlogistics.com, debtrecoveryexperts.com, iamgmb.com, katiewattdesign.com, katiewattsdesign.com, mainwp.itpropartner.com, vigilanttac.com, voipsimplicity.com
|
||||
|
||||
## Snapshot Contents
|
||||
Each snapshot directory contains:
|
||||
- `files.tar.gz` — WordPress document root tarball
|
||||
- `database.sql` — MySQL dump
|
||||
- `size.txt` — Total backup size in bytes
|
||||
- `note.txt` — Optional manual backup note
|
||||
|
||||
## API Endpoints
|
||||
| Method | Path | Auth | Description |
|
||||
|--------|------|------|-------------|
|
||||
| GET | /backups/ | No | Backup dashboard page |
|
||||
| POST | /api/backup | No | Trigger manual backup |
|
||||
| POST | /api/restore | No | Restore a snapshot |
|
||||
| POST | /api/delete | No | Delete a snapshot |
|
||||
| GET | /api/download/<domain>/<id> | No | Download snapshot archive |
|
||||
| GET | /api/log | No | Restore history |
|
||||
|
||||
## Caddy Routes (on Core)
|
||||
```
|
||||
handle /api/backup → app3:8090
|
||||
handle /api/restore → app3:8090 (flush_interval -1, 300s timeouts)
|
||||
handle /api/download/* → app3:8090
|
||||
handle /api/log → app3:8090
|
||||
handle_path /backups/* → app3:8090 (300s timeouts)
|
||||
```
|
||||
|
||||
## Recovery
|
||||
```
|
||||
systemctl restart backup-restore
|
||||
# Manual snapshot:
|
||||
/opt/backup-restore/snapshot.sh
|
||||
# Manual restore via curl:
|
||||
curl -X POST https://my.itpropartner.com/api/restore \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"domain":"katiewattdesign.com","snapshot":"2026-07-20_130001"}'
|
||||
```
|
||||
@@ -0,0 +1,39 @@
|
||||
# Ops Portal Changelog
|
||||
|
||||
## 2026-07-20 — Major audit and fix session
|
||||
|
||||
### Fixed
|
||||
- **`/api/health` returning 404:** Caddy `handle_path` was stripping the path → changed to `handle`, port corrected to 8090
|
||||
- **`/api/servers` returning null:** Server list now returns all 5 servers with live ping health
|
||||
- **Server IPs stale:** app1-bu updated to 5.161.225.131, legacy entries removed
|
||||
- **Page titles inconsistent:** All 11 pages standardized to "X — IT Pro Partner Ops" format
|
||||
- **Missing nav icons:** All 11 nav items now have SVG icons
|
||||
- **FleetTracker360 missing from nav:** Added to navigation with car icon
|
||||
- **Backups page no data:** `s3_buckets` → `s3_backups` key fix
|
||||
- **FleetTracker360 page no nav:** Added ops.css, app.js, utils.js includes
|
||||
- **Network page dark sidebar:** Replaced with standard top nav bar
|
||||
- **Cache-busting broken:** All JS/CSS references now versioned with timestamps
|
||||
- **Mobile nav broken:** `.nav-links.open` CSS rule missing → hamburger menu now toggles properly on iOS/Android
|
||||
- **Auth guard race condition:** IIFE scripts replaced with DOMContentLoaded event listeners — pages now load data when user is authenticated
|
||||
- **Cost page broken:** Missing `loadData` function → defined and wired
|
||||
- **Dependency diagram 404:** File copied to static dir, link corrected
|
||||
- **Logs page mangled title:** Triple-nested `<title>` tags from sed accident → cleaned
|
||||
- **Config page scripts directory:** Now populates directory listing when clicked
|
||||
- **Services page:** Server column added showing "Core (152.53.192.33)"
|
||||
- **Dashboard auto-refresh on tab focus:** Visibility API handler added
|
||||
- **Critical service protection:** hermes, caddy, ops-portal restarts blocked via API
|
||||
|
||||
### Removed
|
||||
- Duplicate server entries: "app1 (AI Stack)" and "Docker Box (legacy)"
|
||||
- Server count: 7 → 5 clean entries
|
||||
|
||||
### Changed
|
||||
- Admin credentials: germaine/itpp2026! → ippadmin/LoveMyBoys.1520!
|
||||
- JWT_SECRET made persistent in /root/.hermes/.env to survive restarts
|
||||
- Collector timeout: 20s → 90s to handle 94K-file S3 bucket scanning
|
||||
|
||||
## Jul 17, 2026 — Initial deployment
|
||||
- Ops portal deployed on Core as FastAPI app
|
||||
- Caddy reverse proxy configured
|
||||
- 10 HTML pages created
|
||||
- Ops collector built for S3, system health, server status
|
||||
@@ -0,0 +1,55 @@
|
||||
# Ops Portal — ops.itpropartner.com
|
||||
|
||||
## Architecture
|
||||
- **Server:** Core (152.53.192.33, netcup RS 2000)
|
||||
- **Backend:** FastAPI at `/opt/ops-portal/server.py` (port 8090)
|
||||
- **Proxy:** Caddy → reverse_proxy to 127.0.0.1:8090
|
||||
- **Static files:** `/opt/ops-portal/static/` — 11 HTML pages, ops.css, app.js, utils.js
|
||||
- **Auth:** JWT via `POST /api/auth/login`, token in localStorage
|
||||
- **Data:** `/var/www/ops/data/ops-status.json` (5-min collector refresh)
|
||||
- **Collector:** `/root/.hermes/scripts/ops-data-collector.py` — Wazuh, Bitdefender, S3, UISP, system health
|
||||
- **Systemd:** `ops-portal.service`, env from `/root/.hermes/.env`
|
||||
- **Credentials:** ippadmin / LoveMyBoys.1520!
|
||||
|
||||
## Pages (11 total)
|
||||
| Page | Path | Description |
|
||||
|------|------|-------------|
|
||||
| Dashboard | / | System health, widgets, audit log |
|
||||
| Services | /services.html | Systemd service control, audit log, server column |
|
||||
| Servers | /servers.html | 5 servers with ping health |
|
||||
| Network | /network.html | UISP data (44 sites, 90 devices), DNS zones |
|
||||
| Backups | /backups.html | S3 bucket status (6 buckets) |
|
||||
| FleetTracker | /fleettracker360.html | Traccar device tracking |
|
||||
| Cron Jobs | /cron.html | Hermes cron jobs with expandable scripts |
|
||||
| Config | /config.html | Active configs, /root/.hermes/scripts/ listing |
|
||||
| Logs | /logs.html | Aggregated log viewer |
|
||||
| Audit | /audit.html | Full audit trail |
|
||||
| Costs | /cost.html | API cost tracking by model |
|
||||
|
||||
## API Endpoints
|
||||
| Method | Path | Auth | Description |
|
||||
|--------|------|------|-------------|
|
||||
| POST | /api/auth/login | No | JWT authentication |
|
||||
| GET | /api/health | No | Health check, DB status |
|
||||
| GET | /api/status | JWT | Full dashboard data (17 sections) |
|
||||
| GET | /api/servers | JWT | Server list with IPs |
|
||||
| GET | /api/servers/health | JWT | Ping health (all 7 LIVE) |
|
||||
| GET | /api/audit-log?limit=N | JWT | Audit trail entries |
|
||||
| GET | /api/ft360/status | JWT | FleetTracker device data |
|
||||
|
||||
## Critical Services (API restart blocked)
|
||||
hermes, hermes-assistant, hermes-browser, caddy, ops-portal, mysql-tunnel
|
||||
|
||||
## Dashboard Widgets
|
||||
- System Health — Core metrics (jobs, disk, memory, S3, APIs)
|
||||
- Wazuh Security — agent count, alerts
|
||||
- Bitdefender GravityZone — 9 managed endpoints
|
||||
- Alerts and Notifications — DR issues, backup failures, cron errors
|
||||
- Quick Actions — Restart Ops Portal
|
||||
|
||||
## Recovery
|
||||
```
|
||||
systemctl restart ops-portal
|
||||
systemctl reload caddy
|
||||
python3 /root/.hermes/scripts/ops-data-collector.py
|
||||
```
|
||||
@@ -0,0 +1,30 @@
|
||||
# Project Log — All Completed Projects
|
||||
|
||||
## 2026-07-20
|
||||
|
||||
### Ops Portal Audit and Overhaul
|
||||
- Full audit of all 11 pages, 7 API endpoints, and 5 dashboard widgets
|
||||
- Fixed 15 bugs: auth guards, cache-busting, mobile nav, page titles, missing icons, data keys
|
||||
- Added 3 new widgets: Wazuh Security, Bitdefender GravityZone, Alerts and Notifications
|
||||
- Standardized credentials: ippadmin / LoveMyBoys.1520!
|
||||
- Added critical service protection (hermes/caddy/ops-portal restart blocked via API)
|
||||
- Server list cleaned up (7→5), dependency diagram fixed, config page scripts listing
|
||||
|
||||
### Backup-Restore Enhancements
|
||||
- Added manual backup with domain dropdown and note field
|
||||
- Added restore history logging with formatted 4-column table
|
||||
- Fixed Caddy routing and timeouts (restore was returning 404 via proxy)
|
||||
- Fixed mobile toggle on domain expansion cards
|
||||
- 9 WordPress sites under daily backup (1 AM and 1 PM)
|
||||
|
||||
### Docs Written
|
||||
- `/root/projects/ops-portal/README.md` + `CHANGELOG.md`
|
||||
- `/root/projects/backup-restore/README.md` + `CHANGELOG.md`
|
||||
|
||||
---
|
||||
|
||||
## 2026-07-17 — Backup-Restore Initial Deployment
|
||||
- Flask backup/restore app deployed on app3 (152.53.241.111)
|
||||
- Daily snapshots scheduled at 1 AM and 1 PM
|
||||
- Caddy reverse proxy from my.itpropartner.com
|
||||
- 9 WordPress sites configured
|
||||
Reference in New Issue
Block a user