60 lines
2.7 KiB
Markdown
60 lines
2.7 KiB
Markdown
# Standby Audit — July 8, 2026
|
|
|
|
## Server: app1-bu.itpropartner.com (5.161.114.8)
|
|
|
|
Hetzner CPX11 (2 vCPU / 2 GB RAM / 40 GB SSD) — warm standby for app1 (152.53.192.33, netcup).
|
|
|
|
### Power State
|
|
- Hetzner API: `running` (server is powered on, OS up)
|
|
- Per DR plan: documented as "🟢 Dormant" but the intent in `hermes-standby-deployment` skill says "Stay powered on with Hermes off" — so running is **correct**, but server-inventory.md calls it "dormant" which is ambiguous.
|
|
- **Note:** If electricity cost ($5/mo) is a concern, this could be a cold standby (boot on demand) with a ~60s slower failover.
|
|
|
|
### Hermes Gateway
|
|
- `systemctl is-active hermes-gateway` → `inactive` ✅
|
|
|
|
### Standby Service
|
|
- `systemctl is-active hermes-standby` → `active (exited)` ✅
|
|
- Last boot: Jul 5, 2026
|
|
- Uptime at audit: 3 days, 1:16
|
|
- Restore log: `Live box is REACHABLE — staying dormant (clean exit)` ✅
|
|
|
|
### Watchdog Cron
|
|
- `crontab -l` → `*/10 * * * * /root/.hermes/scripts/hermes-standby-watchdog.sh` ✅
|
|
- Watchdog log: empty (no failures detected) ✅
|
|
- Disk usage: 16G / 38G (43%)
|
|
|
|
### SSH Access
|
|
- Key: `itpp-infra` (Ed25519) ✅
|
|
- Recent auth log shows SSH from 152.53.192.33 (live box) — heartbeat or previous manual check
|
|
|
|
### S3 Artifacts
|
|
- `s3://hermes-vps-backups/standby/` — all 6 files present ✅
|
|
- `DR-PLAN.md` — 13 KB, present ✅
|
|
- `hermes-standby-restore.sh` — 2.8 KB, matches deployed version ✅
|
|
- `hermes-standby-watchdog.sh` — 4.3 KB, matches deployed version ✅
|
|
- `hermes-standby.service` — 367 B, matches deployed version ✅
|
|
- `server-inventory.md` — 8.1 KB, present ✅
|
|
- `recovery-bundle-2026-07-05.md` — 280 KB, present ✅
|
|
|
|
### Live Sync Freshness
|
|
- `hermes-live-sync` cron runs every 15 min on live box ✅
|
|
- `state.db` on S3: 2026-07-08T23:19:08Z (~5 min from audit time) ✅
|
|
- `gateway_state.json`: 2026-07-08T23:19:06Z ✅
|
|
- `channel_directory.json`: 2026-07-08T23:18:59Z ✅
|
|
- Total S3 `live/` data: 32,151 files / 4.8 GiB
|
|
|
|
### Credential Observations
|
|
- `HETZNER_API_TOKEN` exists as a shell env var
|
|
- `/root/.hermes/scripts/.hetzner_token` file is **missing** (deleted during prior cleanup?)
|
|
- Recovery bundle references the file approach — potential doc/setup drift
|
|
|
|
### Issues Found
|
|
1. Minor: server-inventory.md says "dormant" (implies off) but server is running — warm standby is correct per skill, but docs ambiguous
|
|
2. Minor: `.hetzner_token` file deleted; token only exists in env var — recovery bundle instructs restoring the file
|
|
3. No critical issues
|
|
|
|
### Next Audit Recommendation
|
|
- Quarterly or after any DR-plan changes
|
|
- Also verify the live-sync script hasn't drifted (exclusion patterns, credentials)
|
|
- Check Hetzner snapshot cron (`hetzner-weekly-snapshots`) still runs
|