From 81762a33bd4178186ccb2fa680c0ae109bbb6c36 Mon Sep 17 00:00:00 2001 From: root Date: Sat, 8 Aug 2026 18:20:03 -0400 Subject: [PATCH] =?UTF-8?q?CRITICAL=20#4:=20Auth=20API=20+=20Stack=20Auth?= =?UTF-8?q?=20backups=20=E2=80=94=20auth-api-backup.sh=20(SQLite=20hot),?= =?UTF-8?q?=20hexclave-backup.sh=20(PG=20dump=20+=20compose).=20Auth=20API?= =?UTF-8?q?=20Core=203:15=20AM,=20Hexclave=20app3=203:30=20AM.=20Update=20?= =?UTF-8?q?RPO/RTO=20tiers.=20Remove=20Technitium=20from=20Unbacked=20(now?= =?UTF-8?q?=20in=20Low=20tier).?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- backup-plan.md | 23 +++++++++++++++-------- 1 file changed, 15 insertions(+), 8 deletions(-) diff --git a/backup-plan.md b/backup-plan.md index a652c0c..d93a33e 100644 --- a/backup-plan.md +++ b/backup-plan.md @@ -21,6 +21,7 @@ | Uptime Kuma | `core-services-backup.sh` — SQLite DB dump | `s3://hermes-vps-backups/core/uptime-kuma/` | 1:30 AM | 2026-07-28 | | Docker Volumes | `core-services-backup.sh` — tar of key compose volumes | `s3://hermes-vps-backups/volumes/` | 1:30 AM | 2026-07-28 | | Prometheus | `core-services-backup.sh` — TSDB snapshot | `s3://hermes-vps-backups/core/prometheus/` | 1:30 AM | 2026-07-28 | +| Auth API | `auth-api-backup.sh` — SQLite .backup + .env | `s3://hermes-vps-backups/core/auth-api/` | 3:15 AM | 2026-08-08 | ### App1 (152.53.36.131) — RS 4000 @@ -45,6 +46,7 @@ | UNMS | `unms-backup-sync.sh` — auto-backup .unms | `s3://hermes-vps-backups/unms-backups/live/backups/` | 6:00 AM | 2026-07-28 | | UniFi | `unifi-backup-sync.sh` — auto-backup .unf | `s3://hermes-vps-backups/unifi-backups/` | 2:00 AM | 2026-07-18 | | Traccar | `app2-backup.sh` — H2 DB + config | `s3://hermes-vps-backups/app2/traccar/` | 2:30 AM | 2026-07-28 | +| Technitium DNS | `technitium-backup.sh` — data dir + compose | `s3://hermes-vps-backups/app2/technitium/` | 2:45 AM | 2026-08-08 | ### App3 (152.53.241.111) — RS 4000 @@ -55,6 +57,7 @@ | WordPress Files | `app3-backup.sh` — wp-content tar.gz | `s3://hermes-vps-backups/app3/wordpress/` | 3:00 AM | 2026-07-28 | | Nginx Configs | `app3-backup.sh` — sites-enabled + config | `s3://hermes-vps-backups/app3/config/` | 3:00 AM | 2026-07-28 | | WordPress Snapshots | `/opt/backup-restore/snapshot.sh` — per-site tar.gz | `/opt/backup-restore/snapshots/` (local, 30-day retention) | 1 AM / 1 PM | 2026-08-08 | +| Hexclave (Stack Auth) | `hexclave-backup.sh` — PG dump + compose + env | `s3://hermes-vps-backups/app3/hexclave/` | 3:30 AM | 2026-08-08 | ### wphost02 (5.161.62.38) — Hetzner CPX21 @@ -85,9 +88,11 @@ | 1:30 AM | Grafana, Uptime Kuma, Docker volumes, Prometheus | crontab | `core-services-backup.sh` | | 2:00 AM | Open WebUI, n8n, MCP configs (App1) + **UniFi sync** (App2) | crontab | `app1-backup.sh`, `unifi-backup-sync.sh` | | 2:30 AM | Vaultwarden (App1) + Traccar (App2) | Hermes cron / crontab | `vaultwarden-backup.sh`, `app2-backup.sh` | +| 2:45 AM | Technitium DNS (App2) | Hermes cron | `technitium-backup.sh` | | 3:00 AM | /root essentials + App3 (CloudPanel, MySQL, WP, Nginx) | crontab | `root-essentials-backup.sh`, `app3-backup.sh` | -| 3:30 AM | LiteLLM (App1) | Hermes cron | `litellm-backup.sh` | -| 3:45 AM | Komodo (App1) | Hermes cron | `komodo-backup.sh` | +| 3:15 AM | Auth API (Core) | Hermes cron | `auth-api-backup.sh` | +| 3:30 AM | Hexclave / Stack Auth (App3) + LiteLLM (App1) | Hermes cron / crontab | `hexclave-backup.sh`, `litellm-backup.sh` | +| 3:45 AM | Komodo (App1) | crontab | `komodo-backup.sh` | | 4:00 AM | DocuSeal (App1) | Hermes cron | `docuseal-backup.sh` | | 4:15 AM | Twenty CRM (App1) | Hermes cron | `twenty-backup.sh` | | 5:00 AM | wphost02 WordPress (SSH) | crontab | `wphost02-backup` | @@ -116,7 +121,10 @@ | `twenty-backup.sh` | Twenty CRM Postgres dump (SSH to App1) | Hermes cron 4:15 AM | | `app1-backup.sh` | Open WebUI, n8n, MCP configs (SSH to App1) | App1 crontab 2:00 AM | | `app2-backup.sh` | Traccar backup (SSH to App2) | App2 crontab 2:30 AM | +| `technitium-backup.sh` | Technitium DNS backup (SSH to App2) | Hermes cron 2:45 AM | | `app3-backup.sh` | CloudPanel, MySQL, WP, Nginx (SSH to App3) | crontab 3:00 AM | +| `auth-api-backup.sh` | Auth API SQLite + config (Core local) | Hermes cron 3:15 AM | +| `hexclave-backup.sh` | Stack Auth PG dump + compose + env (SSH to App3) | Hermes cron 3:30 AM | | `wphost02-backup.sh` | SSH to wphost02, backup all sites | crontab 5:00 AM | | `run-wisp-backup.sh` | MikroTik CCR config export | Hermes cron 6:00 AM | | `unms-backup-sync.sh` | UNMS auto-backup sync to S3 | Hermes cron 6:00 AM | @@ -196,9 +204,9 @@ mikrotik-ccr-backups/ |---|---|---|---|---| | **Critical** | Hermes Agent, Gitea, Traccar, UISP | ≤ 1 hour | ≤ 4 hours | Live sync + daily backups; restore from S3 then replay live-sync | | **High** | LiteLLM, n8n, Open WebUI, Vaultwarden, Twenty CRM | 24 hours | ≤ 8 hours | Daily backups; restore from previous night's dump | -| **Medium** | Hudu, UniFi, Komodo, DocuSeal, App3 WP sites | 24 hours | ≤ 24 hours | Daily backups only; acceptable overnight gap | -| **Low** | Grafana, Uptime Kuma, Prometheus, MikroTik CCR | 24 hours | ≤ 48 hours | Monitoring data is nice-to-have, not blocking | -| **Unbacked** | Dawarich, RAGFlow, Technitium DNS | N/A | N/A | No backup exists — see Unbacked Services below | +|| **Medium** | Hudu, UniFi, Komodo, DocuSeal, App3 WP sites, Auth API, Hexclave (Stack Auth) | 24 hours | ≤ 24 hours | Daily backups only; acceptable overnight gap; Auth API backs all SSO; Hexclave is customer-facing auth | +| **Low** | Grafana, Uptime Kuma, Prometheus, MikroTik CCR, Technitium DNS | 24 hours | ≤ 48 hours | Monitoring data is nice-to-have, not blocking; Technitium DNS is caching-only (no authoritative zones — public DNS is SiteGround/Cloudflare, servers use Tailscale MagicDNS) | +| **Unbacked** | Dawarich, RAGFlow | N/A | N/A | No backup exists — see Unbacked Services below | ## Restore Testing Cadence @@ -226,10 +234,9 @@ These services are running in production with **zero backup coverage**: | Service | Server | Data Path | Risk | |---|---|---|---| | **Dawarich** | app2 | Docker: `dawarich_db`, `dawarich_app` (4 containers) | Location history lost on disk failure | -| **RAGFlow** | app2 | Docker: `ragflow-cpu-1` | RAG pipeline state lost | -| **Technitium DNS** | app2 | `/root/docker/technitium/data` (bind mount) | All DNS zones lost on disk failure; zone files must be recreated manually | +|| **RAGFlow** | app2 | Docker: `ragflow-cpu-1` | RAG pipeline state lost | -> **Action needed:** Create backup scripts for all three. For Technitium, the data directory can be tar'd and sent to `s3://hermes-vps-backups/app2/technitium/`. For Dawarich and RAGFlow, determine database engines before designing backup strategy. +> **Action needed:** Create backup scripts for both remaining unbacked services. For Dawarich and RAGFlow, determine database engines before designing backup strategy. ---