# Hermes DR Plan v2 — Pending Implementation Checklist

**Submitted by Germaine:** July 9, 2026
**Status:** Planned (not yet implemented)
**Supersedes:** v1 ping-only watchdog

## Key Architecture Changes

- Two-layer health check (ping + HTTP health endpoint)
- SSH fencing + S3 ownership lock (prevents split-brain)
- Pre-failover integrity check (SQLite PRAGMA)
- 10-min VACUUM snapshots with integrity gate
- Backward heartbeat (Core monitors standby)
- app1-bu upgrade CPX11 → CPX21 for local Ollama (+$6/mo)

## 16-Item Checklist

- [ ] Upgrade app1-bu from CPX11 to CPX21 (Hetzner API)
- [ ] Install Ollama + llama3.2:3b on app1-bu
- [ ] Deploy /healthz endpoint on Core (Tailscale-only, bearer token)
- [ ] Deploy two-layer watchdog on app1-bu (Layer 1 + Layer 2)
- [ ] Deploy SSH fencing script on app1-bu
- [ ] Deploy S3 ownership lock with refresh
- [ ] Deploy 10-min VACUUM snapshot cron on Core
- [ ] Deploy pre-failover integrity check on app1-bu
- [ ] Deploy backward heartbeat (Core → app1-bu over Tailscale SSH)
- [ ] Update hermes-standby-deployment skill with v2 architecture
- [ ] Whitelist cross-box IPs in fail2ban both directions
- [ ] Verify: 4 consecutive pings → auto-takeover (server dead)
- [ ] Verify: Hermes dead but server up → alert + fence → conditional takeover
- [ ] Verify: state.db corruption → falls back to snapshot, not corrupt file
- [ ] Verify: Core reboot with standby holding lock → Core stays dormant
- [ ] Update dr-issue-log.md