cron/output: stop mirroring job output to S3 (270k objects -> 8), allow-list job state files in live-sync + restore + standby sync

CHANGELOG.md

@@ -29,6 +29,27 @@ Result: a full cycle now completes in **13 seconds** with a 9-line log and **zer
 
 # itpp-infrastructure — CHANGELOG
 
+## 2026-09-15 (third) — Core's stale job output: root cause, cleanup, allow-list fix
+
+**Root cause.** `hermes-live-sync.sh` mirrors `~/.hermes/` to `s3://hermes-vps-backups/live/` every 11 minutes and had **no `cron/output` exclusion**, while `aws s3 sync` only ever adds and updates, never deletes. So every cron job-run output Core has ever produced is still in S3 (July 5 onward), long after Core's own 50-runs-per-job retention pruned it locally. Measured 2026-09-15: **270,262 objects / 624 MB** under `live/cron/output/` against 16,718 files on Core. The cost was never the storage, it was the listing: every restore and every standby sync had to enumerate 270k objects, which is what pushed the standby sync past its own 10-minute interval and made two run concurrently.
+
+**Cleanup**
+
+- **Core local** `~/.hermes/cron/output`: 88 MB / 16,718 files -> **36 MB / 3,921 files**. Removed three output dirs belonging to jobs that no longer exist in `jobs.json` (`0137c3505b72`, `677fc4199392`, `77b99f6e2ff6`; 12,733 files, output stopped mid-July) plus 112 stale flat-file run logs older than 30 days. All eight live state files verified md5-identical before and after.
+- **core-bu**: deleted the frozen 1.2 GB / 194,819-file pulled copy. A standby never reads `cron/output` and the sync no longer pulls it.
+- **S3** `live/cron/output/`: 270,262 objects / 624 MB -> **8 objects / 74 KB**. Deleted 136 per-job prefixes and 112 loose run logs. No history lost: the `hermes-full-backup` tarballs (70 of them, back to 2026-07-04) include `cron/output`.
+
+**Fix — allow-list, not a wildcard exclude.** Applied to Core `hermes-live-sync.sh`, `hermes-standby-restore.sh` (Core archival copy, core-bu's live copy, S3 `standby/`), and core-bu `hermes-standby-sync.sh`:
+
+```
+--exclude "cron/output/*"
+--include "cron/output/osint-discovery.json"   (plus 7 more, named exactly, per script)
+```
+
+A wildcard include such as `--include "cron/output/*.md"` does **not** work: `*` also matches `/`, so it re-admits files nested inside job subdirs. This was proven on a synthetic tree before deploying. The eight allow-listed files are the shared job state jobs read for dedupe (so a takeover does not re-alert everything); run output has no failover value.
+
+**Verification.** After the prune, the 3,921 job-output files still existed locally but were gone from S3, so any failure to exclude would visibly re-upload them. A full live-sync then ran to completion and `live/cron/output/` still held exactly 8 objects. Rollback: `hermes-live-sync.sh.bak-20260915-cronout`, `hermes-standby-restore.sh.bak-20260915-cronout`, `hermes-standby-sync.sh.bak2-20260915` on core-bu.
+
 ## 2026-09-15 — core-bu Warm Standby BUILT and proven (armed later the same day, see the entry above)
 
 - **core-bu (`159.195.204.203`, netcup Nuremberg) is now a working Hermes warm standby.** Every step was executed on the box and verified by reading the result back.