Hermes Memory Consolidation — Architecture Proposal

Author: Sho'Nuff Date: July 9, 2026 Status: DRAFT — awaiting Network Services Team input

1. Current Problem

MEMORY.md is a flat list of freeform text entries separated by §. The consolidation system (hermes-consolidate.py) prunes stale entries using:

A static PROTECT regex listing keywords that prevent deletion
Stale-word detection (complete, done, resolved, etc.)
A size safety valve that drops oldest-entered entries when over 7,000 chars

Failure observed on first run: 3 critical entries were pruned because they didn't match the PROTECT regex. The S3 backup preserved them, but without that safety net they would have been permanently lost.

Root Cause

The system has no way to judge importance. It treats all unprotected entries equally and has no semantic understanding of what's durable vs ephemeral.

2. Proposed Solutions

Option A: Priority Tagging (Recommended)

Add explicit priority metadata to each entry:

[P1] Core: netcup KVM, 8C/15G/512GB, Debian 13...
[P1] Germaine's #1 rule...
[P2] Apex on wphost02...
[P3] Shark game updates complete...

Pruner logic:

Keep all [P1] entries (never pruned)
Keep [P2] entries unless over 80% of limit
Prune [P3] entries first (completed tasks, transient state)
Never prune below [P1] + [P2] combined size

Pros: Simple, explicit, user can tag importance when writing Cons: Requires tagging discipline at entry creation time

Option B: Dual-Store Architecture

Split memory into two files:

memories/MEMORY.md      — Durable facts (never auto-pruned)
memories/WORKING.md     — Transient state (auto-pruned aggressively)

Durable store (MEMORY.md): identity, rules, credentials, infrastructure, user preferences, team definitions. Only manually edited. No auto-prune.

Working store (WORKING.md): task status, queue items, completed work, session notes, temporary context. Auto-pruned every 10 min.

Pros: No risk of losing durable facts. Working store can be more aggressive. Cons: Two files to manage. Boundary between them requires judgement.

Option C: Template-Based Structuring

Each entry follows a defined schema:

type: infrastructure | rule | preference | task | personal | team
priority: p1 | p2 | p3
expires: 2026-08-01 | never
content: ...

Pruner:

expires: never and priority: p1 → never pruned
priority: p3 or past expiry → always pruned
Others → size-based pruning

Pros: Surgical precision. No false positives. Cons: Highest initial investment. More complex parsing. Schema changes require migration.

3. Comparison

Criteria	Current	Option A (Tags)	Option B (Dual)	Option C (Schema)
Setup effort	✅ Done	🟡 1 hour	🟡 1 hour	🔴 3+ hours
Risk of data loss	🔴 HIGH	🟡 Low	✅ Minimal	✅ Minimal
Maintenance burden	🟡 Medium	🟡 Low	✅ Low	🔴 Medium
User discipline needed	🔴 None (but risky)	🟡 Some	🟡 Some	🔴 High
Machine-readable	❌ No	🟡 Partial	🟡 Partial	✅ Yes
Migration complexity	—	🟡 Low	🟡 Low	🔴 Medium

4. Recommendation

Option A (Priority Tagging) + apply to dual-store concept:

Keep one MEMORY.md for now
Add [P1], [P2], [P3] tags
Auto-tag entries where possible (infrastructure = P1, completed tasks = P3)
The consolidation system protects P1 + P2, prunes P3 aggressively
If the file still grows, split into durable/working stores in v2

5. DR Plan per Server — Note

This proposal covers Hermes memory only. Germaine has also requested a comprehensive Disaster Recovery plan for each server in the ITPP infrastructure. That is a separate deliverable currently being compiled in /root/.hermes/references/server-dr-plans.md.

Network Services Team review requested.

4.0 KiB Raw Permalink Blame History