124 lines
4.0 KiB
Markdown
124 lines
4.0 KiB
Markdown
# Hermes Memory Consolidation — Architecture Proposal
|
|
|
|
**Author:** Sho'Nuff
|
|
**Date:** July 9, 2026
|
|
**Status:** DRAFT — awaiting Network Services Team input
|
|
|
|
---
|
|
|
|
## 1. Current Problem
|
|
|
|
MEMORY.md is a flat list of freeform text entries separated by `§`.
|
|
The consolidation system (`hermes-consolidate.py`) prunes stale entries using:
|
|
|
|
- **A static PROTECT regex** listing keywords that prevent deletion
|
|
- **Stale-word detection** (`complete`, `done`, `resolved`, etc.)
|
|
- **A size safety valve** that drops oldest-entered entries when over 7,000 chars
|
|
|
|
**Failure observed on first run:** 3 critical entries were pruned because they
|
|
didn't match the PROTECT regex. The S3 backup preserved them, but without that
|
|
safety net they would have been permanently lost.
|
|
|
|
### Root Cause
|
|
|
|
The system has no way to **judge importance**. It treats all unprotected entries
|
|
equally and has no semantic understanding of what's durable vs ephemeral.
|
|
|
|
---
|
|
|
|
## 2. Proposed Solutions
|
|
|
|
### Option A: Priority Tagging (Recommended)
|
|
|
|
Add explicit priority metadata to each entry:
|
|
|
|
```
|
|
[P1] Core: netcup KVM, 8C/15G/512GB, Debian 13...
|
|
[P1] Germaine's #1 rule...
|
|
[P2] Apex on wphost02...
|
|
[P3] Shark game updates complete...
|
|
```
|
|
|
|
**Pruner logic:**
|
|
1. Keep all `[P1]` entries (never pruned)
|
|
2. Keep `[P2]` entries unless over 80% of limit
|
|
3. Prune `[P3]` entries first (completed tasks, transient state)
|
|
4. Never prune below `[P1]` + `[P2]` combined size
|
|
|
|
**Pros:** Simple, explicit, user can tag importance when writing
|
|
**Cons:** Requires tagging discipline at entry creation time
|
|
|
|
### Option B: Dual-Store Architecture
|
|
|
|
Split memory into two files:
|
|
|
|
```
|
|
memories/MEMORY.md — Durable facts (never auto-pruned)
|
|
memories/WORKING.md — Transient state (auto-pruned aggressively)
|
|
```
|
|
|
|
**Durable store (MEMORY.md):** identity, rules, credentials, infrastructure,
|
|
user preferences, team definitions. Only manually edited. No auto-prune.
|
|
|
|
**Working store (WORKING.md):** task status, queue items, completed work,
|
|
session notes, temporary context. Auto-pruned every 10 min.
|
|
|
|
**Pros:** No risk of losing durable facts. Working store can be more aggressive.
|
|
**Cons:** Two files to manage. Boundary between them requires judgement.
|
|
|
|
### Option C: Template-Based Structuring
|
|
|
|
Each entry follows a defined schema:
|
|
|
|
```yaml
|
|
type: infrastructure | rule | preference | task | personal | team
|
|
priority: p1 | p2 | p3
|
|
expires: 2026-08-01 | never
|
|
content: ...
|
|
```
|
|
|
|
**Pruner:**
|
|
- `expires: never` and `priority: p1` → never pruned
|
|
- `priority: p3` or past expiry → always pruned
|
|
- Others → size-based pruning
|
|
|
|
**Pros:** Surgical precision. No false positives.
|
|
**Cons:** Highest initial investment. More complex parsing. Schema changes
|
|
require migration.
|
|
|
|
---
|
|
|
|
## 3. Comparison
|
|
|
|
| Criteria | Current | Option A (Tags) | Option B (Dual) | Option C (Schema) |
|
|
|----------|---------|-----------------|------------------|-------------------|
|
|
| Setup effort | ✅ Done | 🟡 1 hour | 🟡 1 hour | 🔴 3+ hours |
|
|
| Risk of data loss | 🔴 HIGH | 🟡 Low | ✅ Minimal | ✅ Minimal |
|
|
| Maintenance burden | 🟡 Medium | 🟡 Low | ✅ Low | 🔴 Medium |
|
|
| User discipline needed | 🔴 None (but risky) | 🟡 Some | 🟡 Some | 🔴 High |
|
|
| Machine-readable | ❌ No | 🟡 Partial | 🟡 Partial | ✅ Yes |
|
|
| Migration complexity | — | 🟡 Low | 🟡 Low | 🔴 Medium |
|
|
|
|
---
|
|
|
|
## 4. Recommendation
|
|
|
|
**Option A (Priority Tagging) + apply to dual-store concept:**
|
|
|
|
- Keep one MEMORY.md for now
|
|
- Add `[P1]`, `[P2]`, `[P3]` tags
|
|
- Auto-tag entries where possible (infrastructure = P1, completed tasks = P3)
|
|
- The consolidation system protects P1 + P2, prunes P3 aggressively
|
|
- If the file still grows, split into durable/working stores in v2
|
|
|
|
---
|
|
|
|
## 5. DR Plan per Server — Note
|
|
|
|
This proposal covers Hermes memory only. Germaine has also requested a
|
|
comprehensive Disaster Recovery plan for **each server** in the ITPP
|
|
infrastructure. That is a separate deliverable currently being compiled
|
|
in `/root/.hermes/references/server-dr-plans.md`.
|
|
|
|
*Network Services Team review requested.*
|