Initial skills documentation — 25 categories, all SKILL.md + references + scripts

2026-07-15 17:42:20 -04:00
commit 1b4fcd4427
976 changed files with 188344 additions and 0 deletions
@@ -0,0 +1,66 @@
+# Inbox Solicitation Triaging — Design & Pitfalls
+
+Pattern for cleaning an inbox by classifying and moving solicitations.
+
+## Core Technique
+
+1. **Connect via IMAP (imaplib)**, select INBOX, fetch ALL messages
+2. **Iterate each message** with `BODY.PEEK[]` (non-destructive read — doesn't mark as seen)
+3. **Run multi-heuristic solicitor detection** on sender domain, subject line, and body text
+4. **Copy flagged to target folder**, then `+FLAGS \\Deleted` + `EXPUNGE` from INBOX
+5. **Save detailed JSON report** and print a grouped summary
+
+## Multi-Heuristic Detection Strategy
+
+Four layered checks:
+
+### 1. Allowlist Bypass (highest priority)
+- **Internal senders** (first-name patterns, company domain): skip entirely
+- **Legit domains** (`boxpilotlogistics.com`, etc.): skip entirely
+
+### 2. Subject-Line Keywords (high signal)
+Immediate giveaways: "insurance quote", "free quote", "factoring", "compliance", "load board", "mc number", "advertisement"
+
+### 3. Body Content Analysis (by sender domain class)
+
+**Free/personal domains** (gmail, yahoo, outlook, etc. — defined in FREE_DOMAINS set):
+- If sender has free domain AND references an external business domain in the body → strong solicitation signal
+- Also scan for solicitation keywords + phrases in body text
+
+**Business-domain senders** (not free, not internal):
+- Count multiple trucking-solicitation signals (keywords + phrases)
+- Threshold: 2+ signals = solicitation
+
+**Transactional invoice guard:** If subject/body contains ≥2 invoice keywords ("invoice", "receipt", "payment confirmation") and no other signals triggered, keep in inbox.
+
+### 4. Combined Signal Accumulation
+- Body keywords: +1 each
+- Solicitation phrases: +2 each (trucking-specific phrases are stronger)
+- Subject keyword match: +1
+- Threshold: 2+ total = flagged
+
+## Known False-Positive Categories
+
+| Category | Why flagged | Typical senders | Action |
+|---|---|---|---|
+| Government registration notices | Body has "fmcsa", "usdot", "authority" | ucr.gov, login.gov, dot.gov | Move back — official notices |
+| Bank product promos | Boilerplate contains "eld", "limited time" | americanexpress@blueprint.americanexpress.com | Review individually |
+| Slack onboarding | Template has "eld", "limited time" in footer | no-reply@email.slackhq.com | Keep transactional, move promos |
+| FMCSA process agent confirmations | Body has "boc-3", "process agent", "fmcsa" | fmcsaprocessagents.com | Keep if service was ordered |
+
+## Pitfalls
+
+- **HTML boilerplate noise**: Some email templates (Amex, Slack) contain trucking keywords like "eld" in their footer HTML. The heuristic catches them as false positives. Mitigation: whitelist known transactional templates or strip common boilerplate before matching.
+- **UCR.gov is legitimate** but its subject/body triggers "compliance", "authority", "usdot". These are official registration notices, not solicitations.
+- **BODY.PEEK[] vs BODY[]**: Always use PEEK to avoid marking messages as read during review.
+- **IMAP COPY + STORE \\Deleted** is the portable move pattern. Some servers support `MOVE` but not all.
+- **conn.expunge()** must be called after all deletions or the messages remain hidden but not freed.
+- **Character encodings**: Fall back through `utf-8` → `iso-8859-1` → `windows-1252` when `get_content_charset()` returns None.
+- **HTML-only emails**: Need regex tag stripping (`re.sub(r'<[^>]+>', ' ', body)`) before keyword matching. CSS class names (`style-Xia0GT4t_-text`) are noise — filter them out.
+
+## Script
+
+See `scripts/inbox-solicitation-cleaner.py` — a standalone runnable script with the full heuristic set. Adjust the Config block (host, credentials, folder name) and run:
+```bash
+python3 scripts/inbox-solicitation-cleaner.py
+```