71 lines
2.7 KiB
Markdown
71 lines
2.7 KiB
Markdown
# DRE Mail Poller Pattern
|
|
|
|
Session: July 8, 2026
|
|
|
|
## Overview
|
|
|
|
Lightweight no-LLM IMAP poller that checks multiple DRE mailboxes on MXroute, parses headers + body, and writes a rolling JSON database for a web portal.
|
|
|
|
## Key Files
|
|
|
|
| File | Purpose |
|
|
|------|---------|
|
|
| `/root/.hermes/scripts/dre-mail-poller.py` | Main poller script |
|
|
| `/var/www/internal/data/dre-mails.json` | Output JSON database |
|
|
| `/var/log/dre-mail-poller.log` | Log file |
|
|
| `/etc/cron.d/dre-mail-poller` | Cron config (every minute) |
|
|
|
|
## IMAP Server
|
|
|
|
- **Host:** `heracles.mxrouting.net:993`
|
|
- **Mailboxes:**
|
|
- `dre@debtrecoveryexperts.com` / `M8ke.Money.Honey!`
|
|
- `collections@debtrecoveryexperts.com` / `M8ke.Money.Honey!`
|
|
|
|
The password is shared across both mailboxes — the MXroute cPanel config uses the same password for all mailboxes under that domain.
|
|
|
|
## Script Architecture
|
|
|
|
```python
|
|
MAILBOXES = [
|
|
{"label": "dre", "user": "dre@debtrecoveryexperts.com", "pass": "M8ke.Money.Honey!"},
|
|
{"label": "collections", "user": "collections@debtrecoveryexperts.com", "pass": "M8ke.Money.Honey!"},
|
|
]
|
|
```
|
|
|
|
Each mailbox iterates independently. A connection failure on one doesn't block the other.
|
|
|
|
## Deduplication
|
|
|
|
Composite key: `{label}:{IMAP_UID}`. UIDs are stable per-mailbox in IMAP. Existing IDs loaded from JSON file into a `set()` on each run; new messages appended only when `id` not in set.
|
|
|
|
## Output schema
|
|
|
|
```json
|
|
{
|
|
"id": "dre:12345",
|
|
"mailbox": "dre",
|
|
"from": "John Doe <john@example.com>",
|
|
"to": "dre@debtrecoveryexperts.com",
|
|
"subject": "Invoice #123",
|
|
"date": "2025-07-08T09:38:00+00:00",
|
|
"body_preview": "First 500 chars...",
|
|
"body": "Full plain-text body...",
|
|
"is_read": false,
|
|
"claim_match": "DRE-2025-0001"
|
|
}
|
|
```
|
|
|
|
`claim_match` is extracted via `re.search(r'DRE-\d{4}-\d{4}', subject)`.
|
|
|
|
## Caddy integration
|
|
|
|
No Caddy changes needed — the existing `internal.debtrecoveryexperts.com` block already has `root * /var/www/internal` and `file_server`. The JSON is served at `/data/dre-mails.json` without any extra config.
|
|
|
|
## Pitfalls
|
|
|
|
- **Connection timeouts:** MXroute can be slow (1-2s per connect). Script uses `imaplib.IMAP4_SSL` with default timeout which is usually sufficient.
|
|
- **Large bodies:** Some emails have megabytes of HTML. Body extraction with stdlib is fast but the rolling DB cap (500 entries) prevents unbounded growth.
|
|
- **UNSEEN only:** The poller fetches `UNSEEN` and never marks messages as `SEEN`. This means the same mail appears on every poll until a real email client reads it. The dedup set prevents duplicate entries in the JSON.
|
|
- **Body extraction order:** `text/plain` → `text/html` → any `text/*`. HTML-only messages (common from Apple Mail replies) are handled by the HTML fallback.
|