DRE customer portal: FastAPI + SQLite backend (18 endpoints)

- Magic-link auth (sha256-only token storage, 15-min single-use, 7-day sessions)
- Staff-key auth via X-DRE-Staff-Key (constant-time compare)
- SQLite WAL, foreign_keys, parameterized queries, atomic DRE/CLT sequence allocation
- Intake validator rejects SSN/PAN patterns (FDCPA/TDCPA compliance)
- Document upload allowlist + magic-byte check, 20MB cap
- Unified error envelope, money as integer cents
- systemd unit (port 8093, User=root, hardening directives)
- Fixes import bug (auth.py relative imports) and audit_log placeholder mismatch
This commit is contained in:
root
2026-08-21 18:43:33 -04:00
parent c573ed1a14
commit be0750d001
14 changed files with 2662 additions and 0 deletions
+23
View File
@@ -0,0 +1,23 @@
# DRE Portal backend environment — copy to /opt/dre-portal/.env and fill in secrets (chmod 600)
# Generated fresh DRE_STAFF_KEY on deploy (do NOT commit this file).
# 64-hex random staff key (X-DRE-Staff-Key header) — generate with: python3 -c "import secrets; print(secrets.token_hex(32))"
DRE_STAFF_KEY=__GENERATE_ME__
# SQLite DB path + uploads dir
DRE_DB_PATH=/opt/dre-portal/data/dre.db
DRE_UPLOAD_DIR=/opt/dre-portal/data/uploads
# Base URL for magic-link emails
DRE_BASE_URL=https://portal.debtrecoveryexperts.com
# SMTP relay (germainebrown.com:2525 STARTTLS) — best-effort, failures never fail the request
DRE_SMTP_HOST=mail.germainebrown.com
DRE_SMTP_PORT=2525
DRE_SMTP_FROM=dre@debtrecoveryexperts.com
DRE_TEAM_NOTIFY=dre@debtrecoveryexperts.com
DRE_SMTP_USER=shonuff@germainebrown.com
DRE_SMTP_PASS=__FROM_HIMALAYA_PASSFILE__
# Optional Cloudflare Turnstile (intake skips captcha if unset)
# TURNSTILE_SECRET=
+1
View File
@@ -0,0 +1 @@
# DRE Portal API package
+227
View File
@@ -0,0 +1,227 @@
"""Magic-link auth, sessions, staff-key, rate limiting (in-memory sliding window).
Constant-time comparison via hmac.compare_digest for token hashes + staff key.
"""
from __future__ import annotations
import hashlib
import hmac
import logging
import os
import secrets
import time
from collections import defaultdict, deque
from datetime import datetime, timedelta, timezone
from fastapi import Depends, HTTPException, Request, status
from . import db
from .db import get_conn, new_uuid, utcnow_iso
logger = logging.getLogger("dre.auth")
MAGIC_TOKEN_TTL_MIN = 15
SESSION_TTL_DAYS = 7
# ---------------------------------------------------------------
# Rate limiting — in-memory sliding window (single-instance v1)
# ---------------------------------------------------------------
class RateLimiter:
def __init__(self) -> None:
self._by_email: dict[str, deque[float]] = defaultdict(deque)
self._by_ip: dict[str, deque[float]] = defaultdict(deque)
def _prune(self, dq: deque[float], window_sec: float) -> None:
cutoff = time.time() - window_sec
while dq and dq[0] < cutoff:
dq.popleft()
def check_email(self, email: str, max_count: int, window_sec: float) -> bool:
dq = self._by_email[email]
self._prune(dq, window_sec)
if len(dq) >= max_count:
return False
dq.append(time.time())
return True
def check_ip(self, ip: str, max_count: int, window_sec: float) -> bool:
dq = self._by_ip[ip]
self._prune(dq, window_sec)
if len(dq) >= max_count:
return False
dq.append(time.time())
return True
def check(self, email: str | None, ip: str, max_email: int, email_window: float,
max_ip: int, ip_window: float) -> bool:
if email and not self.check_email(email, max_email, email_window):
return False
if not self.check_ip(ip, max_ip, ip_window):
return False
return True
_limiter = RateLimiter()
def rate_limit_auth_request(email: str | None, ip: str) -> bool:
return _limiter.check(email, ip, max_email=3, email_window=900, max_ip=10, ip_window=3600)
def rate_limit_auth_verify(ip: str) -> bool:
return _limiter.check(None, ip, max_email=999, email_window=1, max_ip=10, ip_window=900)
def rate_limit_intake(ip: str) -> bool:
return _limiter.check_ip(ip, max_count=20, window_sec=3600)
# ---------------------------------------------------------------
# Token / hash helpers
# ---------------------------------------------------------------
def _sha256_hex(value: str) -> str:
return hashlib.sha256(value.encode("utf-8")).hexdigest()
def generate_magic_token() -> str:
return secrets.token_urlsafe(32)
def generate_session_token() -> str:
return secrets.token_urlsafe(32)
def _compare_hash(a: str, b: str) -> bool:
return hmac.compare_digest(a, b)
# ---------------------------------------------------------------
# Staff key
# ---------------------------------------------------------------
def _staff_key() -> str:
return os.environ.get("DRE_STAFF_KEY", "")
def verify_staff_key(provided: str | None) -> bool:
key = _staff_key()
if not key or not provided:
return False
return _compare_hash(provided, key)
# ---------------------------------------------------------------
# FastAPI dependencies
# ---------------------------------------------------------------
def get_client_ip(request: Request) -> str:
# Cloudflare / Caddy may set X-Forwarded-For; use first hop
xff = request.headers.get("x-forwarded-for")
if xff:
return xff.split(",")[0].strip()
return request.client.host if request.client else "unknown"
def require_staff(request: Request) -> None:
"""Staff-key auth dependency. Raises 403 if missing/wrong."""
provided = request.headers.get("x-dre-staff-key")
if not verify_staff_key(provided):
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail={"code": "forbidden", "message": "Valid staff key required."},
)
def require_client(request: Request) -> dict:
"""Client session auth dependency. Returns {'client_id':..., 'client_number':...}.
Raises 401 if missing/invalid/expired."""
auth = request.headers.get("authorization", "")
if not auth.lower().startswith("bearer "):
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail={"code": "unauthorized", "message": "Authentication required."},
)
raw_token = auth.split(" ", 1)[1].strip()
if not raw_token:
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail={"code": "unauthorized", "message": "Authentication required."},
)
token_hash = _sha256_hex(raw_token)
now = utcnow_iso()
with get_conn() as conn:
row = conn.execute(
"SELECT s.id, s.client_id, s.expires_at, s.revoked_at, c.client_number "
"FROM sessions s JOIN clients c ON c.id = s.client_id "
"WHERE s.session_hash = ?",
(token_hash,),
).fetchone()
if row is None:
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail={"code": "unauthorized", "message": "Invalid or expired session."},
)
if row["revoked_at"] is not None:
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail={"code": "unauthorized", "message": "Session revoked."},
)
if row["expires_at"] < now:
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail={"code": "unauthorized", "message": "Session expired."},
)
# Touch last_seen
conn.execute(
"UPDATE sessions SET last_seen_at = ? WHERE id = ?",
(now, row["id"]),
)
conn.commit()
return {"client_id": row["client_id"], "client_number": row["client_number"]}
def create_magic_token(conn, client_id: str, ip: str) -> str:
"""Create a magic-link token row. Returns the RAW token (caller emails it, never stores it)."""
raw = generate_magic_token()
token_hash = _sha256_hex(raw)
now = utcnow_iso()
expires_at = (datetime.now(timezone.utc) + timedelta(minutes=MAGIC_TOKEN_TTL_MIN)).strftime("%Y-%m-%dT%H:%M:%SZ")
conn.execute(
"INSERT INTO auth_tokens (id, client_id, token_hash, expires_at, consumed_at, requested_ip, created_at) "
"VALUES (?, ?, ?, ?, NULL, ?, ?)",
(new_uuid(), client_id, token_hash, expires_at, ip, now),
)
return raw
def prune_expired_tokens(conn) -> None:
"""Delete auth_token rows older than 1 day (lazy sweep on verify)."""
cutoff = (datetime.now(timezone.utc) - timedelta(days=1)).strftime("%Y-%m-%dT%H:%M:%SZ")
conn.execute("DELETE FROM auth_tokens WHERE expires_at < ?", (cutoff,))
def verify_magic_token(conn, raw_token: str) -> str | None:
"""Hash token, find unexpired+unconsumed row. If found: mark consumed, return client_id.
Returns None if no match."""
token_hash = _sha256_hex(raw_token)
now = utcnow_iso()
prune_expired_tokens(conn)
row = conn.execute(
"SELECT id, client_id FROM auth_tokens WHERE token_hash = ? AND consumed_at IS NULL AND expires_at > ?",
(token_hash, now),
).fetchone()
if row is None:
return None
conn.execute("UPDATE auth_tokens SET consumed_at = ? WHERE id = ?", (now, row["id"]))
return row["client_id"]
def create_session(conn, client_id: str) -> tuple[str, str]:
"""Create a session. Returns (raw_session_token, expires_at_iso)."""
raw = generate_session_token()
session_hash = _sha256_hex(raw)
now = utcnow_iso()
expires_at = (datetime.now(timezone.utc) + timedelta(days=SESSION_TTL_DAYS)).strftime("%Y-%m-%dT%H:%M:%SZ")
conn.execute(
"INSERT INTO sessions (id, client_id, session_hash, expires_at, revoked_at, created_at, last_seen_at) "
"VALUES (?, ?, ?, ?, NULL, ?, NULL)",
(new_uuid(), client_id, session_hash, expires_at, now),
)
return raw, expires_at
+331
View File
@@ -0,0 +1,331 @@
"""Client-facing claim endpoints: list, detail, document upload/download, messages.
All queries scoped by session client_id. 404 (not 403) for other clients' claims.
"""
from __future__ import annotations
import hashlib
import html
import logging
import os
import uuid
from fastapi import APIRouter, Depends, File, HTTPException, Request, UploadFile, status
from fastapi.responses import FileResponse, JSONResponse
from . import auth as authmod
from . import db
from . import dreemail
from .db import get_conn, get_upload_dir, new_uuid, utcnow_iso
from .models import MessageCreate
logger = logging.getLogger("dre.claims")
router = APIRouter()
# Document constraints
MAX_FILE_BYTES = 20 * 1024 * 1024 # 20 MB
MAX_DOCS_PER_CLAIM = 50
ALLOWED_EXT = {".pdf", ".jpg", ".jpeg", ".png", ".doc", ".docx"}
MAGIC_BYTES = {
"pdf": (b"%PDF",),
"jpg": (b"\xff\xd8\xff",),
"png": (b"\x89PNG\r\n\x1a\n", b"\x89PNG"),
"docx": (b"PK\x03\x04",),
"doc": (b"PK\x03\x04", b"\xd0\xcf\x11\xe0"), # docx is ZIP; legacy .doc is OLE
}
EXT_TO_KIND = {
".pdf": "pdf", ".jpg": "jpg", ".jpeg": "jpg", ".png": "png",
".doc": "doc", ".docx": "docx",
}
STATUS_LABELS = {
"NEW": "Received", "UNDER_REVIEW": "Under Review", "ACTIVE": "In Progress",
"NEGOTIATION": "In Negotiation", "LEGAL": "Legal Action", "SETTLED": "Settled",
"CLOSED": "Closed — Recovered", "WRITE_OFF": "Closed — Uncollectible",
"REJECTED": "Not Accepted",
}
TIER_STEPS = {"TIER_1": 1, "TIER_2": 2, "TIER_2_5": 2, "TIER_3": 3, "TIER_4": 4}
def _money(cents: int) -> str:
return f"${cents / 100.0:,.2f}"
def _err(code: str, message: str, status_code: int):
return JSONResponse(status_code=status_code,
content={"error": {"code": code, "message": message}})
# ---------------------------------------------------------------
# GET /api/claims
# ---------------------------------------------------------------
@router.get("/api/claims")
async def list_claims(session: dict = Depends(authmod.require_client)):
client_id = session["client_id"]
with get_conn() as conn:
rows = conn.execute(
"SELECT c.claim_number, c.status, c.tier, c.amount_cents, c.created_at, c.date_resolved, "
"d.name AS debtor_name FROM claims c JOIN debtors d ON d.id = c.debtor_id "
"WHERE c.client_id = ? ORDER BY c.created_at DESC",
(client_id,),
).fetchall()
claims = [
{
"claim_number": r["claim_number"],
"status": r["status"],
"status_label": STATUS_LABELS.get(r["status"], r["status"]),
"tier": r["tier"],
"amount_cents": r["amount_cents"],
"amount_display": _money(r["amount_cents"]),
"debtor_name": r["debtor_name"],
"created_at": r["created_at"],
"date_resolved": r["date_resolved"],
}
for r in rows
]
return {"claims": claims}
# ---------------------------------------------------------------
# GET /api/claims/{claim_number}
# ---------------------------------------------------------------
@router.get("/api/claims/{claim_number}")
async def get_claim(claim_number: str, session: dict = Depends(authmod.require_client)):
client_id = session["client_id"]
with get_conn() as conn:
row = conn.execute(
"SELECT c.*, d.name AS debtor_name, d.business_type FROM claims c "
"JOIN debtors d ON d.id = c.debtor_id "
"WHERE c.claim_number = ? AND c.client_id = ?",
(claim_number, client_id),
).fetchone()
if row is None:
return _err("not_found", "Claim not found.", status.HTTP_404_NOT_FOUND)
docs = conn.execute(
"SELECT id, original_name, size_bytes, mime_type, uploaded_by, created_at FROM documents WHERE claim_id = ?",
(row["id"],),
).fetchall()
notes = conn.execute(
"SELECT author_type, author_name, subject, content, created_at FROM case_notes "
"WHERE claim_id = ? AND visibility = 'SHARED' ORDER BY created_at ASC",
(row["id"],),
).fetchall()
return {
"claim_number": row["claim_number"],
"status": row["status"],
"status_label": STATUS_LABELS.get(row["status"], row["status"]),
"tier": row["tier"],
"tier_step": TIER_STEPS.get(row["tier"], 1),
"amount_cents": row["amount_cents"],
"amount_display": _money(row["amount_cents"]),
"description": row["description"],
"client_reference": row["client_reference"],
"invoice_date": row["invoice_date"],
"date_assigned": row["date_assigned"],
"date_resolved": row["date_resolved"],
"debtor": {"name": row["debtor_name"], "business_type": row["business_type"]},
"documents": [
{
"id": d["id"],
"original_name": d["original_name"],
"size_bytes": d["size_bytes"],
"mime_type": d["mime_type"],
"uploaded_by": d["uploaded_by"],
"created_at": d["created_at"],
}
for d in docs
],
"notes": [
{
"author_type": n["author_type"],
"author_name": n["author_name"],
"subject": n["subject"],
# XSS: escape content server-side; frontend renders via textContent
"content": html.escape(n["content"]),
"created_at": n["created_at"],
}
for n in notes
],
}
# ---------------------------------------------------------------
# POST /api/claims/{claim_number}/documents (upload)
# ---------------------------------------------------------------
@router.post("/api/claims/{claim_number}/documents")
async def upload_document(claim_number: str, request: Request, file: UploadFile = File(...),
session: dict = Depends(authmod.require_client)):
client_id = session["client_id"]
# Check content length early
cl = request.headers.get("content-length")
if cl and int(cl) > MAX_FILE_BYTES + 4096:
return _err("payload_too_large", "File exceeds 20 MB limit.", status.HTTP_413_REQUEST_ENTITY_TOO_LARGE)
ext = os.path.splitext(file.filename or "")[1].lower()
if ext not in ALLOWED_EXT:
return _err("unsupported_media_type", "File type not allowed.", status.HTTP_415_UNSUPPORTED_MEDIA_TYPE)
kind = EXT_TO_KIND.get(ext)
if not kind:
return _err("unsupported_media_type", "File type not allowed.", status.HTTP_415_UNSUPPORTED_MEDIA_TYPE)
with get_conn() as conn:
row = conn.execute(
"SELECT id FROM claims WHERE claim_number = ? AND client_id = ?",
(claim_number, client_id),
).fetchone()
if row is None:
return _err("not_found", "Claim not found.", status.HTTP_404_NOT_FOUND)
claim_id = row["id"]
count = conn.execute("SELECT COUNT(*) AS n FROM documents WHERE claim_id = ?", (claim_id,)).fetchone()["n"]
if count >= MAX_DOCS_PER_CLAIM:
return _err("conflict", "Document limit reached for this claim.", status.HTTP_409_CONFLICT)
upload_dir = os.path.join(get_upload_dir(), claim_id)
os.makedirs(upload_dir, exist_ok=True)
stored_uuid = str(uuid.uuid4())
stored_name = f"{stored_uuid}{ext}"
stored_path = os.path.join(upload_dir, stored_name)
sha = hashlib.sha256()
total = 0
magic_seen = False
with open(stored_path, "wb") as f:
while True:
chunk = await file.read(64 * 1024)
if not chunk:
break
total += len(chunk)
if total > MAX_FILE_BYTES:
f.close()
os.remove(stored_path)
return _err("payload_too_large", "File exceeds 20 MB limit.",
status.HTTP_413_REQUEST_ENTITY_TOO_LARGE)
if not magic_seen:
sigs = MAGIC_BYTES.get(kind, ())
if sigs and any(chunk.startswith(s) for s in sigs):
magic_seen = True
elif sigs:
f.close()
os.remove(stored_path)
return _err("unsupported_media_type", "File content does not match extension.",
status.HTTP_415_UNSUPPORTED_MEDIA_TYPE)
sha.update(chunk)
f.write(chunk)
os.chmod(stored_path, 0o640)
now = utcnow_iso()
doc_id = new_uuid()
mime = file.content_type or "application/octet-stream"
conn.execute(
"INSERT INTO documents (id, claim_id, original_name, stored_path, mime_type, size_bytes, sha256, uploaded_by, twentycrm_id, created_at) "
"VALUES (?, ?, ?, ?, ?, ?, ?, 'CLIENT', NULL, ?)",
(doc_id, claim_id, os.path.basename(file.filename or "file"), stored_path, mime, total, sha.hexdigest(), now),
)
# SYSTEM note + audit
conn.execute(
"INSERT INTO case_notes (id, claim_id, author_type, author_name, subject, content, visibility, twentycrm_id, created_at) "
"VALUES (?, ?, 'SYSTEM', 'System', NULL, ?, 'SHARED', NULL, ?)",
(new_uuid(), claim_id, f"Document uploaded: {os.path.basename(file.filename or 'file')}", now),
)
conn.execute(
"INSERT INTO audit_log (id, entity_type, entity_id, action, actor, created_at) VALUES (?, ?, ?, ?, ?, ?)",
(new_uuid(), "document", doc_id, "upload", session["client_number"], now),
)
conn.commit()
return {
"id": doc_id,
"original_name": os.path.basename(file.filename or "file"),
"size_bytes": total,
"mime_type": mime,
"uploaded_by": "CLIENT",
"created_at": now,
}
# ---------------------------------------------------------------
# GET /api/claims/{claim_number}/documents/{document_id} (download)
# ---------------------------------------------------------------
@router.get("/api/claims/{claim_number}/documents/{document_id}")
async def download_document(claim_number: str, document_id: str,
request: Request, session: dict | None = None):
# Auth: client OR staff
is_staff = authmod.verify_staff_key(request.headers.get("x-dre-staff-key"))
client_session = None
if not is_staff:
try:
client_session = authmod.require_client(request)
except HTTPException:
return _err("unauthorized", "Authentication required.", status.HTTP_401_UNAUTHORIZED)
with get_conn() as conn:
row = conn.execute(
"SELECT d.*, c.claim_number FROM documents d JOIN claims c ON c.id = d.claim_id "
"WHERE d.id = ? AND c.claim_number = ?",
(document_id, claim_number),
).fetchone()
if row is None:
return _err("not_found", "Document not found.", status.HTTP_404_NOT_FOUND)
if not is_staff and row["claim_id"]:
# verify ownership
owner = conn.execute("SELECT client_id FROM claims WHERE id = ?", (row["claim_id"],)).fetchone()
if owner is None or owner["client_id"] != client_session["client_id"]:
return _err("not_found", "Document not found.", status.HTTP_404_NOT_FOUND)
if not os.path.exists(row["stored_path"]):
return _err("not_found", "File missing on disk.", status.HTTP_404_NOT_FOUND)
return FileResponse(
row["stored_path"],
media_type=row["mime_type"] or "application/octet-stream",
filename=row["original_name"],
)
# ---------------------------------------------------------------
# POST /api/claims/{claim_number}/messages
# ---------------------------------------------------------------
@router.post("/api/claims/{claim_number}/messages")
async def create_message(claim_number: str, request: Request,
session: dict = Depends(authmod.require_client)):
client_id = session["client_id"]
try:
body = await request.json()
except Exception:
return _err("validation_error", "Invalid JSON body.", status.HTTP_422_UNPROCESSABLE_ENTITY)
try:
msg = MessageCreate.model_validate(body)
except Exception as exc:
from pydantic import ValidationError
if isinstance(exc, ValidationError):
parts = [f"{'.'.join(str(x) for x in e['loc'])}: {e['msg']}" for e in exc.errors()]
return _err("validation_error", "; ".join(parts), status.HTTP_422_UNPROCESSABLE_ENTITY)
return _err("validation_error", str(exc), status.HTTP_422_UNPROCESSABLE_ENTITY)
with get_conn() as conn:
row = conn.execute(
"SELECT id FROM claims WHERE claim_number = ? AND client_id = ?",
(claim_number, client_id),
).fetchone()
if row is None:
return _err("not_found", "Claim not found.", status.HTTP_404_NOT_FOUND)
claim_id = row["id"]
# Get client contact name for author
c = conn.execute("SELECT contact_name, company_name FROM clients WHERE id = ?", (client_id,)).fetchone()
if c is None:
return _err("not_found", "Client not found.", status.HTTP_404_NOT_FOUND)
now = utcnow_iso()
note_id = new_uuid()
conn.execute(
"INSERT INTO case_notes (id, claim_id, author_type, author_name, subject, content, visibility, twentycrm_id, created_at) "
"VALUES (?, ?, 'CLIENT', ?, ?, ?, 'SHARED', NULL, ?)",
(note_id, claim_id, c["contact_name"], msg.subject, msg.content, now),
)
conn.execute(
"INSERT INTO audit_log (id, entity_type, entity_id, action, actor, created_at) VALUES (?, ?, ?, ?, ?, ?)",
(new_uuid(), "note", note_id, "note_add", session["client_number"], now),
)
conn.commit()
# Best-effort team notification
try:
dreemail.notify_team_message(claim_number, msg.subject, msg.content, c["contact_name"])
except Exception as exc: # noqa: BLE001
logger.error("message team notify failed: %s", exc)
return {
"id": note_id,
"author_type": "CLIENT",
"author_name": c["contact_name"],
"subject": msg.subject,
"content": html.escape(msg.content),
"visibility": "SHARED",
"created_at": now,
}
+99
View File
@@ -0,0 +1,99 @@
"""SQLite connection helper — stdlib sqlite3, parameterized queries only.
PRAGMA foreign_keys=ON, journal_mode=WAL, busy_timeout=5000 on every connection.
No ORM. Initializes schema on first boot and seeds number_sequences for current year.
"""
from __future__ import annotations
import os
import sqlite3
import uuid
from datetime import datetime, timezone
from pathlib import Path
from typing import Any
_SCHEMA_FILE = Path(__file__).resolve().parent / "schema.sql"
def _utcnow_iso() -> str:
return datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
def get_db_path() -> str:
return os.environ.get("DRE_DB_PATH", "/opt/dre-portal/data/dre.db")
def get_upload_dir() -> str:
return os.environ.get("DRE_UPLOAD_DIR", "/opt/dre-portal/data/uploads")
def _apply_pragmas(conn: sqlite3.Connection) -> None:
conn.execute("PRAGMA foreign_keys = ON;")
conn.execute("PRAGMA journal_mode = WAL;")
conn.execute("PRAGMA busy_timeout = 5000;")
def get_conn() -> sqlite3.Connection:
"""Return a connection with pragmas applied and row factory."""
db_path = get_db_path()
Path(db_path).parent.mkdir(parents=True, exist_ok=True)
conn = sqlite3.connect(db_path, timeout=5.0)
conn.row_factory = sqlite3.Row
_apply_pragmas(conn)
return conn
def init_db() -> None:
"""Create schema if absent and seed number_sequences for the current year."""
with get_conn() as conn:
schema_sql = _SCHEMA_FILE.read_text()
conn.executescript(schema_sql)
# Seed sequences for current year (idempotent)
year = datetime.now(timezone.utc).year
for prefix in ("DRE", "CLT"):
conn.execute(
"INSERT OR IGNORE INTO number_sequences (prefix, year, last_value) VALUES (?, ?, 0)",
(prefix, year),
)
conn.commit()
def new_uuid() -> str:
return str(uuid.uuid4())
def utcnow_iso() -> str:
return _utcnow_iso()
def next_sequence_number(conn: sqlite3.Connection, prefix: str) -> str:
"""Atomically allocate the next DRE-YYYY-NNNN / CLT-YYYY-NNNN in a single transaction.
Uses UPDATE ... RETURNING (SQLite 3.35+) — no races."""
year = datetime.now(timezone.utc).year
# Ensure row exists
conn.execute(
"INSERT OR IGNORE INTO number_sequences (prefix, year, last_value) VALUES (?, ?, 0)",
(prefix, year),
)
cur = conn.execute(
"UPDATE number_sequences SET last_value = last_value + 1 WHERE prefix = ? AND year = ? "
"RETURNING last_value",
(prefix, year),
)
row = cur.fetchone()
if row is None:
# Should not happen, but handle defensively
raise RuntimeError(f"failed to allocate sequence for {prefix}/{year}")
n = row[0] if isinstance(row, tuple) else row["last_value"]
val = n if isinstance(n, int) else int(n)
return f"{prefix}-{year}-{val:04d}"
def query_one(conn: sqlite3.Connection, sql: str, params: tuple[Any, ...] = ()) -> sqlite3.Row | None:
cur = conn.execute(sql, params)
return cur.fetchone()
def query_all(conn: sqlite3.Connection, sql: str, params: tuple[Any, ...] = ()) -> list[sqlite3.Row]:
cur = conn.execute(sql, params)
return cur.fetchall()
+22
View File
@@ -0,0 +1,22 @@
[Unit]
Description=DRE Customer Portal API (FastAPI/uvicorn)
After=network.target
[Service]
Type=simple
WorkingDirectory=/opt/dre-portal
EnvironmentFile=/opt/dre-portal/.env
ExecStart=/opt/dre-portal/venv/bin/uvicorn app.main:app --host 127.0.0.1 --port 8093
Restart=on-failure
RestartSec=3
User=root
# hardening
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=strict
ReadWritePaths=/opt/dre-portal/data
# NOTE: ProtectHome=true would block reading /root; omitted so service can read env.
# (ops-portal.service runs as root without ProtectHome for the same reason.)
[Install]
WantedBy=multi-user.target
+102
View File
@@ -0,0 +1,102 @@
"""Best-effort SMTP email via the germainebrown.com relay (mail.germainebrown.com:2525, STARTTLS).
Email failure must NEVER fail the API request — wrap every send in try/except, log, continue.
Per conductor decision #3.
"""
from __future__ import annotations
import logging
import os
import smtplib
import ssl
from email.mime.text import MIMEText
from email.mime.multipart import MIMEMultipart
logger = logging.getLogger("dre.email")
def _env(name: str, default: str = "") -> str:
return os.environ.get(name, default)
def send_email(to_addr: str, subject: str, body_text: str, html: str | None = None) -> bool:
"""Send an email via the configured SMTP relay. Returns True on success, False on failure.
Never raises — caller proceeds regardless."""
host = _env("DRE_SMTP_HOST", "mail.germainebrown.com")
port = int(_env("DRE_SMTP_PORT", "2525"))
user = _env("DRE_SMTP_USER", "")
pw = _env("DRE_SMTP_PASS", "")
sender = _env("DRE_SMTP_FROM", "dre@debtrecoveryexperts.com")
try:
msg = MIMEMultipart("alternative")
msg["From"] = sender
msg["To"] = to_addr
msg["Subject"] = subject
msg.attach(MIMEText(body_text, "plain", "utf-8"))
if html:
msg.attach(MIMEText(html, "html", "utf-8"))
with smtplib.SMTP(host, port, timeout=15) as server:
server.starttls(context=ssl.create_default_context())
if user and pw:
server.login(user, pw)
server.sendmail(sender, [to_addr], msg.as_string())
return True
except Exception as exc: # noqa: BLE001 — best-effort
logger.error("email send failed to=%s subject=%s err=%s", to_addr, subject, exc)
return False
def notify_team_intake(claim_number: str, client_number: str, company_name: str,
amount_cents: int, debtor_name: str) -> bool:
team = _env("DRE_TEAM_NOTIFY", "dre@debtrecoveryexperts.com")
base = _env("DRE_BASE_URL", "https://portal.debtrecoveryexperts.com")
dollars = amount_cents / 100.0
body = (
f"New claim submitted via portal.\n\n"
f"Claim: {claim_number}\n"
f"Client: {client_number}{company_name}\n"
f"Debtor: {debtor_name}\n"
f"Amount: ${dollars:,.2f}\n\n"
f"Review at: {base}/\n"
)
html = (
f"<h2>New claim submitted</h2>"
f"<p><b>Claim:</b> {claim_number}<br>"
f"<b>Client:</b> {client_number}{company_name}<br>"
f"<b>Debtor:</b> {debtor_name}<br>"
f"<b>Amount:</b> ${dollars:,.2f}</p>"
f"<p><a href=\"{base}/\">Review in portal</a></p>"
)
return send_email(team, f"New DRE Claim: {claim_number}", body, html)
def send_magic_link(to_addr: str, raw_token: str, client_number: str) -> bool:
base = _env("DRE_BASE_URL", "https://portal.debtrecoveryexperts.com")
link = f"{base}/portal/verify?token={raw_token}"
body = (
f"Hello,\n\n"
f"Click the link below to log in to your DRE client portal. "
f"This link expires in 15 minutes and can only be used once.\n\n"
f"{link}\n\n"
f"If you did not request this link, you can ignore this email.\n"
)
html = (
f"<p>Hello,</p>"
f"<p>Click the button below to log in to your DRE client portal. "
f"This link expires in 15 minutes and can only be used once.</p>"
f"<p><a href=\"{link}\" style=\"...\">Log In</a></p>"
f"<p>If the button doesn't work, copy this link: {link}</p>"
f"<p>If you did not request this link, you can ignore this email.</p>"
)
return send_email(to_addr, "Your DRE Portal Login Link", body, html)
def notify_team_message(claim_number: str, subject: str, content: str,
author: str) -> bool:
team = _env("DRE_TEAM_NOTIFY", "dre@debtrecoveryexperts.com")
body = (
f"New client message on claim {claim_number}.\n\n"
f"From: {author}\n"
f"Subject: {subject}\n\n"
f"{content}\n"
)
return send_email(team, f"Client message on {claim_number}: {subject}", body)
+147
View File
@@ -0,0 +1,147 @@
"""Intake endpoint — POST /api/intake. Creates client (+reuse by email) + debtor + claim.
Writes audit rows, SYSTEM note, emails the DRE team (best-effort).
"""
from __future__ import annotations
import logging
from fastapi import APIRouter, Request, status
from pydantic import ValidationError
from . import auth as authmod
from . import db
from . import dreemail
from .db import get_conn, new_uuid, next_sequence_number, utcnow_iso
from .models import IntakeRequest
logger = logging.getLogger("dre.intake")
router = APIRouter()
def _money_display(cents: int) -> str:
return f"${cents / 100.0:,.2f}"
@router.post("/api/intake")
async def intake(request: Request):
# Rate limit
ip = authmod.get_client_ip(request)
if not authmod.rate_limit_intake(ip):
return _json_error("rate_limited", "Too many requests. Please try again later.",
status.HTTP_429_TOO_MANY_REQUESTS)
# Parse JSON body
try:
body = await request.json()
except Exception:
return _json_error("validation_error", "Invalid JSON body.", status.HTTP_422_UNPROCESSABLE_ENTITY)
# Validate
try:
req = IntakeRequest.model_validate(body)
except ValidationError as exc:
return _json_error("validation_error", _format_validation_error(exc),
status.HTTP_422_UNPROCESSABLE_ENTITY)
client_in = req.client
debtor_in = req.debtor
claim_in = req.claim
email_lc = client_in.email.lower()
with get_conn() as conn:
now = utcnow_iso()
# Reuse client by email or create new
existing = conn.execute("SELECT id, client_number FROM clients WHERE email = ?", (email_lc,)).fetchone()
if existing is not None:
client_id = existing["id"]
client_number = existing["client_number"]
conn.execute(
"UPDATE clients SET company_name = ?, contact_name = ?, phone = ?, tos_accepted_at = COALESCE(tos_accepted_at, ?), updated_at = ? WHERE id = ?",
(client_in.company_name, client_in.contact_name, client_in.phone, now, now, client_id),
)
# audit update
conn.execute(
"INSERT INTO audit_log (id, entity_type, entity_id, action, field, old_value, new_value, actor, reason, created_at) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)",
(new_uuid(), "client", client_id, "update", "contact_info", None, None, "intake", "reused existing client", now),
)
else:
client_id = new_uuid()
client_number = next_sequence_number(conn, "CLT")
conn.execute(
"INSERT INTO clients (id, client_number, company_name, contact_name, email, phone, tos_accepted_at, twentycrm_id, created_at, updated_at) "
"VALUES (?, ?, ?, ?, ?, ?, ?, NULL, ?, ?)",
(client_id, client_number, client_in.company_name, client_in.contact_name,
email_lc, client_in.phone, now, now, now),
)
conn.execute(
"INSERT INTO audit_log (id, entity_type, entity_id, action, actor, created_at) VALUES (?, ?, ?, ?, ?, ?)",
(new_uuid(), "client", client_id, "create", "intake", now),
)
# Debtor (always new per spec — denormalized avoided, but new claim = new debtor row)
debtor_id = new_uuid()
conn.execute(
"INSERT INTO debtors (id, name, business_type, contact_email, contact_phone, physical_address, twentycrm_id, created_at, updated_at) "
"VALUES (?, ?, ?, ?, ?, ?, NULL, ?, ?)",
(debtor_id, debtor_in.name, debtor_in.business_type, debtor_in.contact_email,
debtor_in.contact_phone, debtor_in.physical_address, now, now),
)
conn.execute(
"INSERT INTO audit_log (id, entity_type, entity_id, action, actor, created_at) VALUES (?, ?, ?, ?, ?, ?)",
(new_uuid(), "debtor", debtor_id, "create", "intake", now),
)
# Claim
claim_id = new_uuid()
claim_number = next_sequence_number(conn, "DRE")
conn.execute(
"INSERT INTO claims (id, claim_number, client_id, debtor_id, amount_cents, currency, status, tier, description, client_reference, invoice_date, date_assigned, date_resolved, twentycrm_id, created_at, updated_at) "
"VALUES (?, ?, ?, ?, ?, 'USD', 'NEW', 'TIER_1', ?, ?, ?, NULL, NULL, NULL, ?, ?)",
(claim_id, claim_number, client_id, debtor_id, claim_in.amount_cents,
claim_in.description, claim_in.client_reference, claim_in.invoice_date, now, now),
)
conn.execute(
"INSERT INTO audit_log (id, entity_type, entity_id, action, old_value, new_value, actor, created_at) VALUES (?, ?, ?, ?, ?, ?, ?, ?)",
(new_uuid(), "claim", claim_id, "create", None, claim_number, "intake", now),
)
# SYSTEM shared note
note_id = new_uuid()
conn.execute(
"INSERT INTO case_notes (id, claim_id, author_type, author_name, subject, content, visibility, twentycrm_id, created_at) "
"VALUES (?, ?, 'SYSTEM', 'System', NULL, 'Claim received.', 'SHARED', NULL, ?)",
(note_id, claim_id, now),
)
conn.execute(
"INSERT INTO audit_log (id, entity_type, entity_id, action, actor, created_at) VALUES (?, ?, ?, ?, ?, ?)",
(new_uuid(), "note", note_id, "note_add", "system", now),
)
conn.commit()
# Best-effort team notification
try:
dreemail.notify_team_intake(claim_number, client_number, client_in.company_name,
claim_in.amount_cents, debtor_in.name)
except Exception as exc: # noqa: BLE001
logger.error("intake team notify failed: %s", exc)
return {
"claim_number": claim_number,
"client_number": client_number,
"status": "NEW",
"message": "Claim received. Our team will review and contact you shortly.",
}
def _format_validation_error(exc: ValidationError) -> str:
parts = []
for err in exc.errors():
loc = ".".join(str(x) for x in err["loc"])
parts.append(f"{loc}: {err['msg']}")
return "; ".join(parts)
def _json_error(code: str, message: str, status_code: int):
from fastapi.responses import JSONResponse
return JSONResponse(
status_code=status_code,
content={"error": {"code": code, "message": message}},
)
+195
View File
@@ -0,0 +1,195 @@
"""DRE Customer Portal API — FastAPI app + all 18 routers.
Port 127.0.0.1:8093. systemd: dre-portal.service (User=root).
"""
from __future__ import annotations
import logging
from fastapi import Depends, FastAPI, HTTPException, Request, status
from fastapi.exceptions import RequestValidationError
from fastapi.middleware.cors import CORSMiddleware
from fastapi.responses import JSONResponse
from pydantic import ValidationError
from . import auth as authmod
from . import db
from . import dreemail
from .db import get_conn, utcnow_iso
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(name)s %(levelname)s %(message)s")
logger = logging.getLogger("dre.main")
app = FastAPI(title="DRE Customer Portal API", version="1.0.0", docs_url="/docs", redoc_url=None)
app.add_middleware(
CORSMiddleware,
allow_origins=["https://portal.debtrecoveryexperts.com", "http://127.0.0.1:8093"],
allow_methods=["GET", "POST", "PATCH"],
allow_headers=["*"],
)
# Include routers (intake, claims, staff)
from .intake import router as intake_router # noqa: E402
from .claims import router as claims_router # noqa: E402
from .staff import router as staff_router # noqa: E402
app.include_router(intake_router)
app.include_router(claims_router)
app.include_router(staff_router)
def _err(code: str, message: str, status_code: int):
return JSONResponse(status_code=status_code,
content={"error": {"code": code, "message": message}})
@app.on_event("startup")
async def _startup():
db.init_db()
logger.info("DRE portal started; db=%s", db.get_db_path())
# ---------------------------------------------------------------
# Error envelopes
# ---------------------------------------------------------------
@app.exception_handler(RequestValidationError)
async def _validation_handler(request: Request, exc: RequestValidationError):
errors = exc.errors()
parts = []
for e in errors:
loc = ".".join(str(x) for x in e.get("loc", []))
parts.append(f"{loc}: {e.get('msg', 'invalid')}")
msg = "; ".join(parts) if parts else "Validation error"
return _err("validation_error", msg, status.HTTP_422_UNPROCESSABLE_ENTITY)
@app.exception_handler(ValidationError)
async def _pyd_validation_handler(request: Request, exc: ValidationError):
parts = [f"{'.'.join(str(x) for x in e['loc'])}: {e['msg']}" for e in exc.errors()]
return _err("validation_error", "; ".join(parts), status.HTTP_422_UNPROCESSABLE_ENTITY)
@app.exception_handler(HTTPException)
async def _http_handler(request: Request, exc: HTTPException):
"""Map auth/forbidden HTTPExceptions onto the unified {'error':{...}} envelope."""
detail = exc.detail
if isinstance(detail, dict) and "code" in detail and "message" in detail:
return _err(detail["code"], detail["message"], exc.status_code)
return _err("internal_error", "An internal error occurred.", exc.status_code)
@app.exception_handler(Exception)
async def _internal_handler(request: Request, exc: Exception):
logger.exception("internal error: %s", exc)
return _err("internal_error", "An internal error occurred.", status.HTTP_500_INTERNAL_SERVER_ERROR)
# ---------------------------------------------------------------
# 1. GET /api/health
# ---------------------------------------------------------------
@app.get("/api/health")
async def health():
return {"status": "ok", "time": utcnow_iso()}
# ---------------------------------------------------------------
# 2-5. Auth endpoints (magic-link)
# ---------------------------------------------------------------
@app.post("/api/auth/request")
async def auth_request(request: Request):
try:
body = await request.json()
except Exception:
return _err("validation_error", "Invalid JSON body.", status.HTTP_422_UNPROCESSABLE_ENTITY)
email_in = body.get("email") if isinstance(body, dict) else None
if not email_in:
return _err("validation_error", "email is required.", status.HTTP_422_UNPROCESSABLE_ENTITY)
ip = authmod.get_client_ip(request)
if not authmod.rate_limit_auth_request(email_in.lower(), ip):
return _err("rate_limited", "Too many requests. Please try again later.",
status.HTTP_429_TOO_MANY_REQUESTS)
email_lc = email_in.lower()
with get_conn() as conn:
row = conn.execute("SELECT id, contact_name, company_name FROM clients WHERE email = ?", (email_lc,)).fetchone()
if row is not None:
raw_token = authmod.create_magic_token(conn, row["id"], ip)
conn.commit()
# Best-effort email
try:
dreemail.send_magic_link(email_lc, raw_token, "")
except Exception as exc: # noqa: BLE001
logger.error("magic link email failed: %s", exc)
# Anti-enumeration: always same response
return {"message": "If an account exists, a login link has been sent."}
@app.post("/api/auth/verify")
async def auth_verify(request: Request):
try:
body = await request.json()
except Exception:
return _err("validation_error", "Invalid JSON body.", status.HTTP_422_UNPROCESSABLE_ENTITY)
token = body.get("token") if isinstance(body, dict) else None
if not token:
return _err("validation_error", "token is required.", status.HTTP_422_UNPROCESSABLE_ENTITY)
ip = authmod.get_client_ip(request)
if not authmod.rate_limit_auth_verify(ip):
return _err("rate_limited", "Too many attempts. Please try again later.",
status.HTTP_429_TOO_MANY_REQUESTS)
with get_conn() as conn:
client_id = authmod.verify_magic_token(conn, token)
if client_id is None:
return _err("unauthorized", "Invalid or expired token.", status.HTTP_401_UNAUTHORIZED)
session_token, expires_at = authmod.create_session(conn, client_id)
client = conn.execute(
"SELECT client_number, company_name, contact_name FROM clients WHERE id = ?", (client_id,)
).fetchone()
conn.commit()
return {
"session_token": session_token,
"expires_at": expires_at,
"client": {
"client_number": client["client_number"],
"company_name": client["company_name"],
"contact_name": client["contact_name"],
},
}
@app.post("/api/auth/logout")
async def auth_logout(request: Request, session: dict = Depends(authmod.require_client)):
"""Revoke the current session by hash."""
auth = request.headers.get("authorization", "")
raw_token = auth.split(" ", 1)[1].strip() if auth.lower().startswith("bearer ") else ""
import hashlib
session_hash = hashlib.sha256(raw_token.encode("utf-8")).hexdigest()
with get_conn() as conn:
conn.execute(
"UPDATE sessions SET revoked_at = ? WHERE session_hash = ?",
(utcnow_iso(), session_hash),
)
conn.commit()
return {"message": "Logged out."}
@app.get("/api/auth/me")
async def auth_me(session: dict = Depends(authmod.require_client)):
import hashlib
client_id = session["client_id"]
with get_conn() as conn:
c = conn.execute(
"SELECT client_number, company_name, contact_name, email, phone, created_at FROM clients WHERE id = ?",
(client_id,),
).fetchone()
claim_count = conn.execute("SELECT COUNT(*) AS n FROM claims WHERE client_id = ?", (client_id,)).fetchone()["n"]
return {
"client": {
"client_number": c["client_number"],
"company_name": c["company_name"],
"contact_name": c["contact_name"],
"email": c["email"],
"phone": c["phone"],
"member_since": c["created_at"],
},
"claim_count": claim_count,
}
+190
View File
@@ -0,0 +1,190 @@
"""Pydantic v2 request/response models. extra='forbid' on every request body.
Includes PII rejection (SSN/PAN regex) on all free-text fields.
"""
from __future__ import annotations
import re
from pydantic import BaseModel, ConfigDict, EmailStr, Field, field_validator
# ---------------------------------------------------------------
# PII rejection — compliance-critical
# ---------------------------------------------------------------
_SSN_RE = re.compile(r"\b\d{3}-?\d{2}-?\d{4}\b")
_PAN_RE = re.compile(r"\b(?:\d[ -]?){13,19}\b")
def _scan_pii(value: str) -> str:
"""Raise ValueError if value matches SSN or PAN regex."""
if value is None:
return value
if _SSN_RE.search(value):
raise ValueError("Do not include Social Security or bank/card numbers.")
if _PAN_RE.search(value):
raise ValueError("Do not include Social Security or bank/card numbers.")
return value
def _pii_validator(field_name: str):
return field_validator(field_name)(lambda v: _scan_pii(v))
# ---------------------------------------------------------------
# Intake request
# ---------------------------------------------------------------
BUSINESS_TYPES = (
"INDIVIDUAL", "SOLE_PROPRIETORSHIP", "LLC", "CORPORATION", "PARTNERSHIP", "OTHER"
)
MESSAGE_SUBJECTS = (
"Question about my claim",
"New information about the debtor",
"Payment received / want to stop recovery",
"Update my contact info",
"Complaint or concern",
"Other",
)
CLAIM_STATUSES = (
"NEW", "UNDER_REVIEW", "ACTIVE", "NEGOTIATION", "LEGAL",
"SETTLED", "CLOSED", "WRITE_OFF", "REJECTED",
)
TIERS = ("TIER_1", "TIER_2", "TIER_2_5", "TIER_3", "TIER_4")
class IntakeClient(BaseModel):
model_config = ConfigDict(extra="forbid")
company_name: str = Field(..., min_length=1, max_length=200)
contact_name: str = Field(..., min_length=1, max_length=200)
email: EmailStr
phone: str | None = Field(None, max_length=50)
@field_validator("company_name", "contact_name", "phone")
@classmethod
def _v(cls, v):
return _scan_pii(v)
class IntakeDebtor(BaseModel):
model_config = ConfigDict(extra="forbid")
name: str = Field(..., min_length=1, max_length=200)
business_type: str = Field("OTHER")
contact_email: str | None = Field(None, max_length=254)
contact_phone: str | None = Field(None, max_length=50)
physical_address: str | None = Field(None, max_length=500)
@field_validator("business_type")
@classmethod
def _bt(cls, v):
v = v.upper()
if v not in BUSINESS_TYPES:
raise ValueError(f"business_type must be one of {BUSINESS_TYPES}")
return v
@field_validator("name", "contact_email", "contact_phone", "physical_address")
@classmethod
def _v(cls, v):
return _scan_pii(v)
class IntakeClaim(BaseModel):
model_config = ConfigDict(extra="forbid")
amount_cents: int = Field(..., gt=0, le=100_000_000)
description: str | None = Field(None, max_length=5000)
client_reference: str | None = Field(None, max_length=200)
invoice_date: str | None = Field(None, max_length=20)
@field_validator("description", "client_reference", "invoice_date")
@classmethod
def _v(cls, v):
return _scan_pii(v)
class IntakeRequest(BaseModel):
model_config = ConfigDict(extra="forbid")
client: IntakeClient
debtor: IntakeDebtor
claim: IntakeClaim
tos_accepted: bool = True
turnstile_token: str | None = None
@field_validator("tos_accepted")
@classmethod
def _tos(cls, v):
if v is not True:
raise ValueError("tos_accepted must be true")
return v
# ---------------------------------------------------------------
# Auth
# ---------------------------------------------------------------
class AuthRequest(BaseModel):
model_config = ConfigDict(extra="forbid")
email: EmailStr
class AuthVerify(BaseModel):
model_config = ConfigDict(extra="forbid")
token: str = Field(..., min_length=10, max_length=200)
# ---------------------------------------------------------------
# Messages
# ---------------------------------------------------------------
class MessageCreate(BaseModel):
model_config = ConfigDict(extra="forbid")
subject: str
content: str = Field(..., min_length=1, max_length=10000)
@field_validator("subject")
@classmethod
def _subj(cls, v):
if v not in MESSAGE_SUBJECTS:
raise ValueError(f"subject must be one of {MESSAGE_SUBJECTS}")
return v
@field_validator("content")
@classmethod
def _cont(cls, v):
return _scan_pii(v)
# ---------------------------------------------------------------
# Staff
# ---------------------------------------------------------------
class StaffClaimPatch(BaseModel):
model_config = ConfigDict(extra="forbid")
status: str | None = None
tier: str | None = None
reason: str | None = Field(None, max_length=500)
@field_validator("status")
@classmethod
def _st(cls, v):
if v is not None and v not in CLAIM_STATUSES:
raise ValueError(f"status must be one of {CLAIM_STATUSES}")
return v
@field_validator("tier")
@classmethod
def _tr(cls, v):
if v is not None and v not in TIERS:
raise ValueError(f"tier must be one of {TIERS}")
return v
class StaffNoteCreate(BaseModel):
model_config = ConfigDict(extra="forbid")
content: str = Field(..., min_length=1, max_length=10000)
visibility: str = "INTERNAL"
author_name: str = Field(..., min_length=1, max_length=200)
@field_validator("visibility")
@classmethod
def _vis(cls, v):
if v not in ("SHARED", "INTERNAL"):
raise ValueError("visibility must be SHARED or INTERNAL")
return v
@field_validator("content", "author_name")
@classmethod
def _v(cls, v):
return _scan_pii(v)
+5
View File
@@ -0,0 +1,5 @@
fastapi>=0.110
uvicorn[standard]>=0.29
python-multipart>=0.0.9
email-validator>=2.0
pydantic>=2.6
+139
View File
@@ -0,0 +1,139 @@
-- ============================================================
-- DRE Customer Portal — SQLite Schema (spec §1)
-- DB file: /opt/dre-portal/data/dre.db
-- Pragmas (set on every connection): foreign_keys=ON, journal_mode=WAL, busy_timeout=5000
-- All timestamps ISO-8601 UTC TEXT. Money as INTEGER cents. PKs TEXT UUID4.
-- ============================================================
CREATE TABLE IF NOT EXISTS clients (
id TEXT PRIMARY KEY, -- uuid4
client_number TEXT UNIQUE NOT NULL, -- CLT-YYYY-NNNN
company_name TEXT NOT NULL,
contact_name TEXT NOT NULL,
email TEXT UNIQUE NOT NULL, -- lowercased; magic-link identity
phone TEXT,
tos_accepted_at TEXT, -- set when ToS accepted at intake
twentycrm_id TEXT, -- nullable; set by future CRM sync
created_at TEXT NOT NULL,
updated_at TEXT NOT NULL
);
CREATE INDEX IF NOT EXISTS idx_clients_email ON clients(email);
CREATE TABLE IF NOT EXISTS debtors (
id TEXT PRIMARY KEY, -- uuid4
name TEXT NOT NULL, -- business or individual name
business_type TEXT NOT NULL DEFAULT 'OTHER'
CHECK (business_type IN
('INDIVIDUAL','SOLE_PROPRIETORSHIP','LLC','CORPORATION','PARTNERSHIP','OTHER')),
contact_email TEXT,
contact_phone TEXT,
physical_address TEXT, -- free-text single line
twentycrm_id TEXT, -- nullable
created_at TEXT NOT NULL,
updated_at TEXT NOT NULL
);
CREATE TABLE IF NOT EXISTS claims (
id TEXT PRIMARY KEY, -- uuid4
claim_number TEXT UNIQUE NOT NULL, -- DRE-YYYY-NNNN
client_id TEXT NOT NULL REFERENCES clients(id) ON DELETE RESTRICT,
debtor_id TEXT NOT NULL REFERENCES debtors(id) ON DELETE RESTRICT,
amount_cents INTEGER NOT NULL CHECK (amount_cents > 0),
currency TEXT NOT NULL DEFAULT 'USD',
status TEXT NOT NULL DEFAULT 'NEW'
CHECK (status IN
('NEW','UNDER_REVIEW','ACTIVE','NEGOTIATION','LEGAL','SETTLED','CLOSED','WRITE_OFF','REJECTED')),
tier TEXT NOT NULL DEFAULT 'TIER_1'
CHECK (tier IN ('TIER_1','TIER_2','TIER_2_5','TIER_3','TIER_4')),
description TEXT,
client_reference TEXT,
invoice_date TEXT, -- ISO date
date_assigned TEXT, -- set when moved out of NEW
date_resolved TEXT, -- set on SETTLED/CLOSED/WRITE_OFF
twentycrm_id TEXT, -- nullable
created_at TEXT NOT NULL,
updated_at TEXT NOT NULL
);
CREATE INDEX IF NOT EXISTS idx_claims_client ON claims(client_id);
CREATE INDEX IF NOT EXISTS idx_claims_status ON claims(status);
CREATE INDEX IF NOT EXISTS idx_claims_debtor ON claims(debtor_id);
CREATE TABLE IF NOT EXISTS documents (
id TEXT PRIMARY KEY, -- uuid4
claim_id TEXT NOT NULL REFERENCES claims(id) ON DELETE CASCADE,
original_name TEXT NOT NULL, -- sanitized display name
stored_path TEXT NOT NULL, -- absolute path on disk (uuid-named)
mime_type TEXT NOT NULL,
size_bytes INTEGER NOT NULL,
sha256 TEXT NOT NULL, -- integrity + dedupe
uploaded_by TEXT NOT NULL DEFAULT 'CLIENT' -- CLIENT | STAFF
CHECK (uploaded_by IN ('CLIENT','STAFF')),
twentycrm_id TEXT, -- nullable
created_at TEXT NOT NULL
);
CREATE INDEX IF NOT EXISTS idx_documents_claim ON documents(claim_id);
CREATE TABLE IF NOT EXISTS case_notes (
id TEXT PRIMARY KEY, -- uuid4
claim_id TEXT NOT NULL REFERENCES claims(id) ON DELETE CASCADE,
author_type TEXT NOT NULL
CHECK (author_type IN ('CLIENT','STAFF','SYSTEM')),
author_name TEXT NOT NULL,
subject TEXT, -- for client->team structured messages
content TEXT NOT NULL, -- plaintext; rendered escaped
visibility TEXT NOT NULL DEFAULT 'SHARED'
CHECK (visibility IN ('SHARED','INTERNAL')),
twentycrm_id TEXT, -- nullable
created_at TEXT NOT NULL
);
CREATE INDEX IF NOT EXISTS idx_notes_claim ON case_notes(claim_id);
CREATE TABLE IF NOT EXISTS auth_tokens (
id TEXT PRIMARY KEY, -- uuid4
client_id TEXT NOT NULL REFERENCES clients(id) ON DELETE CASCADE,
token_hash TEXT UNIQUE NOT NULL, -- sha256 of raw token
expires_at TEXT NOT NULL, -- created_at + 15 min
consumed_at TEXT, -- NULL = unused
requested_ip TEXT,
created_at TEXT NOT NULL
);
CREATE INDEX IF NOT EXISTS idx_tokens_hash ON auth_tokens(token_hash);
CREATE INDEX IF NOT EXISTS idx_tokens_client ON auth_tokens(client_id);
CREATE TABLE IF NOT EXISTS sessions (
id TEXT PRIMARY KEY, -- uuid4
client_id TEXT NOT NULL REFERENCES clients(id) ON DELETE CASCADE,
session_hash TEXT UNIQUE NOT NULL, -- sha256 of raw session token
expires_at TEXT NOT NULL, -- + 7 days
revoked_at TEXT,
created_at TEXT NOT NULL,
last_seen_at TEXT
);
CREATE INDEX IF NOT EXISTS idx_sessions_hash ON sessions(session_hash);
CREATE TABLE IF NOT EXISTS audit_log (
id TEXT PRIMARY KEY, -- uuid4
entity_type TEXT NOT NULL, -- 'claim'|'client'|'document'|'note'
entity_id TEXT NOT NULL,
action TEXT NOT NULL, -- 'create'|'status_change'|'update'|'upload'|'note_add'
field TEXT,
old_value TEXT,
new_value TEXT,
actor TEXT NOT NULL,
reason TEXT,
created_at TEXT NOT NULL
);
CREATE INDEX IF NOT EXISTS idx_audit_entity ON audit_log(entity_type, entity_id);
CREATE TABLE IF NOT EXISTS number_sequences (
prefix TEXT NOT NULL, -- 'DRE'|'CLT'
year INTEGER NOT NULL,
last_value INTEGER NOT NULL DEFAULT 0,
PRIMARY KEY (prefix, year)
);
-- Migration tracking table
CREATE TABLE IF NOT EXISTS schema_migrations (
version TEXT PRIMARY KEY,
applied_at TEXT NOT NULL
);
+460
View File
@@ -0,0 +1,460 @@
"""Internal staff endpoints (staff-key auth): claims list/detail/patch, notes, documents, stats, audit."""
from __future__ import annotations
import hashlib
import html
import logging
import os
import uuid
from fastapi import APIRouter, Depends, File, HTTPException, Request, UploadFile, status
from fastapi.responses import JSONResponse
from pydantic import ValidationError
from . import auth as authmod
from . import db
from .db import get_conn, get_upload_dir, new_uuid, utcnow_iso
from .models import StaffClaimPatch, StaffNoteCreate
from .claims import ( # noqa: E402
MAX_FILE_BYTES, ALLOWED_EXT, MAGIC_BYTES, EXT_TO_KIND, STATUS_LABELS,
TIER_STEPS, _money, _err,
)
logger = logging.getLogger("dre.staff")
router = APIRouter()
# ---------------------------------------------------------------
# GET /api/staff/claims
# ---------------------------------------------------------------
@router.get("/api/staff/claims")
async def staff_list_claims(request: Request, _staff=Depends(authmod.require_staff)):
status_filter = request.query_params.get("status")
tier_filter = request.query_params.get("tier")
q = request.query_params.get("q", "").strip()
limit = min(int(request.query_params.get("limit", "50")), 200)
offset = max(int(request.query_params.get("offset", "0")), 0)
where = []
params: list = []
if status_filter:
where.append("c.status = ?")
params.append(status_filter)
if tier_filter:
where.append("c.tier = ?")
params.append(tier_filter)
if q:
where.append("(c.claim_number LIKE ? OR cl.company_name LIKE ? OR d.name LIKE ?)")
like = f"%{q}%"
params.extend([like, like, like])
where_sql = ("WHERE " + " AND ".join(where)) if where else ""
sql = (
"SELECT c.claim_number, c.status, c.tier, c.amount_cents, c.created_at, c.date_resolved, "
"c.date_assigned, cl.company_name, cl.client_number, d.name AS debtor_name, d.business_type "
"FROM claims c JOIN clients cl ON cl.id = c.client_id JOIN debtors d ON d.id = c.debtor_id "
f"{where_sql} ORDER BY c.created_at DESC LIMIT ? OFFSET ?"
)
params.extend([limit, offset])
with get_conn() as conn:
rows = conn.execute(sql, tuple(params)).fetchall()
total = conn.execute(f"SELECT COUNT(*) AS n FROM claims c JOIN clients cl ON cl.id=c.client_id JOIN debtors d ON d.id=c.debtor_id {where_sql}", tuple(params[:-2])).fetchone()["n"]
return {
"claims": [
{
"claim_number": r["claim_number"],
"status": r["status"],
"status_label": STATUS_LABELS.get(r["status"], r["status"]),
"tier": r["tier"],
"amount_cents": r["amount_cents"],
"amount_display": _money(r["amount_cents"]),
"company_name": r["company_name"],
"client_number": r["client_number"],
"debtor_name": r["debtor_name"],
"business_type": r["business_type"],
"created_at": r["created_at"],
"date_assigned": r["date_assigned"],
"date_resolved": r["date_resolved"],
}
for r in rows
],
"total": total,
"limit": limit,
"offset": offset,
}
# ---------------------------------------------------------------
# GET /api/staff/claims/{claim_number}
# ---------------------------------------------------------------
@router.get("/api/staff/claims/{claim_number}")
async def staff_get_claim(claim_number: str, _staff=Depends(authmod.require_staff)):
with get_conn() as conn:
row = conn.execute(
"SELECT c.*, cl.company_name, cl.contact_name, cl.email AS client_email, cl.phone AS client_phone, "
"cl.client_number, d.name AS debtor_name, d.business_type, d.contact_email AS debtor_email, "
"d.contact_phone AS debtor_phone, d.physical_address AS debtor_address "
"FROM claims c JOIN clients cl ON cl.id = c.client_id JOIN debtors d ON d.id = c.debtor_id "
"WHERE c.claim_number = ?",
(claim_number,),
).fetchone()
if row is None:
return _err("not_found", "Claim not found.", status.HTTP_404_NOT_FOUND)
docs = conn.execute(
"SELECT id, original_name, size_bytes, mime_type, uploaded_by, created_at FROM documents WHERE claim_id = ?",
(row["id"],),
).fetchall()
notes = conn.execute(
"SELECT author_type, author_name, subject, content, visibility, created_at FROM case_notes "
"WHERE claim_id = ? ORDER BY created_at ASC",
(row["id"],),
).fetchall()
audit = conn.execute(
"SELECT entity_type, entity_id, action, field, old_value, new_value, actor, reason, created_at "
"FROM audit_log WHERE entity_type IN ('claim','document','note') AND entity_id = ? "
"ORDER BY created_at ASC",
(row["id"],),
).fetchall()
return {
"claim_number": row["claim_number"],
"status": row["status"],
"status_label": STATUS_LABELS.get(row["status"], row["status"]),
"tier": row["tier"],
"tier_step": TIER_STEPS.get(row["tier"], 1),
"amount_cents": row["amount_cents"],
"amount_display": _money(row["amount_cents"]),
"description": row["description"],
"client_reference": row["client_reference"],
"invoice_date": row["invoice_date"],
"date_assigned": row["date_assigned"],
"date_resolved": row["date_resolved"],
"created_at": row["created_at"],
"client": {
"client_number": row["client_number"],
"company_name": row["company_name"],
"contact_name": row["contact_name"],
"email": row["client_email"],
"phone": row["client_phone"],
},
"debtor": {
"name": row["debtor_name"],
"business_type": row["business_type"],
"contact_email": row["debtor_email"],
"contact_phone": row["debtor_phone"],
"physical_address": row["debtor_address"],
},
"documents": [
{
"id": d["id"], "original_name": d["original_name"], "size_bytes": d["size_bytes"],
"mime_type": d["mime_type"], "uploaded_by": d["uploaded_by"], "created_at": d["created_at"],
} for d in docs
],
"notes": [
{
"author_type": n["author_type"], "author_name": n["author_name"], "subject": n["subject"],
"content": html.escape(n["content"]), "visibility": n["visibility"], "created_at": n["created_at"],
} for n in notes
],
"audit": [
{
"entity_type": a["entity_type"], "entity_id": a["entity_id"], "action": a["action"],
"field": a["field"], "old_value": a["old_value"], "new_value": a["new_value"],
"actor": a["actor"], "reason": a["reason"], "created_at": a["created_at"],
} for a in audit
],
}
# ---------------------------------------------------------------
# PATCH /api/staff/claims/{claim_number}
# ---------------------------------------------------------------
@router.patch("/api/staff/claims/{claim_number}")
async def staff_patch_claim(claim_number: str, request: Request, _staff=Depends(authmod.require_staff)):
try:
body = await request.json()
except Exception:
return _err("validation_error", "Invalid JSON body.", status.HTTP_422_UNPROCESSABLE_ENTITY)
try:
patch = StaffClaimPatch.model_validate(body)
except ValidationError as exc:
parts = [f"{'.'.join(str(x) for x in e['loc'])}: {e['msg']}" for e in exc.errors()]
return _err("validation_error", "; ".join(parts), status.HTTP_422_UNPROCESSABLE_ENTITY)
with get_conn() as conn:
row = conn.execute("SELECT * FROM claims WHERE claim_number = ?", (claim_number,)).fetchone()
if row is None:
return _err("not_found", "Claim not found.", status.HTTP_404_NOT_FOUND)
claim_id = row["id"]
now = utcnow_iso()
updates: list[tuple] = []
new_status = row["status"]
new_tier = row["tier"]
date_assigned = row["date_assigned"]
date_resolved = row["date_resolved"]
# Status change
if patch.status is not None and patch.status != row["status"]:
conn.execute(
"INSERT INTO audit_log (id, entity_type, entity_id, action, field, old_value, new_value, actor, reason, created_at) "
"VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)",
(new_uuid(), "claim", claim_id, "status_change", "status", row["status"], patch.status, "staff", patch.reason, now),
)
new_status = patch.status
# set date_assigned on first move out of NEW
if row["status"] == "NEW" and patch.status != "NEW" and not date_assigned:
date_assigned = now
# set date_resolved on resolved states
if patch.status in ("SETTLED", "CLOSED", "WRITE_OFF", "REJECTED") and not date_resolved:
date_resolved = now
# auto-add SYSTEM/SHARED note so client sees status change
conn.execute(
"INSERT INTO case_notes (id, claim_id, author_type, author_name, subject, content, visibility, twentycrm_id, created_at) "
"VALUES (?, ?, 'SYSTEM', 'System', NULL, ?, 'SHARED', NULL, ?)",
(new_uuid(), claim_id, f"Status updated to: {STATUS_LABELS.get(patch.status, patch.status)}", now),
)
# Tier change
if patch.tier is not None and patch.tier != row["tier"]:
conn.execute(
"INSERT INTO audit_log (id, entity_type, entity_id, action, field, old_value, new_value, actor, reason, created_at) "
"VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)",
(new_uuid(), "claim", claim_id, "status_change", "tier", row["tier"], patch.tier, "staff", patch.reason, now),
)
new_tier = patch.tier
conn.execute(
"UPDATE claims SET status = ?, tier = ?, date_assigned = ?, date_resolved = ?, updated_at = ? WHERE id = ?",
(new_status, new_tier, date_assigned, date_resolved, now, claim_id),
)
conn.commit()
# Return updated detail
return await staff_get_claim_inner(claim_number)
async def staff_get_claim_inner(claim_number: str):
"""Reuse detail query without re-checking staff auth."""
with get_conn() as conn:
row = conn.execute(
"SELECT c.*, cl.company_name, cl.contact_name, cl.email AS client_email, cl.phone AS client_phone, "
"cl.client_number, d.name AS debtor_name, d.business_type "
"FROM claims c JOIN clients cl ON cl.id = c.client_id JOIN debtors d ON d.id = c.debtor_id "
"WHERE c.claim_number = ?",
(claim_number,),
).fetchone()
if row is None:
return JSONResponse(status_code=status.HTTP_404_NOT_FOUND,
content={"error": {"code": "not_found", "message": "Claim not found."}})
return {
"claim_number": row["claim_number"],
"status": row["status"],
"status_label": STATUS_LABELS.get(row["status"], row["status"]),
"tier": row["tier"],
"tier_step": TIER_STEPS.get(row["tier"], 1),
"amount_cents": row["amount_cents"],
"amount_display": _money(row["amount_cents"]),
"date_assigned": row["date_assigned"],
"date_resolved": row["date_resolved"],
"updated_at": row["updated_at"],
}
# ---------------------------------------------------------------
# POST /api/staff/claims/{claim_number}/notes
# ---------------------------------------------------------------
@router.post("/api/staff/claims/{claim_number}/notes")
async def staff_add_note(claim_number: str, request: Request, _staff=Depends(authmod.require_staff)):
try:
body = await request.json()
except Exception:
return _err("validation_error", "Invalid JSON body.", status.HTTP_422_UNPROCESSABLE_ENTITY)
try:
note = StaffNoteCreate.model_validate(body)
except ValidationError as exc:
parts = [f"{'.'.join(str(x) for x in e['loc'])}: {e['msg']}" for e in exc.errors()]
return _err("validation_error", "; ".join(parts), status.HTTP_422_UNPROCESSABLE_ENTITY)
with get_conn() as conn:
row = conn.execute("SELECT id FROM claims WHERE claim_number = ?", (claim_number,)).fetchone()
if row is None:
return _err("not_found", "Claim not found.", status.HTTP_404_NOT_FOUND)
claim_id = row["id"]
now = utcnow_iso()
note_id = new_uuid()
conn.execute(
"INSERT INTO case_notes (id, claim_id, author_type, author_name, subject, content, visibility, twentycrm_id, created_at) "
"VALUES (?, ?, 'STAFF', ?, NULL, ?, ?, NULL, ?)",
(note_id, claim_id, note.author_name, note.content, note.visibility, now),
)
conn.execute(
"INSERT INTO audit_log (id, entity_type, entity_id, action, actor, created_at) VALUES (?, ?, ?, ?, ?, ?)",
(new_uuid(), "note", note_id, "note_add", f"staff:{note.author_name}", now),
)
conn.commit()
return {
"id": note_id,
"author_type": "STAFF",
"author_name": note.author_name,
"content": html.escape(note.content),
"visibility": note.visibility,
"created_at": now,
}
# ---------------------------------------------------------------
# POST /api/staff/claims/{claim_number}/documents
# ---------------------------------------------------------------
@router.post("/api/staff/claims/{claim_number}/documents")
async def staff_upload_document(claim_number: str, request: Request, file: UploadFile = File(...),
_staff=Depends(authmod.require_staff)):
cl = request.headers.get("content-length")
if cl and int(cl) > MAX_FILE_BYTES + 4096:
return _err("payload_too_large", "File exceeds 20 MB limit.", status.HTTP_413_REQUEST_ENTITY_TOO_LARGE)
ext = os.path.splitext(file.filename or "")[1].lower()
if ext not in ALLOWED_EXT:
return _err("unsupported_media_type", "File type not allowed.", status.HTTP_415_UNSUPPORTED_MEDIA_TYPE)
kind = EXT_TO_KIND.get(ext)
if not kind:
return _err("unsupported_media_type", "File type not allowed.", status.HTTP_415_UNSUPPORTED_MEDIA_TYPE)
with get_conn() as conn:
row = conn.execute("SELECT id FROM claims WHERE claim_number = ?", (claim_number,)).fetchone()
if row is None:
return _err("not_found", "Claim not found.", status.HTTP_404_NOT_FOUND)
claim_id = row["id"]
count = conn.execute("SELECT COUNT(*) AS n FROM documents WHERE claim_id = ?", (claim_id,)).fetchone()["n"]
if count >= 50:
return _err("conflict", "Document limit reached for this claim.", status.HTTP_409_CONFLICT)
upload_dir = os.path.join(get_upload_dir(), claim_id)
os.makedirs(upload_dir, exist_ok=True)
stored_uuid = str(uuid.uuid4())
stored_name = f"{stored_uuid}{ext}"
stored_path = os.path.join(upload_dir, stored_name)
sha = hashlib.sha256()
total = 0
magic_seen = False
with open(stored_path, "wb") as f:
while True:
chunk = await file.read(64 * 1024)
if not chunk:
break
total += len(chunk)
if total > MAX_FILE_BYTES:
f.close()
os.remove(stored_path)
return _err("payload_too_large", "File exceeds 20 MB limit.", status.HTTP_413_REQUEST_ENTITY_TOO_LARGE)
if not magic_seen:
sigs = MAGIC_BYTES.get(kind, ())
if sigs and any(chunk.startswith(s) for s in sigs):
magic_seen = True
elif sigs:
f.close()
os.remove(stored_path)
return _err("unsupported_media_type", "File content does not match extension.", status.HTTP_415_UNSUPPORTED_MEDIA_TYPE)
sha.update(chunk)
f.write(chunk)
os.chmod(stored_path, 0o640)
now = utcnow_iso()
doc_id = new_uuid()
mime = file.content_type or "application/octet-stream"
conn.execute(
"INSERT INTO documents (id, claim_id, original_name, stored_path, mime_type, size_bytes, sha256, uploaded_by, twentycrm_id, created_at) "
"VALUES (?, ?, ?, ?, ?, ?, ?, 'STAFF', NULL, ?)",
(doc_id, claim_id, os.path.basename(file.filename or "file"), stored_path, mime, total, sha.hexdigest(), now),
)
conn.execute(
"INSERT INTO audit_log (id, entity_type, entity_id, action, actor, created_at) VALUES (?, ?, ?, ?, ?, ?)",
(new_uuid(), "document", doc_id, "upload", "staff", now),
)
conn.commit()
return {
"id": doc_id,
"original_name": os.path.basename(file.filename or "file"),
"size_bytes": total,
"mime_type": mime,
"uploaded_by": "STAFF",
"created_at": now,
}
# ---------------------------------------------------------------
# GET /api/staff/stats
# ---------------------------------------------------------------
@router.get("/api/staff/stats")
async def staff_stats(_staff=Depends(authmod.require_staff)):
with get_conn() as conn:
total = conn.execute("SELECT COUNT(*) AS n FROM claims").fetchone()["n"]
by_status = {}
for r in conn.execute("SELECT status, COUNT(*) AS n FROM claims GROUP BY status").fetchall():
by_status[r["status"]] = r["n"]
by_tier = {}
for r in conn.execute("SELECT tier, COUNT(*) AS n FROM claims GROUP BY tier").fetchall():
by_tier[r["tier"]] = r["n"]
total_amount = conn.execute("SELECT COALESCE(SUM(amount_cents),0) AS s FROM claims").fetchone()["s"]
# recovered = claims in SETTLED/CLOSED
recovered = conn.execute(
"SELECT COALESCE(SUM(amount_cents),0) AS s FROM claims WHERE status IN ('SETTLED','CLOSED')"
).fetchone()["s"]
open_amount = total_amount - recovered
# aging: claims not resolved, by age
now_iso = utcnow_iso()
over_30 = conn.execute(
"SELECT COUNT(*) AS n FROM claims WHERE date_resolved IS NULL AND created_at < ?",
(over_30_cutoff(),),
).fetchone()["n"]
over_60 = conn.execute(
"SELECT COUNT(*) AS n FROM claims WHERE date_resolved IS NULL AND created_at < ?",
(over_60_cutoff(),),
).fetchone()["n"]
return {
"total_claims": total,
"by_status": by_status,
"by_tier": by_tier,
"total_amount_cents": total_amount,
"total_amount_display": _money(total_amount),
"recovered_amount_cents": recovered,
"recovered_amount_display": _money(recovered),
"open_amount_cents": open_amount,
"open_amount_display": _money(open_amount),
"aging": {"over_30_days": over_30, "over_60_days": over_60},
}
def over_30_cutoff() -> str:
from datetime import datetime, timedelta, timezone
return (datetime.now(timezone.utc) - timedelta(days=30)).strftime("%Y-%m-%dT%H:%M:%SZ")
def over_60_cutoff() -> str:
from datetime import datetime, timedelta, timezone
return (datetime.now(timezone.utc) - timedelta(days=60)).strftime("%Y-%m-%dT%H:%M:%SZ")
# ---------------------------------------------------------------
# GET /api/staff/audit
# ---------------------------------------------------------------
@router.get("/api/staff/audit")
async def staff_audit(request: Request, _staff=Depends(authmod.require_staff)):
entity_type = request.query_params.get("entity_type")
entity_id = request.query_params.get("entity_id")
limit = min(int(request.query_params.get("limit", "100")), 500)
offset = max(int(request.query_params.get("offset", "0")), 0)
where = []
params: list = []
if entity_type:
where.append("entity_type = ?")
params.append(entity_type)
if entity_id:
where.append("entity_id = ?")
params.append(entity_id)
where_sql = ("WHERE " + " AND ".join(where)) if where else ""
with get_conn() as conn:
rows = conn.execute(
f"SELECT * FROM audit_log {where_sql} ORDER BY created_at DESC LIMIT ? OFFSET ?",
tuple(params + [limit, offset]),
).fetchall()
total = conn.execute(f"SELECT COUNT(*) AS n FROM audit_log {where_sql}", tuple(params)).fetchone()["n"]
return {
"audit": [
{
"id": r["id"], "entity_type": r["entity_type"], "entity_id": r["entity_id"],
"action": r["action"], "field": r["field"], "old_value": r["old_value"],
"new_value": r["new_value"], "actor": r["actor"], "reason": r["reason"],
"created_at": r["created_at"],
} for r in rows
],
"total": total,
"limit": limit,
"offset": offset,
}
+721
View File
@@ -0,0 +1,721 @@
# DRE Customer Portal — Backend Architecture Specification
**Author:** Claude Opus 4-8 (System Architect)
**Date:** 2026-08-21
**Status:** BUILD-READY — hand off to GLM-5.2 (backend) + Sonnet 5 (frontend)
**Stack:** FastAPI + SQLite (single source of truth), magic-link auth, systemd + uvicorn behind Caddy
---
## 0. Scope & Principles
This spec defines the FIRST production backend for DRE. It replaces the 12 static mockups' dead
`<form>` with a live intake pipeline and adds a magic-link customer portal.
**In scope:** self-serve claim intake → creates client + claim → emails DRE team → returns claim
number; email magic-link auth (no passwords); customer portal (claim list/detail, document upload,
messaging); internal staff read/write endpoints (staff-key auth) that back the existing dashboards.
**Out of scope (fast-follow, do NOT block):** TwentyCRM sync, DocuSeal LPOA wiring, Stripe, AI
scoring, LetterStream, RON. Schema carries a nullable `twentycrm_id` on every synced entity so a
later one-way push is clean.
**Non-negotiable compliance:**
- NEVER collect/store SSNs, full bank account numbers, or card data. No column exists for them; the
intake validator rejects any field that pattern-matches a 9-digit SSN or a 13-19 digit PAN.
- All debtor + client data is PII. HTTPS only (Caddy terminates TLS). Secrets via env only.
- Store only what recovery needs (contract/invoice metadata + uploaded docs).
**Core conventions (locked, from platform spec):**
- Claim number: `DRE-YYYY-NNNN` (per-year sequence, zero-padded to 4).
- Client ID: `CLT-YYYY-NNNN` (per-year sequence, zero-padded to 4).
- Both generated server-side on first submission. Sequences are per calendar year.
---
## 1. SQLite Schema
**DB file:** `/opt/dre-portal/data/dre.db`
**Pragmas (set on every connection):** `PRAGMA foreign_keys = ON; PRAGMA journal_mode = WAL;
PRAGMA busy_timeout = 5000;`
All timestamps are ISO-8601 UTC strings (`YYYY-MM-DDTHH:MM:SSZ`), stored as TEXT. All monetary
amounts stored as INTEGER cents (never float). All primary keys are TEXT UUID4 unless noted.
```sql
-- ============================================================
-- clients : one row per customer account (the creditor / claimant)
-- ============================================================
CREATE TABLE clients (
id TEXT PRIMARY KEY, -- uuid4
client_number TEXT UNIQUE NOT NULL, -- CLT-YYYY-NNNN
company_name TEXT NOT NULL,
contact_name TEXT NOT NULL,
email TEXT UNIQUE NOT NULL, -- lowercased; magic-link identity
phone TEXT,
tos_accepted_at TEXT, -- set when ToS accepted at intake
twentycrm_id TEXT, -- nullable; set by future CRM sync
created_at TEXT NOT NULL,
updated_at TEXT NOT NULL
);
CREATE INDEX idx_clients_email ON clients(email);
-- ============================================================
-- debtors : the party the money is owed by (denormalized per claim is avoided;
-- one debtor row, referenced by claims). Minimal PII.
-- ============================================================
CREATE TABLE debtors (
id TEXT PRIMARY KEY, -- uuid4
name TEXT NOT NULL, -- business or individual name
business_type TEXT NOT NULL DEFAULT 'OTHER' -- enum below
CHECK (business_type IN
('INDIVIDUAL','SOLE_PROPRIETORSHIP','LLC','CORPORATION','PARTNERSHIP','OTHER')),
contact_email TEXT,
contact_phone TEXT,
physical_address TEXT, -- free-text single line; NOT named "address"
twentycrm_id TEXT, -- nullable
created_at TEXT NOT NULL,
updated_at TEXT NOT NULL
);
-- ============================================================
-- claims : the collection case. Belongs to one client, one debtor.
-- ============================================================
CREATE TABLE claims (
id TEXT PRIMARY KEY, -- uuid4
claim_number TEXT UNIQUE NOT NULL, -- DRE-YYYY-NNNN
client_id TEXT NOT NULL REFERENCES clients(id) ON DELETE RESTRICT,
debtor_id TEXT NOT NULL REFERENCES debtors(id) ON DELETE RESTRICT,
amount_cents INTEGER NOT NULL CHECK (amount_cents > 0),
currency TEXT NOT NULL DEFAULT 'USD',
status TEXT NOT NULL DEFAULT 'NEW' -- lifecycle enum below
CHECK (status IN
('NEW','UNDER_REVIEW','ACTIVE','NEGOTIATION','LEGAL','SETTLED','CLOSED','WRITE_OFF','REJECTED')),
tier TEXT NOT NULL DEFAULT 'TIER_1'
CHECK (tier IN ('TIER_1','TIER_2','TIER_2_5','TIER_3','TIER_4')),
description TEXT, -- what the debt is for (invoice desc, service)
client_reference TEXT, -- customer's own invoice/PO number
invoice_date TEXT, -- ISO date; when debt originated
date_assigned TEXT, -- set when moved out of NEW
date_resolved TEXT, -- set on SETTLED/CLOSED/WRITE_OFF
twentycrm_id TEXT, -- nullable
created_at TEXT NOT NULL,
updated_at TEXT NOT NULL
);
CREATE INDEX idx_claims_client ON claims(client_id);
CREATE INDEX idx_claims_status ON claims(status);
CREATE INDEX idx_claims_debtor ON claims(debtor_id);
-- ============================================================
-- documents : uploaded evidence, stored on disk; row holds metadata only
-- ============================================================
CREATE TABLE documents (
id TEXT PRIMARY KEY, -- uuid4
claim_id TEXT NOT NULL REFERENCES claims(id) ON DELETE CASCADE,
original_name TEXT NOT NULL, -- sanitized display name
stored_path TEXT NOT NULL, -- absolute path on disk (uuid-named)
mime_type TEXT NOT NULL,
size_bytes INTEGER NOT NULL,
sha256 TEXT NOT NULL, -- integrity + dedupe
uploaded_by TEXT NOT NULL DEFAULT 'CLIENT' -- CLIENT | STAFF
CHECK (uploaded_by IN ('CLIENT','STAFF')),
twentycrm_id TEXT, -- nullable
created_at TEXT NOT NULL
);
CREATE INDEX idx_documents_claim ON documents(claim_id);
-- ============================================================
-- case_notes : messages + internal notes on a claim (threaded log)
-- visibility controls whether the client can see it in the portal.
-- ============================================================
CREATE TABLE case_notes (
id TEXT PRIMARY KEY, -- uuid4
claim_id TEXT NOT NULL REFERENCES claims(id) ON DELETE CASCADE,
author_type TEXT NOT NULL -- who wrote it
CHECK (author_type IN ('CLIENT','STAFF','SYSTEM')),
author_name TEXT NOT NULL, -- display name (client contact, staff name, 'System')
subject TEXT, -- for client->team structured messages
content TEXT NOT NULL, -- plaintext; rendered escaped (see security)
visibility TEXT NOT NULL DEFAULT 'SHARED' -- SHARED = client sees it; INTERNAL = staff only
CHECK (visibility IN ('SHARED','INTERNAL')),
twentycrm_id TEXT, -- nullable
created_at TEXT NOT NULL
);
CREATE INDEX idx_notes_claim ON case_notes(claim_id);
-- ============================================================
-- auth_tokens : single-use magic-link tokens
-- ============================================================
CREATE TABLE auth_tokens (
id TEXT PRIMARY KEY, -- uuid4
client_id TEXT NOT NULL REFERENCES clients(id) ON DELETE CASCADE,
token_hash TEXT UNIQUE NOT NULL, -- sha256 of the raw token (raw never stored)
expires_at TEXT NOT NULL, -- created_at + 15 min
consumed_at TEXT, -- set on successful verify; NULL = unused
requested_ip TEXT, -- for rate-limit audit
created_at TEXT NOT NULL
);
CREATE INDEX idx_tokens_hash ON auth_tokens(token_hash);
CREATE INDEX idx_tokens_client ON auth_tokens(client_id);
-- ============================================================
-- sessions : bearer session tokens issued after magic-link verify
-- ============================================================
CREATE TABLE sessions (
id TEXT PRIMARY KEY, -- uuid4
client_id TEXT NOT NULL REFERENCES clients(id) ON DELETE CASCADE,
session_hash TEXT UNIQUE NOT NULL, -- sha256 of the raw session token
expires_at TEXT NOT NULL, -- created_at + 7 days (sliding not required v1)
revoked_at TEXT,
created_at TEXT NOT NULL,
last_seen_at TEXT
);
CREATE INDEX idx_sessions_hash ON sessions(session_hash);
-- ============================================================
-- audit_log : append-only trail for claim/status changes (compliance)
-- ============================================================
CREATE TABLE audit_log (
id TEXT PRIMARY KEY, -- uuid4
entity_type TEXT NOT NULL, -- 'claim' | 'client' | 'document' | 'note'
entity_id TEXT NOT NULL,
action TEXT NOT NULL, -- 'create' | 'status_change' | 'update' | 'upload' | 'note_add'
field TEXT, -- changed field name (nullable)
old_value TEXT,
new_value TEXT,
actor TEXT NOT NULL, -- staff email/name, client_number, or 'system'
reason TEXT,
created_at TEXT NOT NULL
);
CREATE INDEX idx_audit_entity ON audit_log(entity_type, entity_id);
-- ============================================================
-- number_sequences : per-year counters for claim/client numbers
-- (avoids race by using an atomic UPDATE...RETURNING in a txn)
-- ============================================================
CREATE TABLE number_sequences (
prefix TEXT NOT NULL, -- 'DRE' | 'CLT'
year INTEGER NOT NULL,
last_value INTEGER NOT NULL DEFAULT 0,
PRIMARY KEY (prefix, year)
);
```
### 1.1 Claim lifecycle status enum (authoritative)
| Value | Meaning | Client sees |
|----------------|-------------------------------------------------------------------|--------------------------|
| `NEW` | Just submitted via intake; awaiting DRE review | "Received" |
| `UNDER_REVIEW` | DRE reviewing docs / AI analysis | "Under Review" |
| `ACTIVE` | Approved; recovery in progress (tier drives the sub-stage) | "In Progress" |
| `NEGOTIATION` | Debtor engaged; settlement talks | "In Negotiation" |
| `LEGAL` | Referred to partner law firm (Tier 4) | "Legal Action" |
| `SETTLED` | Payment received / agreed; disbursement pending | "Settled" |
| `CLOSED` | Fully resolved & disbursed; binders generated | "Closed — Recovered" |
| `WRITE_OFF` | Uncollectible; case closed without recovery | "Closed — Uncollectible" |
| `REJECTED` | DRE declined the claim at intake review | "Not Accepted" |
**Tier enum:** `TIER_1` (Soft Touch), `TIER_2` (Formal Demand), `TIER_2_5` (Lien Threat), `TIER_3`
(Escalation), `TIER_4` (Legal Action). Tier is independent of status: a claim can be `ACTIVE` at any
tier. The frontend 4-step progress bar maps TIER_1→TIER_4 (TIER_2_5 renders as a sub-badge on TIER_2).
**Note on CRM enum divergence:** TwentyCRM Claims.status uses `NEW, ACTIVE, NEGOTIATION, LEGAL,
SETTLED, CLOSED, WRITE_OFF` and tier `TIER_1/2/3`. Our schema adds `UNDER_REVIEW`, `REJECTED`,
`TIER_2_5`. The future sync layer maps these: `UNDER_REVIEW``NEW`, `REJECTED``CLOSED`(+note),
`TIER_2_5``TIER_2`. This mapping is a sync-layer concern, not a schema constraint — build the
schema as specified above. **[CONDUCTOR DECISION #1 — see §7.]**
---
## 2. REST API Contract
**Base URL:** `https://portal.debtrecoveryexperts.com/api` (Caddy reverse-proxies `/api/*` to
`localhost:8090`; static HTML continues to be served by Caddy from `/var/www/capabilities/`).
**Auth models:**
- **Public** — no auth (intake, magic-link request/verify).
- **Client** — `Authorization: Bearer <session_token>`. Resolves to a `client_id`; every claim
query is scoped to that client. 401 if missing/invalid/expired.
- **Staff** — `X-DRE-Staff-Key: <key>` header, compared constant-time to env `DRE_STAFF_KEY`.
403 if absent/wrong. (v1 uses a single shared staff key; internal pages are already behind
Cloudflare Access, so this is defense-in-depth, not the primary gate.)
**Global conventions:**
- All request/response bodies are JSON (`Content-Type: application/json`) except document upload
(`multipart/form-data`).
- Errors: `{"error": {"code": "<machine_code>", "message": "<human msg>"}}` with appropriate HTTP
status. Codes: `validation_error`, `not_found`, `unauthorized`, `forbidden`, `rate_limited`,
`payload_too_large`, `unsupported_media_type`, `conflict`, `internal_error`.
- Money in responses returned BOTH as `amount_cents` (int) and `amount_display` (e.g. `"$15,000.00"`).
- Timestamps returned as ISO-8601 UTC.
### 2.1 Health
**`GET /api/health`** — Public. → `200 {"status":"ok","time":"<iso>"}`. No DB write.
### 2.2 Intake (public)
**`POST /api/intake`** — Public. Creates client (or reuses by email) + debtor + claim, writes a
`SYSTEM` case note, emails the DRE team, returns the claim number. This is what `debt-recovery.html`
posts to.
Request body:
```json
{
"client": {
"company_name": "Acme Builders LLC",
"contact_name": "Jane Doe",
"email": "jane@acmebuilders.com",
"phone": "512-555-0100"
},
"debtor": {
"name": "Delinquent Corp",
"business_type": "LLC",
"contact_email": "ap@delinquent.com",
"contact_phone": "214-555-0199",
"physical_address": "100 Main St, Dallas, TX 75201"
},
"claim": {
"amount_cents": 1500000,
"description": "Unpaid invoices for framing subcontract",
"client_reference": "INV-2048",
"invoice_date": "2026-03-15"
},
"tos_accepted": true,
"turnstile_token": "<optional; validated if TURNSTILE_SECRET set>"
}
```
Behavior:
- Validate all fields (see §6). `amount_cents` > 0 and ≤ 100_000_000 ($1M cap; larger flagged
`validation_error`**[CONDUCTOR DECISION #2]**). Reject if any free-text field matches an
SSN or PAN regex.
- If a client with this (lowercased) email exists, reuse it and update contact fields; else create
a new client with a fresh `CLT-YYYY-NNNN`. `tos_accepted` must be `true` → set `tos_accepted_at`.
- Always create a new debtor row + new claim (`status=NEW`, `tier=TIER_1`) with `DRE-YYYY-NNNN`.
- Insert `audit_log` create rows; insert a `SYSTEM`/`SHARED` case note "Claim received."
- Send email to `dre@debtrecoveryexperts.com` (team notification) via the germainebrown.com relay
(`mail.germainebrown.com:2525`, per platform email pitfalls) with claim summary. Email send
failure must NOT fail the request — log it, still return success (**[CONDUCTOR DECISION #3]**).
- Fire-and-forget confirmation email to the client (optional v1).
Response `201`:
```json
{
"claim_number": "DRE-2026-0001",
"client_number": "CLT-2026-0001",
"status": "NEW",
"message": "Claim received. Our team will review and contact you shortly."
}
```
### 2.3 Magic-link auth (public)
**`POST /api/auth/request`** — Public. Requests a login link.
```json
{ "email": "jane@acmebuilders.com" }
```
- Always returns `200 {"message":"If an account exists, a login link has been sent."}` regardless of
whether the email exists (no account enumeration).
- If the email maps to a client: generate a 32-byte URL-safe random token, store only its sha256
in `auth_tokens` with `expires_at = now + 15min`, email the link to the client via the DRE relay:
`https://portal.debtrecoveryexperts.com/portal/verify?token=<raw>`.
- Rate limits: max 3 requests per email per 15 min AND max 10 per IP per hour → `429 rate_limited`.
**`POST /api/auth/verify`** — Public. Exchanges a magic-link token for a session.
```json
{ "token": "<raw token from email link>" }
```
- Hash the token, look up an unconsumed, unexpired row. If none → `401 unauthorized`.
- Mark `consumed_at`, create a `sessions` row (7-day expiry), return the session token.
- Response `200`: `{ "session_token": "<raw>", "expires_at": "<iso>", "client": {"client_number":"CLT-2026-0001","company_name":"...","contact_name":"..."} }`
- Frontend stores `session_token` (localStorage or an HttpOnly cookie set by the backend —
**[CONDUCTOR DECISION #4: cookie vs bearer]**; spec defaults to bearer in localStorage for
simplicity, documented XSS mitigations in §6).
**`POST /api/auth/logout`** — Client. Revokes current session. → `200 {"message":"Logged out."}`
**`GET /api/auth/me`** — Client. → `200 { client: {...}, claim_count: N }`. Used by portal to
confirm session on load.
### 2.4 Claims (client)
**`GET /api/claims`** — Client. Lists the caller's claims (newest first).
```json
{ "claims": [
{ "claim_number":"DRE-2026-0001", "status":"ACTIVE", "status_label":"In Progress",
"tier":"TIER_2", "amount_cents":1500000, "amount_display":"$15,000.00",
"debtor_name":"Delinquent Corp", "created_at":"<iso>", "date_resolved":null }
]}
```
**`GET /api/claims/{claim_number}`** — Client. Full detail; 404 if not owned by caller (never leak
existence of other clients' claims — return 404, not 403).
```json
{
"claim_number":"DRE-2026-0001", "status":"ACTIVE", "status_label":"In Progress",
"tier":"TIER_2", "tier_step":2, "amount_cents":1500000, "amount_display":"$15,000.00",
"description":"Unpaid invoices...", "client_reference":"INV-2048",
"invoice_date":"2026-03-15", "date_assigned":"<iso>", "date_resolved":null,
"debtor": { "name":"Delinquent Corp", "business_type":"LLC" },
"documents": [ { "id":"...", "original_name":"invoice.pdf", "size_bytes":48210,
"mime_type":"application/pdf", "uploaded_by":"CLIENT", "created_at":"<iso>" } ],
"notes": [ { "author_type":"STAFF", "author_name":"Anita", "subject":null,
"content":"We've sent the first demand.", "created_at":"<iso>" } ]
}
```
Notes list returns only `visibility='SHARED'` rows for client callers. Debtor block excludes
internal fields (contact/address hidden from client — **[CONDUCTOR DECISION #5]**; spec default:
client sees debtor name + type only).
### 2.5 Documents
**`POST /api/claims/{claim_number}/documents`** — Client. `multipart/form-data`, field `file`.
- Enforce: max 20 MB per file; allowed MIME/extensions `.pdf .jpg .jpeg .png .doc .docx`; verify
magic bytes, not just extension. Reject others → `415 unsupported_media_type` / `413 payload_too_large`.
- Store to `/opt/dre-portal/data/uploads/<claim_id>/<uuid><ext>` (mode 0640), compute sha256, insert
`documents` row + audit + `SYSTEM` shared note "Document uploaded: <name>".
- Response `201`: the document metadata object.
**`GET /api/claims/{claim_number}/documents/{document_id}`** — Client or Staff. Streams the file with
`Content-Disposition: attachment`. 404 if not owned (client) / not found (staff). Never serve uploads
via Caddy static — always through this authenticated endpoint.
### 2.6 Messaging / case notes (client)
**`POST /api/claims/{claim_number}/messages`** — Client. Structured message to the team.
```json
{ "subject": "New information about the debtor", "content": "They changed their address to..." }
```
- `subject` must be one of the fixed options (validated): `Question about my claim`,
`New information about the debtor`, `Payment received / want to stop recovery`,
`Update my contact info`, `Complaint or concern`, `Other`.
- Insert `case_notes` (`author_type=CLIENT`, `visibility=SHARED`), audit, email the DRE team.
- Response `201`: the created note object.
(Client reads notes via the claim-detail endpoint §2.4; no separate GET needed for v1.)
### 2.7 Internal staff endpoints (staff-key auth)
These back the existing internal dashboards (replace mock rows).
**`GET /api/staff/claims`** — Staff. All claims with filters:
`?status=NEW&tier=TIER_2&q=<search>&limit=50&offset=0`. Search matches claim_number, company_name,
debtor_name. Returns claims joined with client + debtor summary + counts.
**`GET /api/staff/claims/{claim_number}`** — Staff. Full detail incl. INTERNAL notes, debtor contact
fields, all documents, and audit trail.
**`PATCH /api/staff/claims/{claim_number}`** — Staff. Update status/tier and resolution dates.
```json
{ "status":"ACTIVE", "tier":"TIER_2", "reason":"Docs approved, moving to formal demand" }
```
- Validate enum values. On status change to a resolved state, set `date_resolved`; on first move out
of `NEW`, set `date_assigned`. Write audit rows (old→new, actor=staff, reason). Optionally auto-add
a `SYSTEM`/`SHARED` note so the client sees the status change. **This is the write-back that flows
to the client portal.**
- Response `200`: updated claim detail.
**`POST /api/staff/claims/{claim_number}/notes`** — Staff. Add a note.
```json
{ "content":"Called debtor, left VM", "visibility":"INTERNAL", "author_name":"Tony" }
```
`visibility` defaults `INTERNAL`; set `SHARED` to make it client-visible. Response `201`.
**`POST /api/staff/claims/{claim_number}/documents`** — Staff. Same as client upload but
`uploaded_by=STAFF`; may be marked to appear (or not) to client via a `client_visible` flag
(**[CONDUCTOR DECISION #6]**; spec default: staff uploads are internal-only, not shown to client).
**`GET /api/staff/stats`** — Staff. Aggregate rollups for dashboard/analytics cards:
```json
{
"total_claims": 12, "by_status": {"NEW":3,"ACTIVE":5,"SETTLED":2,"CLOSED":2},
"by_tier": {"TIER_1":4,"TIER_2":5,"TIER_3":3},
"total_amount_cents": 42000000, "total_amount_display":"$420,000.00",
"recovered_amount_cents": 12000000, "open_amount_cents": 30000000,
"aging": { "over_30_days": 2, "over_60_days": 1 }
}
```
**`GET /api/staff/audit?entity_type=claim&entity_id=<id>`** — Staff. Audit trail for change history UI.
### 2.8 Endpoint summary table
| Method | Path | Auth | Purpose |
|--------|--------------------------------------------------|--------|----------------------------------|
| GET | `/api/health` | Public | Liveness |
| POST | `/api/intake` | Public | Create client+debtor+claim |
| POST | `/api/auth/request` | Public | Request magic link |
| POST | `/api/auth/verify` | Public | Exchange token → session |
| POST | `/api/auth/logout` | Client | Revoke session |
| GET | `/api/auth/me` | Client | Session/account check |
| GET | `/api/claims` | Client | List own claims |
| GET | `/api/claims/{claim_number}` | Client | Own claim detail |
| POST | `/api/claims/{claim_number}/documents` | Client | Upload document |
| GET | `/api/claims/{claim_number}/documents/{id}` | Client/Staff | Download document |
| POST | `/api/claims/{claim_number}/messages` | Client | Message the team |
| GET | `/api/staff/claims` | Staff | All claims + filters |
| GET | `/api/staff/claims/{claim_number}` | Staff | Full internal detail |
| PATCH | `/api/staff/claims/{claim_number}` | Staff | Update status/tier (write-back) |
| POST | `/api/staff/claims/{claim_number}/notes` | Staff | Add internal/shared note |
| POST | `/api/staff/claims/{claim_number}/documents` | Staff | Staff upload |
| GET | `/api/staff/stats` | Staff | Aggregate dashboard metrics |
| GET | `/api/staff/audit` | Staff | Change history |
---
## 3. Magic-Link Auth Flow
**Goal:** passwordless, secure-by-default client login.
1. **Request.** Client enters email on `login.html``POST /api/auth/request {email}`.
2. **Generate.** Backend: if email matches a client, create `token = secrets.token_urlsafe(32)`.
Store ONLY `sha256(token)` in `auth_tokens` with `expires_at = now + 15 minutes`, `consumed_at=NULL`,
`requested_ip`. Never store or log the raw token.
3. **Deliver.** Email the client (via `mail.germainebrown.com:2525`, from `dre@debtrecoveryexperts.com`)
a link: `https://portal.debtrecoveryexperts.com/portal/verify?token=<raw>`. Always respond `200`
with a generic message (anti-enumeration).
4. **Click.** The `verify` page reads `token` from the query string and calls
`POST /api/auth/verify {token}`.
5. **Exchange.** Backend hashes the token, finds a row that is unexpired AND unconsumed. If found:
set `consumed_at=now` (single-use), create a `sessions` row (`session_token=token_urlsafe(32)`,
store `sha256`, `expires_at = now + 7 days`), return the raw session token + client summary.
6. **Authenticated calls.** Frontend sends `Authorization: Bearer <session_token>` on every portal
API call. Backend hashes it, looks up a non-revoked, unexpired session, resolves `client_id`,
updates `last_seen_at`.
7. **Logout.** `POST /api/auth/logout` sets `revoked_at`.
**Security notes:**
- Tokens are 256-bit random (`secrets`), URL-safe. Only sha256 hashes are persisted → DB leak does
not yield usable tokens.
- Magic-link TTL 15 min; single-use (consumed on verify). Session TTL 7 days, revocable.
- Constant-time comparison for hashes and the staff key (`hmac.compare_digest`).
- Rate limit `/api/auth/request` (3/email/15min, 10/IP/hour) to stop link-spam / mailbox flooding.
- No account enumeration: identical `200` response whether or not the email exists.
- Verify page must POST the token (not GET-navigate to the API) so the raw token stays out of the
API's access logs / Referer chains; the page strips `?token=` from the URL after reading it.
- Expired/consumed tokens are pruned by a lightweight sweep on each verify attempt (delete rows
where `expires_at < now - 1 day`).
---
## 4. Frontend Page Inventory + Data Mapping
Existing files live in `/var/www/capabilities/` (public) and `/var/www/internal/` (staff). Sonnet 5
wires these to the API. **New pages** are flagged NEW.
| Page (file) | Location | Auth | Calls | Displays / Action |
|-------------------------------------|------------|-------------|----------------------------------------------------|-------------------|
| `debt-recovery.html` (intake) | public | none | `POST /api/intake` | Wire the dead `<form>`: collect Your Info / Debtor Info / Claim Details, submit JSON, show returned claim number + confirmation. Optional Turnstile. |
| `login.html` | public | none | `POST /api/auth/request` | Add an email field + "Email me a login link" button. Replace/append to the SSO-only pattern. Show "check your email" state. |
| `portal/verify` (NEW) | public | none→client | `POST /api/auth/verify` | Reads `?token`, exchanges for session, stores session token, redirects to client dashboard. Handles invalid/expired token error state. |
| `dre-client-dashboard.html` | public* | client | `GET /api/auth/me`, `GET /api/claims` | Replace empty-state/mock rows with real active + past claims, stat cards (count, recovered, open), tier progress bar. Requires session; redirect to login if 401. |
| `portal/claim` (NEW or extend dash) | public* | client | `GET /api/claims/{n}`, `POST .../documents`, `POST .../messages` | Claim detail: status/tier progress, document list + upload dropzone (real `<input type=file>`), shared notes thread, "message the team" form with fixed subjects. |
| `dre-dashboard.html` (internal) | internal | staff | `GET /api/staff/claims`, `PATCH /api/staff/claims/{n}` | Replace mock claim rows with live data; status/tier update controls that write back. |
| `dre-case-aging.html` | internal | staff | `GET /api/staff/claims` (sort by age), `GET /api/staff/stats` | Aging buckets from real `created_at`/`date_assigned`. |
| `dre-analytics.html` | internal | staff | `GET /api/staff/stats` | Replace static charts with real by_status / by_tier / recovered totals. |
| `inbox.html` | internal | staff | (unchanged — IMAP poller JSON) | Out of scope; keep as-is. |
| `letter-queue.html` | internal | staff | (v1: unchanged; later reads `GET /api/staff/claims`) | Not wired in v1. |
\* Client dashboard/claim pages are currently in the public docroot. Since they now require a session
token (enforced by the API — every data call is 401 without a valid session), they can stay in
`/var/www/capabilities/`; the pages themselves render an empty shell + "please log in" until the
session resolves. **[CONDUCTOR DECISION #7: keep client portal on `portal.` public docroot vs move
behind its own path.]** Spec default: keep in public docroot, gate by API session.
**Frontend session handling:** store the session token in `localStorage` under `dre_session`. Send
as `Authorization: Bearer`. On any `401`, clear it and redirect to `login.html`. (If Conductor picks
HttpOnly cookies in Decision #4, backend sets `Set-Cookie: dre_session=...; HttpOnly; Secure;
SameSite=Lax` and frontend drops the localStorage logic.)
---
## 5. Deployment Plan
Mirror the `/opt/ops-portal` pattern: venv + uvicorn under systemd, localhost port, Caddy in front.
### 5.1 Layout
```
/opt/dre-portal/
├── app/ # FastAPI code (from GLM-5.2)
│ ├── main.py # app + routers
│ ├── db.py # sqlite connection helper (pragmas), migrations runner
│ ├── schema.sql # the CREATE TABLE block from §1
│ ├── auth.py, intake.py, claims.py, staff.py, email.py, ...
├── data/
│ ├── dre.db # SQLite (WAL)
│ └── uploads/<claim_id>/ # uploaded docs, mode 0640
├── .env # secrets (mode 0600)
└── venv/ # python venv
```
Code home for git is `/root/projects/dre/` (repo). Deploy = `git pull` in the repo then rsync/symlink
the `app/` into `/opt/dre-portal/app/` (or clone the repo directly into `/opt/dre-portal` and run
from there — **[CONDUCTOR DECISION #8: run-from-repo vs deploy-copy]**; spec default: clone repo at
`/opt/dre-portal`, `data/` and `.env` gitignored).
### 5.2 Environment (`/opt/dre-portal/.env`, chmod 600)
```
DRE_STAFF_KEY=<64-hex random>
DRE_DB_PATH=/opt/dre-portal/data/dre.db
DRE_UPLOAD_DIR=/opt/dre-portal/data/uploads
DRE_BASE_URL=https://portal.debtrecoveryexperts.com
# Email relay (per platform email pitfalls — use germainebrown.com relay, NOT MXroute:587)
DRE_SMTP_HOST=mail.germainebrown.com
DRE_SMTP_PORT=2525
DRE_SMTP_FROM=dre@debtrecoveryexperts.com
DRE_TEAM_NOTIFY=dre@debtrecoveryexperts.com
DRE_SMTP_USER=<from ~/.hermes/.env DRE_EMAIL_* if relay auth required>
DRE_SMTP_PASS=<...>
TURNSTILE_SECRET=<optional; if unset, intake skips captcha check>
```
Reuse existing `DRE_EMAIL_*` creds from `~/.hermes/.env` for SMTP if the relay needs auth.
### 5.3 systemd unit — `/etc/systemd/system/dre-portal.service`
```ini
[Unit]
Description=DRE Customer Portal API (FastAPI/uvicorn)
After=network.target
[Service]
Type=simple
WorkingDirectory=/opt/dre-portal
EnvironmentFile=/opt/dre-portal/.env
ExecStart=/opt/dre-portal/venv/bin/uvicorn app.main:app --host 127.0.0.1 --port 8090
Restart=on-failure
RestartSec=3
# hardening
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=strict
ReadWritePaths=/opt/dre-portal/data
ProtectHome=true
[Install]
WantedBy=multi-user.target
```
Enable: `systemctl daemon-reload && systemctl enable --now dre-portal`.
### 5.4 Caddy routing
Extend the existing `portal.debtrecoveryexperts.com` block so `/api/*` proxies to the app while
static files continue to serve. **Order matters** — the `handle /api/*` must precede `file_server`.
```caddyfile
portal.debtrecoveryexperts.com {
handle /api/* {
reverse_proxy localhost:8090
}
handle {
root * /var/www/capabilities/
try_files {path} {path}.html /index.html
file_server
}
}
```
`pay.` and `internal.` blocks unchanged. After edit: `caddy validate --config /etc/caddy/Caddyfile
&& systemctl reload caddy`. **The app must NEVER be exposed on a public port — only 127.0.0.1:8090.**
### 5.5 Init / migration steps
1. `python3 -m venv /opt/dre-portal/venv`
2. `venv/bin/pip install fastapi uvicorn[standard] python-multipart` (+ `email-validator`; stdlib
`sqlite3`, `secrets`, `hashlib`, `hmac`, `smtplib` cover the rest — no ORM in v1).
3. On first boot, `db.py` runs `schema.sql` if the DB is absent (idempotent `CREATE TABLE IF NOT
EXISTS`), then seeds `number_sequences` for the current year if missing.
4. Migrations: numbered SQL files in `app/migrations/NNNN_*.sql`, tracked in a `schema_migrations`
table; runner applies unapplied ones on startup. (v1 ships with `0001_init.sql` == schema.sql.)
5. `chown -R dre-portal:dre-portal /opt/dre-portal/data` (or the service user); `chmod 750 data`,
`chmod 640` on the db file. **[CONDUCTOR DECISION #9: dedicated service user vs run as existing
ops user like ops-portal.]** Spec default: reuse the ops-portal service user pattern.
---
## 6. Security & Edge Cases
**Input validation (all endpoints):**
- Use Pydantic models for every request body; reject unknown fields (`extra="forbid"`).
- Email validated (`email-validator`), lowercased before storage/lookup.
- `amount_cents`: positive int, ≤ 100_000_000 (see Decision #2). Reject non-integer / float.
- `business_type`, `status`, `tier`, message `subject` validated against their fixed enums server-side.
- String length caps: names ≤ 200, description ≤ 5000, note content ≤ 10000, address ≤ 500.
- **PII rejection:** run every free-text intake field through SSN regex `\b\d{3}-?\d{2}-?\d{4}\b`
and PAN regex `\b(?:\d[ -]?){13,19}\b`; if matched, reject with `validation_error` "Do not include
Social Security or bank/card numbers." (Compliance-critical.)
**SQL injection:** ALL queries use parameterized statements (`?` placeholders / named params via
`sqlite3`). NEVER f-string/format user input into SQL. Table/column names are never taken from input.
**XSS in case notes / messages:** store content as raw plaintext; the frontend renders it with
`textContent` (never `innerHTML`) OR the backend returns an `content_html` that is HTML-escaped
server-side. Spec: **store raw, escape on output**; API returns escaped `content` and the frontend
inserts via `textContent`. No markdown/HTML allowed in v1. Subject is enum-only (no free text).
**Magic-link rate limiting:** in-process token-bucket / sliding-window counters keyed by email and
by IP (backed by a small in-memory dict with periodic cleanup; acceptable for single-instance v1).
`3/email/15min`, `10/IP/hour` on `/api/auth/request`; `10/IP/15min` on `/api/auth/verify` (brute-force
guard — though 256-bit tokens make guessing infeasible). Intake: `20/IP/hour` + Turnstile if configured.
**File upload:**
- Max 20 MB/file (enforced by reading `Content-Length` AND streaming with a hard byte cap).
- Allowlist extensions + MIME + magic-byte sniff (`python-magic` optional; else check known
signatures for PDF `%PDF`, JPEG `FFD8`, PNG `89504E47`, ZIP-based docx `504B0304`). Reject on
mismatch.
- Store OUTSIDE any web-served directory (`/opt/dre-portal/data/uploads`), uuid-named to prevent
path traversal; never trust `original_name` for the path. Sanitize `original_name` for display.
- Serve only via the authenticated download endpoint with `Content-Disposition: attachment` and a
safe `Content-Type` (or `application/octet-stream`) to prevent inline execution.
- Per-claim document count cap (e.g. 50) to prevent abuse.
**AuthZ / data isolation:** every client claim query filters by the session's `client_id`. Accessing
another client's `claim_number` returns `404` (not `403`) to avoid confirming existence. Staff key
compared with `hmac.compare_digest`.
**Transport & secrets:** HTTPS enforced by Caddy (app only on localhost). Secrets only from `.env`;
never logged. Redact tokens/keys from logs. Access logs must not contain the `?token=` query value
(verify uses POST).
**Other edge cases:**
- Duplicate email at intake → reuse client, still create new claim (a client can have many claims).
- Concurrent number generation → atomic `UPDATE number_sequences SET last_value = last_value + 1 ...
RETURNING last_value` inside the same transaction as the insert; retry on the (rare) SQLite busy.
- Year rollover → sequence keyed by `(prefix, year)`; new year starts at 0001 automatically.
- Email relay down → intake/messages still succeed (email is best-effort); failure logged + surfaced
in `audit_log` as a `note` action so staff can follow up.
- Clock/expiry → all comparisons in UTC; expired tokens/sessions rejected and lazily pruned.
- Empty portal (new client, no claims) → endpoints return empty arrays; frontend shows empty state.
---
## 7. Open Decisions for the Conductor
| # | Decision | Spec default (build this unless overridden) |
|---|----------|---------------------------------------------|
| 1 | Extra statuses (`UNDER_REVIEW`,`REJECTED`,`TIER_2_5`) diverge from TwentyCRM enums. Keep richer local enum? | **Yes** — keep richer enum; sync layer maps down later. |
| 2 | Max claim amount cap. | **$1,000,000** (100_000_000 cents); larger → validation error. |
| 3 | Should intake fail if the team-notification email fails to send? | **No** — email is best-effort; request still returns 201. |
| 4 | Session transport: Bearer token in localStorage vs HttpOnly cookie. | **Bearer in localStorage** (simpler; XSS mitigated by textContent rendering). |
| 5 | Does the client see debtor contact/address in claim detail? | **No** — client sees debtor name + type only. |
| 6 | Are staff-uploaded documents visible to the client? | **No** — staff uploads internal-only by default. |
| 7 | Keep client dashboard/claim pages in public docroot (API-gated) or move behind a path? | **Keep in public docroot**, gate by API session. |
| 8 | Deploy model: run FastAPI directly from the git repo clone at `/opt/dre-portal`, or copy `app/` from `/root/projects/dre`? | **Clone repo at `/opt/dre-portal`**; `data/` + `.env` gitignored. |
| 9 | Service user: dedicated `dre-portal` user vs reuse ops-portal user. | **Reuse ops-portal service-user pattern.** |
**Also flag to conductor (informational, not blocking):**
- TwentyCRM Payment→Claim and CaseNote→Claim relations are still missing (per current state). The
future sync layer will need them; not required for this backend.
- Turnstile secret not yet provisioned — intake ships with captcha check *conditional* on the env var,
so it works with or without it.
- LPOA/DocuSeal, Stripe, LetterStream, AI scoring are all explicitly deferred (fast-follow).
---
## 8. Handoff Notes
- **GLM-5.2 (backend):** implement §1 schema verbatim, §2 endpoints, §3 auth, §5 deploy, §6 security.
No ORM required — stdlib `sqlite3` with parameterized queries + Pydantic for validation. Keep raw
tokens out of the DB and logs. Reuse `~/.hermes/.env` `DRE_EMAIL_*` creds for SMTP via the
germainebrown.com relay.
- **Sonnet 5 (frontend):** wire the pages per §4. Every data call sends `Authorization: Bearer`;
render all user/staff text via `textContent`. Real `<input type=file>` dropzone (see platform
pitfall). `chmod 644` any new HTML in the webroots. Keep the D|R|E logo, nav, and theme
conventions from the platform skill.
- **Both:** the internal dashboards read from `/api/staff/*` with the `X-DRE-Staff-Key` header
(still behind Cloudflare Access). Status changes via `PATCH /api/staff/claims/{n}` are the
write-back that surfaces in the client portal.