From 6ebcb2de72b1d187c49b1f86f1fae3841394e854 Mon Sep 17 00:00:00 2001 From: Germaine Brown Date: Mon, 10 Aug 2026 17:40:15 -0400 Subject: [PATCH] =?UTF-8?q?docs:=20recover=20verdicttank=20architecture=20?= =?UTF-8?q?doc=20=E2=80=94=20now=20in=20MkDocs=20source,=20no=20longer=20o?= =?UTF-8?q?rphaned?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- docs-source/verdicttank/CHANGELOG.md | 6 + docs-source/verdicttank/architecture.md | 329 ++++++++++++++++++++++++ mkdocs.yml | 72 +++--- 3 files changed, 370 insertions(+), 37 deletions(-) create mode 100644 docs-source/verdicttank/architecture.md diff --git a/docs-source/verdicttank/CHANGELOG.md b/docs-source/verdicttank/CHANGELOG.md index 6a61ebf..59c093a 100644 --- a/docs-source/verdicttank/CHANGELOG.md +++ b/docs-source/verdicttank/CHANGELOG.md @@ -62,3 +62,9 @@ - **Fixed:** "Tech Architecture" in Access table now links to `/verdicttank/architecture/` instead of the page itself - **Added:** Callout link to full architecture doc in the Architecture section + +## 2026-08-10 — Architecture doc recovered + +- **Recovered:** Full architecture document at /verdicttank/architecture/ — rebuilt from pipeline spec after rsync --delete wipe +- **Added:** architecture.md is now a first-class MkDocs source file, not a manually deployed orphan + diff --git a/docs-source/verdicttank/architecture.md b/docs-source/verdicttank/architecture.md new file mode 100644 index 0000000..502b1b0 --- /dev/null +++ b/docs-source/verdicttank/architecture.md @@ -0,0 +1,329 @@ +# VerdictTank — Technical Architecture + +> **Version:** v1.0 +> **Status:** 3/3 Unanimous Conditional Go (Aug 10, 2026) +> **Owner:** Germaine Brown +> **Built by:** Sho'Nuff (Hermes Agent) + +--- + +## Overview + +VerdictTank is an AI-powered business proposal review platform. A user submits a business proposal, product spec, or architecture document. Five specialized agents process it across three phases, three independent judges deliver a majority-rules verdict, and a branded PDF report lands in the user's inbox. + +### Pipeline + +``` +User submits proposal + │ + ▼ +Phase 1: Research Agent + Live web verification — competitors, trademarks, domain WHOIS, + market data, pricing benchmarks. Citations attached to every claim. + │ + ▼ +Phase 2: Critic Agent + 10-dimension review (Name, Pricing, PMF, Competition, Financials, + GTM, Risk, Missing Elements, Founder Fit, Overall Verdict). + Scored 1-10 per dimension. + │ + ▼ +Phase 3: Three judges in parallel (majority rules) + Judge A — Independent architecture + Judge B — Independent architecture + Judge C — Independent architecture + │ + ▼ +PDF verdict delivered by email +``` + +### Verdict System + +| Result | Meaning | +|---|---| +| 3/3 Unanimous | Maximum confidence — all judges agree | +| 2/3 Majority | Majority verdict with dissenting opinion | +| 1-1-1 Split | Human (Germaine) breaks the tie | +| Degraded (2 judges) | One judge unavailable — runs with remaining two | + +--- + +## Component Architecture + +### Server: Core (netcup RS 2000 — 152.53.192.33) + +| Component | Technology | Port | Role | +|---|---|---|---| +| Caddy | Caddy 2 | 80/443 | Reverse proxy, TLS termination, static file serving | +| VerdictTank API | FastAPI (Python) | 8201 | Review orchestration, corpus queries, email dispatch | +| Corpus DB | SQLite | — | Proposal corpus, benchmark data, outcome tracking | +| PDF Generator | WeasyPrint | — | HTML-to-PDF rendering from `pdf-template.html` | +| Gitea Actions Runner | Gitea Act Runner | — | CI/CD for VerdictTank repo | + +### AI Models (via admin-ai) + +| Phase | Model | Role | +|---|---|---| +| Research | Worker model | Web search, claim verification | +| Critic | Reviewer model | 10-dimension critical review | +| Judge A | Independent model | Architecture A — validation | +| Judge B | Independent model | Architecture B — cross-check | +| Judge C | Independent model | Architecture C — operational realism | + +All model calls route through `admin-ai.itpropartner.com` with the `verdicttank-prod` API key. Multi-provider architecture ensures no single vendor lock-in and genuine cross-model disagreement. + +### Caddy Routing + +``` +verdicttank.com, www.verdicttank.com { + handle /api/verdicttank/* { + reverse_proxy 127.0.0.1:8201 + } + handle /reports/* { + root * /var/www/verdicttank + file_server + } + redir /reports /reports/ permanent +} +``` + +Crucial detail: use `handle` (not `handle_path`) for `/reports/*` — `handle_path` strips the prefix, breaking file resolution for `/reports/Client-Report.pdf`. + +--- + +## Data Model + +### Review Record + +``` +Review +├── id: UUID +├── client_name: str +├── proposal_text: str (full submitted document) +├── pricing_tier: enum[free, pro, enterprise, whitelabel] +├── status: enum[pending, research, critique, judging, done, failed] +├── created_at: datetime +├── completed_at: datetime? +│ +├── research_brief: ResearchBrief? +├── critic_review: CriticReview? +├── judge_verdicts: [JudgeVerdict] +├── final_verdict: enum[go, no_go, conditional_go]? +├── pdf_path: str? +│ +├── cost_breakdown: CostBreakdown +└── turnaround_seconds: int? +``` + +### ResearchBrief + +``` +ResearchBrief +├── verified_claims: [{claim, source_url, confidence}] +├── discrepancies: [{claimed, actual, severity, evidence}] +├── competitor_analysis: [{name, url, pricing, features, notes}] +├── domain_check: {domain, registered, expiry, registrar} +├── trademark_risk: enum[none, low, medium, high, blocked] +└── market_data: [{statistic, value, source, verified}] +``` + +### CriticReview + +``` +CriticReview +├── dimension_scores: {name: str, score: int, notes: str}[10] +├── fatal_flaws: [{description, severity, condition_to_fix}] +├── strengths: [str] +├── blind_spots: [str] +├── overall_verdict: enum[go, no_go, conditional_go] +└── conditions: [str] +``` + +### JudgeVerdict + +``` +JudgeVerdict +├── judge_id: str (Judge 1/2/3) +├── verdict: enum[go, no_go, conditional_go] +├── confidence: float (0.0-1.0) +├── agreements_with_critic: [str] +├── disagreements_with_critic: [str] +├── novel_insights: [str] ← findings no other judge caught +├── second_order_effects: [str] +└── conditions: [str] +``` + +--- + +## PDF Generation + +The PDF pipeline at `/root/projects/verdicttank/`: + +1. **HTML Template** (`pdf-template.html`) — 11-section branded report: + - Cover page with verdict badge + - Executive summary + - Dimension score table + - Research findings with citations + - Fatal flaws ranked by severity + - Priority-ranked action plan + - Judge cards (anonymized — "Judge 1/2/3", no model names) + - Judge notes section + - **Disclaimer** (Not Legal Advice, Not Financial Advice, AI-Generated Content Disclosure, No Warranty, Limitation of Liability, Confidentiality Notice) + - Citations + +2. **Generator** (`generate-pdf.py`) — Reads structured JSON data, fills the template via WeasyPrint, outputs branded PDF + +3. **Quality Gate** (post-generation): +```python +from pypdf import PdfReader +r = PdfReader('report.pdf') +text = ''.join(p.extract_text() or '' for p in r.pages) +assert '—' not in text, 'EM DASH FOUND' +assert 'dispatch' not in text.lower(), 'DISPATCH FOUND' +assert 'Disclaimer' in text, 'DISCLAIMER SECTION MISSING' +assert len(r.pages) >= 10, f'Expected 10+ pages, got {len(r.pages)}' +``` + +### Sanitization Rules (hard constraints) + +| Rule | Enforcement | +|---|---| +| No model names anywhere in PDF | `build_judge_cards()` uses data `name` field — expects "Judge 1/2/3" | +| No internal methodology exposed | No "pipeline", "conductor", "Opus", "Sonnet", "Qwen", "Gemini" | +| No em dashes | `clean()` function strips `\u2014` and `\u2013` | +| 11-section report (was 10 before Aug 8) | Disclaimer is Section 10, Citations is Section 11 | +| Filename: `-VerdictTank-Report.pdf` | Safe-filename generator strips non-alphanumeric | + +--- + +## Pricing Integration + +| Tier | Price | Reviews/Month | Model Access | +|---|---|---|---| +| Free | $0 | 1 | Limited model set | +| Pro | $79/mo | 10 ($8 overage) | Full model set | +| Enterprise | $499/mo | 50 ($12 overage) | API access, prediction tracking | +| White-Label | $1,999+/mo | Unlimited | Multi-tenant, custom branding | + +Cost per review: $0.70-0.75 (v3 corpus-backed). Turnaround: 5-15 minutes. + +--- + +## Cost Model + +### Per-Review Breakdown + +| Phase | Tokens (est.) | Cost | +|---|---|---| +| Research Agent | ~5K in, ~3K out | ~$0.10 | +| Critic Agent | ~15K in, ~5K out | ~$0.25 | +| Judge A | ~20K in, ~3K out | ~$0.12 | +| Judge B | ~20K in, ~2K out | ~$0.05 | +| Judge C | ~20K in, ~2K out | ~$0.05 | +| PDF Generation | N/A | ~$0.01 | +| **Total** | | **~$0.58-0.75** | + +At 10 reviews/month (Pro tier): ~$7.50 cost, $79 revenue = ~90% gross margin. +At scale (100+ reviews/month): corpus caching reduces research costs significantly. + +--- + +## Security Model + +### Attack Surface + +| Surface | Risk | Mitigation | +|---|---|---| +| Prompt injection via proposal text | HIGH | Input sanitization, structured parsing, no raw tool execution from user input | +| Research agent web access | MEDIUM | URL allowlist, rate limiting, response size caps | +| PDF injection via WeasyPrint | LOW | No user-controlled HTML in template, all content sanitized | +| API key exposure | LOW | `verdicttank-prod` key scoped to admin-ai, usage caps | +| Report access | LOW | `/reports/` directory served by Caddy, no directory listing | +| Corpus poisoning | LOW | SQLite read-only for judge queries, write path gated | + +### Input Sanitization Pipeline + +``` +Raw proposal submission + → Strip control characters + → Validate UTF-8 + → Size cap (100KB) + → MIME type validation + → Pass to Research Agent as structured data, not raw prompt +``` + +--- + +## Failure Modes & Recovery + +| Failure | Impact | Recovery | +|---|---|---| +| Single judge timeout/crash | Degraded to 2 judges | Majority of 2 rules; note missing judge in addendum | +| All judges fail | No verdict produced | Free re-run; investigate admin-ai | +| Research Agent web failure | Incomplete citations | Flag as "limited research" in brief; proceed with available data | +| PDF generation failure | Report not delivered | Retry once; email plain-text summary on persistent failure | +| Core server outage | Platform unavailable | Warm standby (app1-bu) can take over Caddy + API | +| admin-ai outage | All phases blocked | Model failover chain (Flash → Gemini → Grok → Sonnet) | +| SQLite corruption | Corpus lost | Nightly backup to S3; rebuild from backup | + +--- + +## Operations + +### Monitoring + +| Metric | Source | Alert Threshold | +|---|---|---| +| API response time | Prometheus (Core:9090) | >30s p95 | +| Review success rate | App-level counter | <90% over 1h | +| PDF generation failures | App-level counter | Any failure | +| admin-ai latency | LiteLLM metrics | >10s p95 | +| Disk usage (reports) | node_exporter | >80% | + +### Backup + +- **Corpus DB:** Nightly to S3 via `hermes-backup.sh` +- **PDF reports:** `/var/www/verdicttank/reports/` included in full backup +- **API config:** In git repo, pushed to Gitea + +--- + +## Known Limitations + +1. **Single-box architecture** — Core is a single VPS. If Core goes down, VerdictTank is down until app1-bu failover completes (~5-10 min). A dedicated VerdictTank instance would eliminate this dependency. +2. **No payment integration** — Currently manual billing. Stripe integration planned for Phase 2. +3. **No user accounts** — Each review is standalone. User dashboard and review history planned for Phase 2. +4. **Corpus freshness** — Benchmark data is static until manually refreshed. Automated crawl planned. +5. **PDF email delivery** — Via MXroute SMTP, no delivery tracking. Postmark/SendGrid integration planned. + +--- + +## Roadmap + +| Phase | Deliverable | Status | +|---|---|---| +| Phase 0 | PDF pipeline, corpus DB, basic API | **DONE** | +| Phase 1 | Outcome tracking, accuracy engine, corpus refresh | Proposed | +| Phase 2 | Stripe integration, user dashboard, API access | Proposed | +| Phase 3 | Multi-tenant, white-label, SSO | Proposed | + +--- + +## Deployment + +``` +# Deploy site +scp -i /root/.ssh/itpp-infra /tmp/verdicttank-architecture.md \ + root@152.53.192.33:/root/itpp-docs/docs-source/verdicttank/architecture.md + +# Build and deploy +ssh core "cd /root/itpp-docs && mkdocs build --clean && \ + tar czf - site/" | ssh app3 "tar xzf - --strip-components=1 \ + -C /home/ippadmin/htdocs/docs.itpropartner.com/" +``` + +--- + +*Last updated: August 10, 2026* +*Pipeline verdict: 3/3 Unanimous Conditional Go* +*Commit: see git.itpropartner.com/ippadmin/itpp-docs* diff --git a/mkdocs.yml b/mkdocs.yml index 91dc361..a68783a 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -1,9 +1,8 @@ -site_name: "IT Pro Partner Docs" -site_url: "https://docs.itpropartner.com/" -repo_url: "https://git.itpropartner.com/ippadmin/itpp-docs" +site_name: IT Pro Partner Docs +site_url: https://docs.itpropartner.com/ +repo_url: https://git.itpropartner.com/ippadmin/itpp-docs edit_uri: edit/main/docs-source/ docs_dir: docs-source - theme: name: material palette: @@ -11,39 +10,38 @@ theme: primary: indigo accent: indigo features: - - navigation.instant - - navigation.tracking - - navigation.tabs - - navigation.sections - - search.highlight - - search.share - + - navigation.instant + - navigation.tracking + - navigation.tabs + - navigation.sections + - search.highlight + - search.share plugins: - - search - +- search markdown_extensions: - - admonition - - pymdownx.details - - pymdownx.superfences - - pymdownx.highlight - - tables - - toc: - permalink: true - +- admonition +- pymdownx.details +- pymdownx.superfences +- pymdownx.highlight +- tables +- toc: + permalink: true nav: - - Home: index.md - - Projects: - - ITPP Infrastructure: - - Overview: itpp-infrastructure/index.md - - Docs Auth Gate: itpp-infrastructure/docs-auth-gate.md - - ITPP Standards: itpp-standards/index.md - - TransitPin: transitpin/index.md - - HomeLab: homelab/index.md - - Scripts: scripts/index.md - - FleetTracker360: fleettracker360/index.md - - Shark Game: shark-game/index.md - - VerdictTank: verdicttank/index.md - - Apex Track: apex-track/index.md - - BoxPilot: boxpilot/index.md - - OSINT Tool: osint-tool/index.md - - LaunchCheck: launchcheck/index.md +- Home: index.md +- Projects: + - ITPP Infrastructure: + - Overview: itpp-infrastructure/index.md + - Docs Auth Gate: itpp-infrastructure/docs-auth-gate.md + - ITPP Standards: itpp-standards/index.md + - TransitPin: transitpin/index.md + - HomeLab: homelab/index.md + - Scripts: scripts/index.md + - FleetTracker360: fleettracker360/index.md + - Shark Game: shark-game/index.md + - VerdictTank: + - Overview: verdicttank/index.md + - Architecture: verdicttank/architecture.md + - Apex Track: apex-track/index.md + - BoxPilot: boxpilot/index.md + - OSINT Tool: osint-tool/index.md + - LaunchCheck: launchcheck/index.md