330 lines
11 KiB
Markdown
330 lines
11 KiB
Markdown
# VerdictTank — Technical Architecture
|
|
|
|
> **Version:** v1.0
|
|
> **Status:** 3/3 Unanimous Conditional Go (Aug 10, 2026)
|
|
> **Owner:** Germaine Brown
|
|
> **Built by:** Sho'Nuff (Hermes Agent)
|
|
|
|
---
|
|
|
|
## Overview
|
|
|
|
VerdictTank is an AI-powered business proposal review platform. A user submits a business proposal, product spec, or architecture document. Five specialized agents process it across three phases, three independent judges deliver a majority-rules verdict, and a branded PDF report lands in the user's inbox.
|
|
|
|
### Pipeline
|
|
|
|
```
|
|
User submits proposal
|
|
│
|
|
▼
|
|
Phase 1: Research Agent
|
|
Live web verification — competitors, trademarks, domain WHOIS,
|
|
market data, pricing benchmarks. Citations attached to every claim.
|
|
│
|
|
▼
|
|
Phase 2: Critic Agent
|
|
10-dimension review (Name, Pricing, PMF, Competition, Financials,
|
|
GTM, Risk, Missing Elements, Founder Fit, Overall Verdict).
|
|
Scored 1-10 per dimension.
|
|
│
|
|
▼
|
|
Phase 3: Three judges in parallel (majority rules)
|
|
Judge A — Independent architecture
|
|
Judge B — Independent architecture
|
|
Judge C — Independent architecture
|
|
│
|
|
▼
|
|
PDF verdict delivered by email
|
|
```
|
|
|
|
### Verdict System
|
|
|
|
| Result | Meaning |
|
|
|---|---|
|
|
| 3/3 Unanimous | Maximum confidence — all judges agree |
|
|
| 2/3 Majority | Majority verdict with dissenting opinion |
|
|
| 1-1-1 Split | Human (Germaine) breaks the tie |
|
|
| Degraded (2 judges) | One judge unavailable — runs with remaining two |
|
|
|
|
---
|
|
|
|
## Component Architecture
|
|
|
|
### Server: Core (netcup RS 2000 — 152.53.192.33)
|
|
|
|
| Component | Technology | Port | Role |
|
|
|---|---|---|---|
|
|
| Caddy | Caddy 2 | 80/443 | Reverse proxy, TLS termination, static file serving |
|
|
| VerdictTank API | FastAPI (Python) | 8201 | Review orchestration, corpus queries, email dispatch |
|
|
| Corpus DB | SQLite | — | Proposal corpus, benchmark data, outcome tracking |
|
|
| PDF Generator | WeasyPrint | — | HTML-to-PDF rendering from `pdf-template.html` |
|
|
| Gitea Actions Runner | Gitea Act Runner | — | CI/CD for VerdictTank repo |
|
|
|
|
### AI Models (via admin-ai)
|
|
|
|
| Phase | Model | Role |
|
|
|---|---|---|
|
|
| Research | Worker model | Web search, claim verification |
|
|
| Critic | Reviewer model | 10-dimension critical review |
|
|
| Judge A | Independent model | Architecture A — validation |
|
|
| Judge B | Independent model | Architecture B — cross-check |
|
|
| Judge C | Independent model | Architecture C — operational realism |
|
|
|
|
All model calls route through `admin-ai.itpropartner.com` with the `verdicttank-prod` API key. Multi-provider architecture ensures no single vendor lock-in and genuine cross-model disagreement.
|
|
|
|
### Caddy Routing
|
|
|
|
```
|
|
verdicttank.com, www.verdicttank.com {
|
|
handle /api/verdicttank/* {
|
|
reverse_proxy 127.0.0.1:8201
|
|
}
|
|
handle /reports/* {
|
|
root * /var/www/verdicttank
|
|
file_server
|
|
}
|
|
redir /reports /reports/ permanent
|
|
}
|
|
```
|
|
|
|
Crucial detail: use `handle` (not `handle_path`) for `/reports/*` — `handle_path` strips the prefix, breaking file resolution for `/reports/Client-Report.pdf`.
|
|
|
|
---
|
|
|
|
## Data Model
|
|
|
|
### Review Record
|
|
|
|
```
|
|
Review
|
|
├── id: UUID
|
|
├── client_name: str
|
|
├── proposal_text: str (full submitted document)
|
|
├── pricing_tier: enum[free, pro, enterprise, whitelabel]
|
|
├── status: enum[pending, research, critique, judging, done, failed]
|
|
├── created_at: datetime
|
|
├── completed_at: datetime?
|
|
│
|
|
├── research_brief: ResearchBrief?
|
|
├── critic_review: CriticReview?
|
|
├── judge_verdicts: [JudgeVerdict]
|
|
├── final_verdict: enum[go, no_go, conditional_go]?
|
|
├── pdf_path: str?
|
|
│
|
|
├── cost_breakdown: CostBreakdown
|
|
└── turnaround_seconds: int?
|
|
```
|
|
|
|
### ResearchBrief
|
|
|
|
```
|
|
ResearchBrief
|
|
├── verified_claims: [{claim, source_url, confidence}]
|
|
├── discrepancies: [{claimed, actual, severity, evidence}]
|
|
├── competitor_analysis: [{name, url, pricing, features, notes}]
|
|
├── domain_check: {domain, registered, expiry, registrar}
|
|
├── trademark_risk: enum[none, low, medium, high, blocked]
|
|
└── market_data: [{statistic, value, source, verified}]
|
|
```
|
|
|
|
### CriticReview
|
|
|
|
```
|
|
CriticReview
|
|
├── dimension_scores: {name: str, score: int, notes: str}[10]
|
|
├── fatal_flaws: [{description, severity, condition_to_fix}]
|
|
├── strengths: [str]
|
|
├── blind_spots: [str]
|
|
├── overall_verdict: enum[go, no_go, conditional_go]
|
|
└── conditions: [str]
|
|
```
|
|
|
|
### JudgeVerdict
|
|
|
|
```
|
|
JudgeVerdict
|
|
├── judge_id: str (Judge 1/2/3)
|
|
├── verdict: enum[go, no_go, conditional_go]
|
|
├── confidence: float (0.0-1.0)
|
|
├── agreements_with_critic: [str]
|
|
├── disagreements_with_critic: [str]
|
|
├── novel_insights: [str] ← findings no other judge caught
|
|
├── second_order_effects: [str]
|
|
└── conditions: [str]
|
|
```
|
|
|
|
---
|
|
|
|
## PDF Generation
|
|
|
|
The PDF pipeline at `/root/projects/verdicttank/`:
|
|
|
|
1. **HTML Template** (`pdf-template.html`) — 11-section branded report:
|
|
- Cover page with verdict badge
|
|
- Executive summary
|
|
- Dimension score table
|
|
- Research findings with citations
|
|
- Fatal flaws ranked by severity
|
|
- Priority-ranked action plan
|
|
- Judge cards (anonymized — "Judge 1/2/3", no model names)
|
|
- Judge notes section
|
|
- **Disclaimer** (Not Legal Advice, Not Financial Advice, AI-Generated Content Disclosure, No Warranty, Limitation of Liability, Confidentiality Notice)
|
|
- Citations
|
|
|
|
2. **Generator** (`generate-pdf.py`) — Reads structured JSON data, fills the template via WeasyPrint, outputs branded PDF
|
|
|
|
3. **Quality Gate** (post-generation):
|
|
```python
|
|
from pypdf import PdfReader
|
|
r = PdfReader('report.pdf')
|
|
text = ''.join(p.extract_text() or '' for p in r.pages)
|
|
assert '—' not in text, 'EM DASH FOUND'
|
|
assert 'dispatch' not in text.lower(), 'DISPATCH FOUND'
|
|
assert 'Disclaimer' in text, 'DISCLAIMER SECTION MISSING'
|
|
assert len(r.pages) >= 10, f'Expected 10+ pages, got {len(r.pages)}'
|
|
```
|
|
|
|
### Sanitization Rules (hard constraints)
|
|
|
|
| Rule | Enforcement |
|
|
|---|---|
|
|
| No model names anywhere in PDF | `build_judge_cards()` uses data `name` field — expects "Judge 1/2/3" |
|
|
| No internal methodology exposed | No "pipeline", "conductor", "Opus", "Sonnet", "Qwen", "Gemini" |
|
|
| No em dashes | `clean()` function strips `\u2014` and `\u2013` |
|
|
| 11-section report (was 10 before Aug 8) | Disclaimer is Section 10, Citations is Section 11 |
|
|
| Filename: `<Client>-VerdictTank-Report.pdf` | Safe-filename generator strips non-alphanumeric |
|
|
|
|
---
|
|
|
|
## Pricing Integration
|
|
|
|
| Tier | Price | Reviews/Month | Model Access |
|
|
|---|---|---|---|
|
|
| Free | $0 | 1 | Limited model set |
|
|
| Pro | $79/mo | 10 ($8 overage) | Full model set |
|
|
| Enterprise | $499/mo | 50 ($12 overage) | API access, prediction tracking |
|
|
| White-Label | $1,999+/mo | Unlimited | Multi-tenant, custom branding |
|
|
|
|
Cost per review: $0.70-0.75 (v3 corpus-backed). Turnaround: 5-15 minutes.
|
|
|
|
---
|
|
|
|
## Cost Model
|
|
|
|
### Per-Review Breakdown
|
|
|
|
| Phase | Tokens (est.) | Cost |
|
|
|---|---|---|
|
|
| Research Agent | ~5K in, ~3K out | ~$0.10 |
|
|
| Critic Agent | ~15K in, ~5K out | ~$0.25 |
|
|
| Judge A | ~20K in, ~3K out | ~$0.12 |
|
|
| Judge B | ~20K in, ~2K out | ~$0.05 |
|
|
| Judge C | ~20K in, ~2K out | ~$0.05 |
|
|
| PDF Generation | N/A | ~$0.01 |
|
|
| **Total** | | **~$0.58-0.75** |
|
|
|
|
At 10 reviews/month (Pro tier): ~$7.50 cost, $79 revenue = ~90% gross margin.
|
|
At scale (100+ reviews/month): corpus caching reduces research costs significantly.
|
|
|
|
---
|
|
|
|
## Security Model
|
|
|
|
### Attack Surface
|
|
|
|
| Surface | Risk | Mitigation |
|
|
|---|---|---|
|
|
| Prompt injection via proposal text | HIGH | Input sanitization, structured parsing, no raw tool execution from user input |
|
|
| Research agent web access | MEDIUM | URL allowlist, rate limiting, response size caps |
|
|
| PDF injection via WeasyPrint | LOW | No user-controlled HTML in template, all content sanitized |
|
|
| API key exposure | LOW | `verdicttank-prod` key scoped to admin-ai, usage caps |
|
|
| Report access | LOW | `/reports/` directory served by Caddy, no directory listing |
|
|
| Corpus poisoning | LOW | SQLite read-only for judge queries, write path gated |
|
|
|
|
### Input Sanitization Pipeline
|
|
|
|
```
|
|
Raw proposal submission
|
|
→ Strip control characters
|
|
→ Validate UTF-8
|
|
→ Size cap (100KB)
|
|
→ MIME type validation
|
|
→ Pass to Research Agent as structured data, not raw prompt
|
|
```
|
|
|
|
---
|
|
|
|
## Failure Modes & Recovery
|
|
|
|
| Failure | Impact | Recovery |
|
|
|---|---|---|
|
|
| Single judge timeout/crash | Degraded to 2 judges | Majority of 2 rules; note missing judge in addendum |
|
|
| All judges fail | No verdict produced | Free re-run; investigate admin-ai |
|
|
| Research Agent web failure | Incomplete citations | Flag as "limited research" in brief; proceed with available data |
|
|
| PDF generation failure | Report not delivered | Retry once; email plain-text summary on persistent failure |
|
|
| Core server outage | Platform unavailable | Warm standby (app1-bu) can take over Caddy + API |
|
|
| admin-ai outage | All phases blocked | Model failover chain (Flash → Gemini → Grok → Sonnet) |
|
|
| SQLite corruption | Corpus lost | Nightly backup to S3; rebuild from backup |
|
|
|
|
---
|
|
|
|
## Operations
|
|
|
|
### Monitoring
|
|
|
|
| Metric | Source | Alert Threshold |
|
|
|---|---|---|
|
|
| API response time | Prometheus (Core:9090) | >30s p95 |
|
|
| Review success rate | App-level counter | <90% over 1h |
|
|
| PDF generation failures | App-level counter | Any failure |
|
|
| admin-ai latency | LiteLLM metrics | >10s p95 |
|
|
| Disk usage (reports) | node_exporter | >80% |
|
|
|
|
### Backup
|
|
|
|
- **Corpus DB:** Nightly to S3 via `hermes-backup.sh`
|
|
- **PDF reports:** `/var/www/verdicttank/reports/` included in full backup
|
|
- **API config:** In git repo, pushed to Gitea
|
|
|
|
---
|
|
|
|
## Known Limitations
|
|
|
|
1. **Single-box architecture** — Core is a single VPS. If Core goes down, VerdictTank is down until app1-bu failover completes (~5-10 min). A dedicated VerdictTank instance would eliminate this dependency.
|
|
2. **No payment integration** — Currently manual billing. Stripe integration planned for Phase 2.
|
|
3. **No user accounts** — Each review is standalone. User dashboard and review history planned for Phase 2.
|
|
4. **Corpus freshness** — Benchmark data is static until manually refreshed. Automated crawl planned.
|
|
5. **PDF email delivery** — Via MXroute SMTP, no delivery tracking. Postmark/SendGrid integration planned.
|
|
|
|
---
|
|
|
|
## Roadmap
|
|
|
|
| Phase | Deliverable | Status |
|
|
|---|---|---|
|
|
| Phase 0 | PDF pipeline, corpus DB, basic API | **DONE** |
|
|
| Phase 1 | Outcome tracking, accuracy engine, corpus refresh | Proposed |
|
|
| Phase 2 | Stripe integration, user dashboard, API access | Proposed |
|
|
| Phase 3 | Multi-tenant, white-label, SSO | Proposed |
|
|
|
|
---
|
|
|
|
## Deployment
|
|
|
|
```
|
|
# Deploy site
|
|
scp -i /root/.ssh/itpp-infra /tmp/verdicttank-architecture.md \
|
|
root@152.53.192.33:/root/itpp-docs/docs-source/verdicttank/architecture.md
|
|
|
|
# Build and deploy
|
|
ssh core "cd /root/itpp-docs && mkdocs build --clean && \
|
|
tar czf - site/" | ssh app3 "tar xzf - --strip-components=1 \
|
|
-C /home/ippadmin/htdocs/docs.itpropartner.com/"
|
|
```
|
|
|
|
---
|
|
|
|
*Last updated: August 10, 2026*
|
|
*Pipeline verdict: 3/3 Unanimous Conditional Go*
|
|
*Commit: see git.itpropartner.com/ippadmin/itpp-docs*
|