docs: recover verdicttank architecture doc — now in MkDocs source, no longer orphaned
Publish Docs Site / build (push) Failing after 4s
Publish Docs Site / build (push) Failing after 4s
This commit is contained in:
@@ -62,3 +62,9 @@
|
|||||||
|
|
||||||
- **Fixed:** "Tech Architecture" in Access table now links to `/verdicttank/architecture/` instead of the page itself
|
- **Fixed:** "Tech Architecture" in Access table now links to `/verdicttank/architecture/` instead of the page itself
|
||||||
- **Added:** Callout link to full architecture doc in the Architecture section
|
- **Added:** Callout link to full architecture doc in the Architecture section
|
||||||
|
|
||||||
|
## 2026-08-10 — Architecture doc recovered
|
||||||
|
|
||||||
|
- **Recovered:** Full architecture document at /verdicttank/architecture/ — rebuilt from pipeline spec after rsync --delete wipe
|
||||||
|
- **Added:** architecture.md is now a first-class MkDocs source file, not a manually deployed orphan
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,329 @@
|
|||||||
|
# VerdictTank — Technical Architecture
|
||||||
|
|
||||||
|
> **Version:** v1.0
|
||||||
|
> **Status:** 3/3 Unanimous Conditional Go (Aug 10, 2026)
|
||||||
|
> **Owner:** Germaine Brown
|
||||||
|
> **Built by:** Sho'Nuff (Hermes Agent)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
VerdictTank is an AI-powered business proposal review platform. A user submits a business proposal, product spec, or architecture document. Five specialized agents process it across three phases, three independent judges deliver a majority-rules verdict, and a branded PDF report lands in the user's inbox.
|
||||||
|
|
||||||
|
### Pipeline
|
||||||
|
|
||||||
|
```
|
||||||
|
User submits proposal
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
Phase 1: Research Agent
|
||||||
|
Live web verification — competitors, trademarks, domain WHOIS,
|
||||||
|
market data, pricing benchmarks. Citations attached to every claim.
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
Phase 2: Critic Agent
|
||||||
|
10-dimension review (Name, Pricing, PMF, Competition, Financials,
|
||||||
|
GTM, Risk, Missing Elements, Founder Fit, Overall Verdict).
|
||||||
|
Scored 1-10 per dimension.
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
Phase 3: Three judges in parallel (majority rules)
|
||||||
|
Judge A — Independent architecture
|
||||||
|
Judge B — Independent architecture
|
||||||
|
Judge C — Independent architecture
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
PDF verdict delivered by email
|
||||||
|
```
|
||||||
|
|
||||||
|
### Verdict System
|
||||||
|
|
||||||
|
| Result | Meaning |
|
||||||
|
|---|---|
|
||||||
|
| 3/3 Unanimous | Maximum confidence — all judges agree |
|
||||||
|
| 2/3 Majority | Majority verdict with dissenting opinion |
|
||||||
|
| 1-1-1 Split | Human (Germaine) breaks the tie |
|
||||||
|
| Degraded (2 judges) | One judge unavailable — runs with remaining two |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Component Architecture
|
||||||
|
|
||||||
|
### Server: Core (netcup RS 2000 — 152.53.192.33)
|
||||||
|
|
||||||
|
| Component | Technology | Port | Role |
|
||||||
|
|---|---|---|---|
|
||||||
|
| Caddy | Caddy 2 | 80/443 | Reverse proxy, TLS termination, static file serving |
|
||||||
|
| VerdictTank API | FastAPI (Python) | 8201 | Review orchestration, corpus queries, email dispatch |
|
||||||
|
| Corpus DB | SQLite | — | Proposal corpus, benchmark data, outcome tracking |
|
||||||
|
| PDF Generator | WeasyPrint | — | HTML-to-PDF rendering from `pdf-template.html` |
|
||||||
|
| Gitea Actions Runner | Gitea Act Runner | — | CI/CD for VerdictTank repo |
|
||||||
|
|
||||||
|
### AI Models (via admin-ai)
|
||||||
|
|
||||||
|
| Phase | Model | Role |
|
||||||
|
|---|---|---|
|
||||||
|
| Research | Worker model | Web search, claim verification |
|
||||||
|
| Critic | Reviewer model | 10-dimension critical review |
|
||||||
|
| Judge A | Independent model | Architecture A — validation |
|
||||||
|
| Judge B | Independent model | Architecture B — cross-check |
|
||||||
|
| Judge C | Independent model | Architecture C — operational realism |
|
||||||
|
|
||||||
|
All model calls route through `admin-ai.itpropartner.com` with the `verdicttank-prod` API key. Multi-provider architecture ensures no single vendor lock-in and genuine cross-model disagreement.
|
||||||
|
|
||||||
|
### Caddy Routing
|
||||||
|
|
||||||
|
```
|
||||||
|
verdicttank.com, www.verdicttank.com {
|
||||||
|
handle /api/verdicttank/* {
|
||||||
|
reverse_proxy 127.0.0.1:8201
|
||||||
|
}
|
||||||
|
handle /reports/* {
|
||||||
|
root * /var/www/verdicttank
|
||||||
|
file_server
|
||||||
|
}
|
||||||
|
redir /reports /reports/ permanent
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Crucial detail: use `handle` (not `handle_path`) for `/reports/*` — `handle_path` strips the prefix, breaking file resolution for `/reports/Client-Report.pdf`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Data Model
|
||||||
|
|
||||||
|
### Review Record
|
||||||
|
|
||||||
|
```
|
||||||
|
Review
|
||||||
|
├── id: UUID
|
||||||
|
├── client_name: str
|
||||||
|
├── proposal_text: str (full submitted document)
|
||||||
|
├── pricing_tier: enum[free, pro, enterprise, whitelabel]
|
||||||
|
├── status: enum[pending, research, critique, judging, done, failed]
|
||||||
|
├── created_at: datetime
|
||||||
|
├── completed_at: datetime?
|
||||||
|
│
|
||||||
|
├── research_brief: ResearchBrief?
|
||||||
|
├── critic_review: CriticReview?
|
||||||
|
├── judge_verdicts: [JudgeVerdict]
|
||||||
|
├── final_verdict: enum[go, no_go, conditional_go]?
|
||||||
|
├── pdf_path: str?
|
||||||
|
│
|
||||||
|
├── cost_breakdown: CostBreakdown
|
||||||
|
└── turnaround_seconds: int?
|
||||||
|
```
|
||||||
|
|
||||||
|
### ResearchBrief
|
||||||
|
|
||||||
|
```
|
||||||
|
ResearchBrief
|
||||||
|
├── verified_claims: [{claim, source_url, confidence}]
|
||||||
|
├── discrepancies: [{claimed, actual, severity, evidence}]
|
||||||
|
├── competitor_analysis: [{name, url, pricing, features, notes}]
|
||||||
|
├── domain_check: {domain, registered, expiry, registrar}
|
||||||
|
├── trademark_risk: enum[none, low, medium, high, blocked]
|
||||||
|
└── market_data: [{statistic, value, source, verified}]
|
||||||
|
```
|
||||||
|
|
||||||
|
### CriticReview
|
||||||
|
|
||||||
|
```
|
||||||
|
CriticReview
|
||||||
|
├── dimension_scores: {name: str, score: int, notes: str}[10]
|
||||||
|
├── fatal_flaws: [{description, severity, condition_to_fix}]
|
||||||
|
├── strengths: [str]
|
||||||
|
├── blind_spots: [str]
|
||||||
|
├── overall_verdict: enum[go, no_go, conditional_go]
|
||||||
|
└── conditions: [str]
|
||||||
|
```
|
||||||
|
|
||||||
|
### JudgeVerdict
|
||||||
|
|
||||||
|
```
|
||||||
|
JudgeVerdict
|
||||||
|
├── judge_id: str (Judge 1/2/3)
|
||||||
|
├── verdict: enum[go, no_go, conditional_go]
|
||||||
|
├── confidence: float (0.0-1.0)
|
||||||
|
├── agreements_with_critic: [str]
|
||||||
|
├── disagreements_with_critic: [str]
|
||||||
|
├── novel_insights: [str] ← findings no other judge caught
|
||||||
|
├── second_order_effects: [str]
|
||||||
|
└── conditions: [str]
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## PDF Generation
|
||||||
|
|
||||||
|
The PDF pipeline at `/root/projects/verdicttank/`:
|
||||||
|
|
||||||
|
1. **HTML Template** (`pdf-template.html`) — 11-section branded report:
|
||||||
|
- Cover page with verdict badge
|
||||||
|
- Executive summary
|
||||||
|
- Dimension score table
|
||||||
|
- Research findings with citations
|
||||||
|
- Fatal flaws ranked by severity
|
||||||
|
- Priority-ranked action plan
|
||||||
|
- Judge cards (anonymized — "Judge 1/2/3", no model names)
|
||||||
|
- Judge notes section
|
||||||
|
- **Disclaimer** (Not Legal Advice, Not Financial Advice, AI-Generated Content Disclosure, No Warranty, Limitation of Liability, Confidentiality Notice)
|
||||||
|
- Citations
|
||||||
|
|
||||||
|
2. **Generator** (`generate-pdf.py`) — Reads structured JSON data, fills the template via WeasyPrint, outputs branded PDF
|
||||||
|
|
||||||
|
3. **Quality Gate** (post-generation):
|
||||||
|
```python
|
||||||
|
from pypdf import PdfReader
|
||||||
|
r = PdfReader('report.pdf')
|
||||||
|
text = ''.join(p.extract_text() or '' for p in r.pages)
|
||||||
|
assert '—' not in text, 'EM DASH FOUND'
|
||||||
|
assert 'dispatch' not in text.lower(), 'DISPATCH FOUND'
|
||||||
|
assert 'Disclaimer' in text, 'DISCLAIMER SECTION MISSING'
|
||||||
|
assert len(r.pages) >= 10, f'Expected 10+ pages, got {len(r.pages)}'
|
||||||
|
```
|
||||||
|
|
||||||
|
### Sanitization Rules (hard constraints)
|
||||||
|
|
||||||
|
| Rule | Enforcement |
|
||||||
|
|---|---|
|
||||||
|
| No model names anywhere in PDF | `build_judge_cards()` uses data `name` field — expects "Judge 1/2/3" |
|
||||||
|
| No internal methodology exposed | No "pipeline", "conductor", "Opus", "Sonnet", "Qwen", "Gemini" |
|
||||||
|
| No em dashes | `clean()` function strips `\u2014` and `\u2013` |
|
||||||
|
| 11-section report (was 10 before Aug 8) | Disclaimer is Section 10, Citations is Section 11 |
|
||||||
|
| Filename: `<Client>-VerdictTank-Report.pdf` | Safe-filename generator strips non-alphanumeric |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Pricing Integration
|
||||||
|
|
||||||
|
| Tier | Price | Reviews/Month | Model Access |
|
||||||
|
|---|---|---|---|
|
||||||
|
| Free | $0 | 1 | Limited model set |
|
||||||
|
| Pro | $79/mo | 10 ($8 overage) | Full model set |
|
||||||
|
| Enterprise | $499/mo | 50 ($12 overage) | API access, prediction tracking |
|
||||||
|
| White-Label | $1,999+/mo | Unlimited | Multi-tenant, custom branding |
|
||||||
|
|
||||||
|
Cost per review: $0.70-0.75 (v3 corpus-backed). Turnaround: 5-15 minutes.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Cost Model
|
||||||
|
|
||||||
|
### Per-Review Breakdown
|
||||||
|
|
||||||
|
| Phase | Tokens (est.) | Cost |
|
||||||
|
|---|---|---|
|
||||||
|
| Research Agent | ~5K in, ~3K out | ~$0.10 |
|
||||||
|
| Critic Agent | ~15K in, ~5K out | ~$0.25 |
|
||||||
|
| Judge A | ~20K in, ~3K out | ~$0.12 |
|
||||||
|
| Judge B | ~20K in, ~2K out | ~$0.05 |
|
||||||
|
| Judge C | ~20K in, ~2K out | ~$0.05 |
|
||||||
|
| PDF Generation | N/A | ~$0.01 |
|
||||||
|
| **Total** | | **~$0.58-0.75** |
|
||||||
|
|
||||||
|
At 10 reviews/month (Pro tier): ~$7.50 cost, $79 revenue = ~90% gross margin.
|
||||||
|
At scale (100+ reviews/month): corpus caching reduces research costs significantly.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Security Model
|
||||||
|
|
||||||
|
### Attack Surface
|
||||||
|
|
||||||
|
| Surface | Risk | Mitigation |
|
||||||
|
|---|---|---|
|
||||||
|
| Prompt injection via proposal text | HIGH | Input sanitization, structured parsing, no raw tool execution from user input |
|
||||||
|
| Research agent web access | MEDIUM | URL allowlist, rate limiting, response size caps |
|
||||||
|
| PDF injection via WeasyPrint | LOW | No user-controlled HTML in template, all content sanitized |
|
||||||
|
| API key exposure | LOW | `verdicttank-prod` key scoped to admin-ai, usage caps |
|
||||||
|
| Report access | LOW | `/reports/` directory served by Caddy, no directory listing |
|
||||||
|
| Corpus poisoning | LOW | SQLite read-only for judge queries, write path gated |
|
||||||
|
|
||||||
|
### Input Sanitization Pipeline
|
||||||
|
|
||||||
|
```
|
||||||
|
Raw proposal submission
|
||||||
|
→ Strip control characters
|
||||||
|
→ Validate UTF-8
|
||||||
|
→ Size cap (100KB)
|
||||||
|
→ MIME type validation
|
||||||
|
→ Pass to Research Agent as structured data, not raw prompt
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Failure Modes & Recovery
|
||||||
|
|
||||||
|
| Failure | Impact | Recovery |
|
||||||
|
|---|---|---|
|
||||||
|
| Single judge timeout/crash | Degraded to 2 judges | Majority of 2 rules; note missing judge in addendum |
|
||||||
|
| All judges fail | No verdict produced | Free re-run; investigate admin-ai |
|
||||||
|
| Research Agent web failure | Incomplete citations | Flag as "limited research" in brief; proceed with available data |
|
||||||
|
| PDF generation failure | Report not delivered | Retry once; email plain-text summary on persistent failure |
|
||||||
|
| Core server outage | Platform unavailable | Warm standby (app1-bu) can take over Caddy + API |
|
||||||
|
| admin-ai outage | All phases blocked | Model failover chain (Flash → Gemini → Grok → Sonnet) |
|
||||||
|
| SQLite corruption | Corpus lost | Nightly backup to S3; rebuild from backup |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Operations
|
||||||
|
|
||||||
|
### Monitoring
|
||||||
|
|
||||||
|
| Metric | Source | Alert Threshold |
|
||||||
|
|---|---|---|
|
||||||
|
| API response time | Prometheus (Core:9090) | >30s p95 |
|
||||||
|
| Review success rate | App-level counter | <90% over 1h |
|
||||||
|
| PDF generation failures | App-level counter | Any failure |
|
||||||
|
| admin-ai latency | LiteLLM metrics | >10s p95 |
|
||||||
|
| Disk usage (reports) | node_exporter | >80% |
|
||||||
|
|
||||||
|
### Backup
|
||||||
|
|
||||||
|
- **Corpus DB:** Nightly to S3 via `hermes-backup.sh`
|
||||||
|
- **PDF reports:** `/var/www/verdicttank/reports/` included in full backup
|
||||||
|
- **API config:** In git repo, pushed to Gitea
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Known Limitations
|
||||||
|
|
||||||
|
1. **Single-box architecture** — Core is a single VPS. If Core goes down, VerdictTank is down until app1-bu failover completes (~5-10 min). A dedicated VerdictTank instance would eliminate this dependency.
|
||||||
|
2. **No payment integration** — Currently manual billing. Stripe integration planned for Phase 2.
|
||||||
|
3. **No user accounts** — Each review is standalone. User dashboard and review history planned for Phase 2.
|
||||||
|
4. **Corpus freshness** — Benchmark data is static until manually refreshed. Automated crawl planned.
|
||||||
|
5. **PDF email delivery** — Via MXroute SMTP, no delivery tracking. Postmark/SendGrid integration planned.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Roadmap
|
||||||
|
|
||||||
|
| Phase | Deliverable | Status |
|
||||||
|
|---|---|---|
|
||||||
|
| Phase 0 | PDF pipeline, corpus DB, basic API | **DONE** |
|
||||||
|
| Phase 1 | Outcome tracking, accuracy engine, corpus refresh | Proposed |
|
||||||
|
| Phase 2 | Stripe integration, user dashboard, API access | Proposed |
|
||||||
|
| Phase 3 | Multi-tenant, white-label, SSO | Proposed |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Deployment
|
||||||
|
|
||||||
|
```
|
||||||
|
# Deploy site
|
||||||
|
scp -i /root/.ssh/itpp-infra /tmp/verdicttank-architecture.md \
|
||||||
|
root@152.53.192.33:/root/itpp-docs/docs-source/verdicttank/architecture.md
|
||||||
|
|
||||||
|
# Build and deploy
|
||||||
|
ssh core "cd /root/itpp-docs && mkdocs build --clean && \
|
||||||
|
tar czf - site/" | ssh app3 "tar xzf - --strip-components=1 \
|
||||||
|
-C /home/ippadmin/htdocs/docs.itpropartner.com/"
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
*Last updated: August 10, 2026*
|
||||||
|
*Pipeline verdict: 3/3 Unanimous Conditional Go*
|
||||||
|
*Commit: see git.itpropartner.com/ippadmin/itpp-docs*
|
||||||
+35
-37
@@ -1,9 +1,8 @@
|
|||||||
site_name: "IT Pro Partner Docs"
|
site_name: IT Pro Partner Docs
|
||||||
site_url: "https://docs.itpropartner.com/"
|
site_url: https://docs.itpropartner.com/
|
||||||
repo_url: "https://git.itpropartner.com/ippadmin/itpp-docs"
|
repo_url: https://git.itpropartner.com/ippadmin/itpp-docs
|
||||||
edit_uri: edit/main/docs-source/
|
edit_uri: edit/main/docs-source/
|
||||||
docs_dir: docs-source
|
docs_dir: docs-source
|
||||||
|
|
||||||
theme:
|
theme:
|
||||||
name: material
|
name: material
|
||||||
palette:
|
palette:
|
||||||
@@ -11,39 +10,38 @@ theme:
|
|||||||
primary: indigo
|
primary: indigo
|
||||||
accent: indigo
|
accent: indigo
|
||||||
features:
|
features:
|
||||||
- navigation.instant
|
- navigation.instant
|
||||||
- navigation.tracking
|
- navigation.tracking
|
||||||
- navigation.tabs
|
- navigation.tabs
|
||||||
- navigation.sections
|
- navigation.sections
|
||||||
- search.highlight
|
- search.highlight
|
||||||
- search.share
|
- search.share
|
||||||
|
|
||||||
plugins:
|
plugins:
|
||||||
- search
|
- search
|
||||||
|
|
||||||
markdown_extensions:
|
markdown_extensions:
|
||||||
- admonition
|
- admonition
|
||||||
- pymdownx.details
|
- pymdownx.details
|
||||||
- pymdownx.superfences
|
- pymdownx.superfences
|
||||||
- pymdownx.highlight
|
- pymdownx.highlight
|
||||||
- tables
|
- tables
|
||||||
- toc:
|
- toc:
|
||||||
permalink: true
|
permalink: true
|
||||||
|
|
||||||
nav:
|
nav:
|
||||||
- Home: index.md
|
- Home: index.md
|
||||||
- Projects:
|
- Projects:
|
||||||
- ITPP Infrastructure:
|
- ITPP Infrastructure:
|
||||||
- Overview: itpp-infrastructure/index.md
|
- Overview: itpp-infrastructure/index.md
|
||||||
- Docs Auth Gate: itpp-infrastructure/docs-auth-gate.md
|
- Docs Auth Gate: itpp-infrastructure/docs-auth-gate.md
|
||||||
- ITPP Standards: itpp-standards/index.md
|
- ITPP Standards: itpp-standards/index.md
|
||||||
- TransitPin: transitpin/index.md
|
- TransitPin: transitpin/index.md
|
||||||
- HomeLab: homelab/index.md
|
- HomeLab: homelab/index.md
|
||||||
- Scripts: scripts/index.md
|
- Scripts: scripts/index.md
|
||||||
- FleetTracker360: fleettracker360/index.md
|
- FleetTracker360: fleettracker360/index.md
|
||||||
- Shark Game: shark-game/index.md
|
- Shark Game: shark-game/index.md
|
||||||
- VerdictTank: verdicttank/index.md
|
- VerdictTank:
|
||||||
- Apex Track: apex-track/index.md
|
- Overview: verdicttank/index.md
|
||||||
- BoxPilot: boxpilot/index.md
|
- Architecture: verdicttank/architecture.md
|
||||||
- OSINT Tool: osint-tool/index.md
|
- Apex Track: apex-track/index.md
|
||||||
- LaunchCheck: launchcheck/index.md
|
- BoxPilot: boxpilot/index.md
|
||||||
|
- OSINT Tool: osint-tool/index.md
|
||||||
|
- LaunchCheck: launchcheck/index.md
|
||||||
|
|||||||
Reference in New Issue
Block a user