# VerdictTank v4.0 — Dogfood Verdict Report

**Date:** 2026-08-18
**Subject:** The v4.0 proposal itself, run through the v4.0 single-pass panel (10 seats: 9 scoring + synthesis gate)
**Verdict:** **CONDITIONAL**

---

## 1. Headline Scores

| Metric | Score |
|---|---|
| Proposal Strength | **71** |
| Investor Readiness | **44** |
| Composite | **58** |
| Divergence | 27 (proposal ahead) |
| Blocking findings | **Yes (6)** |

## 2. Dimension Means

| Dimension | Score |
|---|---|
| Structure & Communication | 93.8 |
| Internal Consistency | 86.2 |
| Solution Coherence | 84.9 |
| Problem Clarity | 81.2 |
| Risk & Compliance | 64.3 |
| Financial Integrity | 61.8 |
| Execution Feasibility | 50.0 |
| Team & Capability | 31.7 |
| Evidence Quality | 24.0 |
| Market Reality | 15.7 |

## 3. Panel Read

The panel's one-line take: **a well-designed critique loop that reads as an internal blueprint, not a funded product.** It scores 80+ on the four dimensions that measure writing quality, and collapses on the three that measure proof: market reality (15.7), evidence (24.0), team (31.7). Every margin claim, pricing scenario, and shipped-date assertion is unverified.

## 4. Six Blocking Conditions

1. **Market evidence** — bottom-up SAM, three named competitors with ARR/pricing, current paid account count by tier + trailing 30-day volume.
2. **Production proof** — anonymized logs: 500+ reviews processed, median latency <90s, uptime >99.5% over 30 days.
3. **Verified COGS** — vendor invoices/token logs from 100+ real reviews, reconciled to the $0.36 standard / $3.00 White-Label ceiling.
4. **Zero data-retention agreements** — executed with all nine vendors, or drop those vendors from the roster.
5. **Senior technical hire** — name a platform engineer/cofounder with multi-tenant SaaS track record.
6. **Arithmetic fix** — ceiling volume is 2,690 paid reviews, not 3,940 (a 1,250 gap), and the annual MRR reduction recomputes to $2,619, not $2,946.

---

## 5. Panel Integrity Findings (from running the panel itself)

### 5.1 glm-5.2 was not flaky — it was token-starved

**Root cause:** glm-5.2 is a reasoning model. On the financial seat (the heaviest brief), it consumed its entire token budget on hidden reasoning and returned zero visible tokens.

| Metric | Before (12k floor) | After (24k floor) |
|---|---|---|
| finish_reason | `length` | `stop` |
| reasoning_tokens | 11,998 | 14,635 |
| visible content | 0-3,015 chars | 8,545 chars |
| JSON parsed | no | yes (all 10 keys) |

**Conclusion:** glm-5.2 stays in the roster. The financial seat needs ~17k completion tokens (14.6k reasoning + 2k output). This was a budget bug, not a model defect.

### 5.2 Two production blockers found and fixed

1. **VerdictTank-Key allowlist was stale** (v3.5-era, 11 models). It 403'd on six v4.0 roster models: `sonar`, `gpt-5.2`, `glm-5.2`, `claude-haiku-4-5`, `xai/grok-4.3`, and bare `gemini-pro-latest` (it only knew `gemini/gemini-pro-latest`).
2. **Reasoning-model token floors too low** — deployed worker had glm-5.2 and deepseek-v4-pro at 8,000, which silently truncated reasoning models.

## 6. Deployment Fixes Applied

### Fix A — VerdictTank-Key allowlist (LiteLLM)
Added the 6 missing v4 models, keeping the existing 11 for backward compatibility. Total now 17 models.

### Fix B — MODEL_QUIRKS token floors (`/opt/verdicttank/worker.py`)
- `glm-5.2`: 8,000 → 24,000
- `deepseek-v4-pro`: 8,000 → 12,000
- `gemini-pro-latest`: added at 12,000
- `kimi-k2.6`: unchanged (already 16,000)

### Fix C — Robust JSON extraction (`/opt/verdicttank/worker.py`)
Replaced the fragile non-greedy regex + `rfind` parser (which silently dropped the financial seat) with brace-matching that handles nested objects, strings containing braces, and trailing commas in arrays. Verified against 6 test cases (fenced nested, fenced explicit, truncated, plain nested, trailing comma, empty) — all pass.

## 7. Remaining Work

- [ ] Re-run the full panel against the deployed worker to confirm end-to-end parity with the runner
- [ ] Decide whether glm-5.2 stays as permanent financial seat or whether the 24k floor is acceptable long-term cost
- [ ] Clear the six blocking conditions in §4 before GA
- [ ] Rotate the temp key `verdicttank-v4-test` (or delete it) once the production key is confirmed working