Sync docs, audit artifacts, project notes, and VerdictTank proposal docs
- audit/phase-one + phase-two: security audit briefs, findings, credential-rotation plan, Docker-USER hardening scripts, rollback refs - disaster-recovery/restore-test-log.md + backup-dr-audit-2026-08-10.md - clients/ (modelortho SEO audit, ai-biz-dev competitive landscape), notes/ (tiktok strategy) - projects/: front-desk-voice-agent, seo-visibility-checker product plan, hotnow-savannah HTML, resend-transactional-email, backup-dashboard-enhancements, code-review-graph, seo-ci-architecture - proposals/verdicttank/: architecture v4.0, methodology, judge-pool review, consolidation reasoning, cross-check review - docs/super-search/firecrawl-provider-strategy.md - updates: CHANGELOG, model-chain, projects-master-readme, intelsight.io - .gitignore: exclude nested standalone repos (seo-tool, venturebuilt)
This commit is contained in:
root
@@ -0,0 +1,119 @@
|
||||
# SEO + AI Visibility Checker — Product Plan
|
||||
|
||||
**Status:** OPEN · Draft v1
|
||||
**Date:** 2026-08-23
|
||||
**Trigger:** Competitor review of tranx.io "Alice" (SEO + AI visibility check)
|
||||
|
||||
---
|
||||
|
||||
## 1. Why this exists
|
||||
|
||||
Two things changed that make a pure Google-only SEO scanner obsolete:
|
||||
|
||||
1. **AI crawlers are a second index.** ChatGPT, Perplexity, Claude, and Gemini answer questions by rendering and retrieving pages — and they each read `/llms.txt`, `robots.txt` (AI user-agents), and structured data differently from Googlebot. A site can rank #1 on Google and be invisible to every AI assistant.
|
||||
2. **AI-visibility is not measured by anyone cheap.** Screaming Frog / Ahrefs / Semrush measure Google. `llms.txt` and AI-crawler access are a blind spot. The only tools touching it (Alice, Profound, Peec) are either bare or lock it behind enterprise signup.
|
||||
|
||||
We already own the hard parts: Super Search (22 tools, web_extract, AI-answer probing) and a production-tested `seo-audit` skill with a browser_cdp methodology. This tool is productizing that skill + adding the AI-visibility layer nobody else has.
|
||||
|
||||
This follows the existing "obstacles as products" pattern: we needed to SEO-optimize our own 15 sites, so we build the checker first.
|
||||
|
||||
---
|
||||
|
||||
## 2. Product definition
|
||||
|
||||
**One-line:** A 30-second scan that answers "can Google *and* ChatGPT/Perplexity/Claude find, render, and cite this site?" — with a 0-100 score, severity-ranked issues, and copy-paste fixes.
|
||||
|
||||
**Core promise:** "Know how visible you are to search *and* AI — in 30 seconds, no signup."
|
||||
|
||||
### The five check families (parity with Alice, plus our edge)
|
||||
|
||||
| Family | Checks |
|
||||
|---|---|
|
||||
| **On-page SEO** | title, meta description, H1, canonical, Open Graph, structured data (JSON-LD), lang, viewport |
|
||||
| **Crawlability** | robots.txt (status + directive parse), sitemap.xml (status + URL count), HTTPS, redirects (www↔non-www, http→https) |
|
||||
| **Indexability** | brand SERP probe, indexed-page estimate (site: query), sitemap-vs-index gap |
|
||||
| **AI visibility** ← *our moat* | `/llms.txt` presence + parse, per-AI-crawler robots directives (GPTBot, ClaudeBot, PerplexityBot, Google-Extended), server-rendered vs JS-only content, schema markup richness, and a **live "how does ChatGPT/Perplexity see this brand" probe** via Super Search |
|
||||
| **Measurement** | GTM, GA4, Google Ads, Meta Pixel, Klaviyo detection + ID health |
|
||||
|
||||
### Output
|
||||
|
||||
- 0-100 score + one-line narrative
|
||||
- Issues ranked healthy / warning / error with per-issue fix snippets
|
||||
- Top-3 action list ("moves score most for least work")
|
||||
- Optional: connect Search Console / GA4 / Ads for query-level depth (Phase 2, paid)
|
||||
|
||||
---
|
||||
|
||||
## 3. Differentiation vs Alice (tranx.io)
|
||||
|
||||
| Axis | Alice | Ours |
|
||||
|---|---|---|
|
||||
| AI visibility | `/llms.txt` check only | `/llms.txt` + per-AI-crawler robots + **live AI-answer probe** (does ChatGPT actually cite this brand?) |
|
||||
| Depth | On-page + crawl + index + tags | Same + AI-answer retrieval testing + sitemap/index gap analysis |
|
||||
| Stack | Closed SaaS, credit-metered-ish upsell | Super Search backend, self-hosted, MCP-native |
|
||||
| Pricing | Freemium, enterprise upsell | Freemium scan + flat monthly (no per-lookup) |
|
||||
| Output | Score + issues + top-3 | Same + copy-paste fix snippets + export (JSON/PDF) |
|
||||
|
||||
**Moat:** the "does an AI actually cite you" probe. That's the question every founder will have in 2026 and nobody answers it in a free tool. It requires an LLM + search backend, which we already run.
|
||||
|
||||
---
|
||||
|
||||
## 4. Architecture
|
||||
|
||||
```
|
||||
[Web] seo-check.<tld> (single-file SPA, static)
|
||||
│ POST /scan {url}
|
||||
▼
|
||||
[FastAPI backend] :8088 (deploy like other python-web-service-deployment)
|
||||
├─ fetch + parse target (requests + BeautifulSoup + lxml)
|
||||
├─ robots/sitemap/redirect checks (httpx)
|
||||
├─ structured-data validation (JSON-LD parse)
|
||||
├─ llms.txt fetch + parse
|
||||
├─ per-AI-crawler robots analysis
|
||||
├─ JS-render probe (optional headless via existing Crawl4AI / browserless)
|
||||
└─ AI-answer probe → Super Search MCP (web_search + web_extract, brand+category query)
|
||||
▼
|
||||
[Super Search MCP] (already live, 22 tools)
|
||||
└─ "what does {brand} do" → check if brand appears in top AI/search results
|
||||
```
|
||||
|
||||
**Reuse:** Super Search (search + extract), Crawl4AI :8910 (JS render when needed), existing `seo-audit` skill logic as the scoring spec. No new infra — this is a thin FastAPI service on Core behind Caddy.
|
||||
|
||||
---
|
||||
|
||||
## 5. Build path (phased)
|
||||
|
||||
| Phase | Scope | Exit criterion |
|
||||
|---|---|---|
|
||||
| **P1 — MVP (free scan)** | On-page + crawl + index + AI-visibility (llms.txt + crawler robots + AI-answer probe) + measurement. Score + top-3 + snippets. | Live scan of our own 15 sites, all green |
|
||||
| **P2 — Auth + depth** | Central auth login, saved scans, GSC/GA4/Ads connect, query-level data | Paying first user |
|
||||
| **P3 — API + MCP** | REST API + MCP server (`seo-check` tool) so Hermes and other agents can run scans programmatically | MCP server live, documented |
|
||||
|
||||
**P1 is the same work as "SEO-optimize our 15 sites"** — build the checker, run it on ourselves, fix what it finds. The tool and the site-fix are one effort.
|
||||
|
||||
---
|
||||
|
||||
## 6. Pricing (value-based, premium — never undercut)
|
||||
|
||||
| Tier | Price | Includes |
|
||||
|---|---|---|
|
||||
| Free | $0 | 1-off scans, rate-limited per IP, no signup |
|
||||
| Pro | $29/mo | Unlimited scans, saved reports, scheduled re-scans, AI-answer monitoring alerts |
|
||||
| Agency | $99/mo | 50 domains, white-label PDF reports, API access |
|
||||
| API | usage | Per-scan API + MCP server for agent platforms |
|
||||
|
||||
Anchored against Alice (free) and Semrush/Ahrefs ($129+/mo). We undercut enterprise but never race to the bottom on the free tier — the free tier is a lead-gen funnel, not the product.
|
||||
|
||||
---
|
||||
|
||||
## 7. Open decisions (need Germaine)
|
||||
|
||||
- [ ] **Brand/domain:** own product name + domain, or a subdomain under an existing property? (Suggests: standalone `.io`/`.com` like the other micro-SaaS products — IntelSight, VerdictTank.)
|
||||
- [ ] **Scope of "AI-answer probe":** how deep — full retrieval comparison or a yes/no "brand cited" flag for v1?
|
||||
- [ ] **Ship target:** P1 as a free public tool first (lead-gen), or internal-only until our 15 sites are clean?
|
||||
|
||||
---
|
||||
|
||||
## 8. Immediate next step
|
||||
|
||||
The P1 build and the "optimize our 15 sites" ask are the same workstream. Sweep results are in `sites-seo-sweep-2026-08-23.md`. Fix the 9 failing sites first (proves the scoring model), then ship the checker as a product.
|
||||
Reference in New Issue
Block a user