Sync docs, audit artifacts, project notes, and VerdictTank proposal docs
- audit/phase-one + phase-two: security audit briefs, findings, credential-rotation plan, Docker-USER hardening scripts, rollback refs - disaster-recovery/restore-test-log.md + backup-dr-audit-2026-08-10.md - clients/ (modelortho SEO audit, ai-biz-dev competitive landscape), notes/ (tiktok strategy) - projects/: front-desk-voice-agent, seo-visibility-checker product plan, hotnow-savannah HTML, resend-transactional-email, backup-dashboard-enhancements, code-review-graph, seo-ci-architecture - proposals/verdicttank/: architecture v4.0, methodology, judge-pool review, consolidation reasoning, cross-check review - docs/super-search/firecrawl-provider-strategy.md - updates: CHANGELOG, model-chain, projects-master-readme, intelsight.io - .gitignore: exclude nested standalone repos (seo-tool, venturebuilt)
This commit is contained in:
root
@@ -0,0 +1,250 @@
|
||||
# SEO Tool + IntelSight — Shared Architecture
|
||||
|
||||
**Status:** OPEN
|
||||
**Date:** 2026-08-10
|
||||
**Products:** SEO Audit Tool (`git.itpropartner.com/ippadmin/seo-tool`), IntelSight (`intelsight.io`)
|
||||
|
||||
---
|
||||
|
||||
## 1. Core Principle
|
||||
|
||||
Two products. One infrastructure layer. Zero product merge.
|
||||
|
||||
| | SEO Audit Tool | IntelSight |
|
||||
|---|---|---|
|
||||
| **Buyer** | ITPP hosting client (small business owner) | Marketing VP, Founder, Strategy lead |
|
||||
| **Question** | "Why isn't my site getting traffic?" | "What are my competitors doing?" |
|
||||
| **Pricing** | Free / $49/mo Pro | $199–$1,499/mo |
|
||||
| **Current state** | Proposal (repo created Aug 10) | Proposal (July 25, 2026) |
|
||||
|
||||
They share the crawl engine, the audit checker library, the report renderer, and the Super Search backend. They sell to different people, at different price points, through different channels.
|
||||
|
||||
---
|
||||
|
||||
## 2. Shared Infrastructure Layer
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ SHARED INFRASTRUCTURE │
|
||||
│ │
|
||||
│ ┌──────────┐ ┌───────────┐ ┌──────────┐ ┌────────┐ │
|
||||
│ │ Crawler │ │ Audit │ │ Report │ │ Super │ │
|
||||
│ │ Engine │ │ Checkers │ │ Renderer │ │ Search │ │
|
||||
│ │ (Python/ │ │ (50+ rules│ │ (PDF/HTML│ │ (7 │ │
|
||||
│ │ requests)│ │ library) │ │ /email) │ │providers│ │
|
||||
│ └────┬─────┘ └─────┬─────┘ └────┬─────┘ └───┬────┘ │
|
||||
│ │ │ │ │ │
|
||||
│ ┌────┴──────────────┴─────────────┴────────────┴───┐ │
|
||||
│ │ API Gateway (FastAPI) │ │
|
||||
│ └──────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ ┌──────────────────┐ ┌──────────────────┐ │
|
||||
│ │ IntelSight DB │ │ SEO Audit DB │ │
|
||||
│ │ (competitor │ │ (site audits, │ │
|
||||
│ │ profiles, │ │ scores, trends,│ │
|
||||
│ │ alerts, CI) │ │ client sites) │ │
|
||||
│ └──────────────────┘ └──────────────────┘ │
|
||||
└─────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### Crawler Engine
|
||||
|
||||
- HTTP crawler that fetches and parses any URL
|
||||
- Extracts: HTML structure, meta tags, headings, links, images, structured data, performance metrics
|
||||
- Used by both products — SEO tool crawls the client's own site; IntelSight crawls competitor sites
|
||||
- Configurable depth, rate limiting, politeness
|
||||
|
||||
### Audit Checker Library
|
||||
|
||||
- 50+ reusable check functions: `check_meta_description()`, `check_og_tags()`, `check_canonical()`, `check_structured_data()`, `check_heading_hierarchy()`, `check_image_alts()`, `check_internal_links()`, `check_llms_txt()`
|
||||
- Each check returns: `{ status: pass|fail|warn, score: 0-100, fix: "<html snippet>", impact: high|medium|low, effort: low|medium|high }`
|
||||
- Used by both products — SEO tool runs all checks on the client's site; IntelSight runs summary checks on competitor sites
|
||||
|
||||
### Report Renderer
|
||||
|
||||
- Takes audit results and produces: HTML dashboard, PDF report, email summary
|
||||
- SEO tool renders: per-page issues, fix-it snippets, trend charts
|
||||
- IntelSight renders: competitive SEO comparison, AI visibility matrix, CI + SEO combined report
|
||||
|
||||
### Super Search Backend
|
||||
|
||||
- Already production-hardened: 7 providers, circuit breakers, caching, port 8899
|
||||
- Powers IntelSight's competitive intelligence queries
|
||||
- Powers SEO tool's AI engine visibility checks (GEO/AEO)
|
||||
|
||||
---
|
||||
|
||||
## 3. Product-Specific Architecture
|
||||
|
||||
### SEO Audit Tool
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────┐
|
||||
│ SEO AUDIT TOOL │
|
||||
│ │
|
||||
│ Launch Audit Monthly Crawl │
|
||||
│ (on deploy) (cron, 30 days) │
|
||||
│ │ │ │
|
||||
│ └──────┬───────┘ │
|
||||
│ │ │
|
||||
│ ┌────────┴────────┐ │
|
||||
│ │ Per-Site Audit │ │
|
||||
│ │ Dashboard │ │
|
||||
│ │ seo.itpropartner│ │
|
||||
│ │ .com/{site} │ │
|
||||
│ └────────┬────────┘ │
|
||||
│ │ │
|
||||
│ ┌───────────┴───────────┐ │
|
||||
│ │ Fix-It Snippets │ │
|
||||
│ │ (copy-paste HTML │ │
|
||||
│ │ per issue, per page)│ │
|
||||
│ └───────────────────────┘ │
|
||||
│ │
|
||||
│ AI Fix Prompts AI Visibility │
|
||||
│ (Claude/Cursor/ (GEO/AEO check │
|
||||
│ Codex ready-made via Super Search)│
|
||||
│ fix instructions) │
|
||||
└─────────────────────────────────────┘
|
||||
```
|
||||
|
||||
**Feature Enhancements (from SeoLoupe analysis):**
|
||||
|
||||
| Enhancement | Effort | Value | Description |
|
||||
|---|---|---|---|
|
||||
| **AI Fix Prompt Generation** | Low | High | Each finding generates a ready-made prompt for Claude/Cursor/Codex — "Fix the meta description on /pricing. Current: '...' Suggested: '...'" |
|
||||
| **llms.txt Checking** | Low | Medium | Check if llms.txt exists and is well-formed. Flag missing or malformed. Zero-cost add to crawl phase |
|
||||
| **GEO/AEO Visibility Scan** | Medium | High | Query Super Search providers: "Is {domain} cited in AI search results?" Surface how the site appears in ChatGPT, Gemini, Claude, Perplexity |
|
||||
| **Impact x Effort Matrix** | Low | Medium | Formalize priority ranking as a 2x2 matrix (not just P0/P1/P2 list). Makes reports more actionable |
|
||||
| **<10s Scan Speed Target** | Medium | High | Automate end-to-end pipeline for instant feedback. Dopamine hit = conversion |
|
||||
| **MCP Server (Pro tier)** | Medium | High | Expose audit as MCP tools so external AI agents can trigger scans. First-mover in the agent ecosystem |
|
||||
|
||||
### IntelSight
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────┐
|
||||
│ INTELSIGHT │
|
||||
│ │
|
||||
│ Competitor Market Analysis │
|
||||
│ Profiles (funding, moves, │
|
||||
│ (Crunchbase, team changes, │
|
||||
│ Hunter.io, product launches) │
|
||||
│ OSINT) │
|
||||
│ │ │ │
|
||||
│ └──────┬───────┘ │
|
||||
│ │ │
|
||||
│ ┌────────┴────────┐ │
|
||||
│ │ CI Dashboard │ │
|
||||
│ │ intelsight.io │ │
|
||||
│ │ /dashboard │ │
|
||||
│ └────────┬────────┘ │
|
||||
│ │ │
|
||||
│ ┌───────────┴───────────┐ │
|
||||
│ │ Competitive SEO Scan │ ← NEW │
|
||||
│ │ (competitor site │ │
|
||||
│ │ health vs yours) │ │
|
||||
│ └───────────────────────┘ │
|
||||
│ │
|
||||
│ Competitor AI Alert Engine │
|
||||
│ Visibility (funding, team, │
|
||||
│ (GEO matrix) product changes) │
|
||||
└─────────────────────────────────────┘
|
||||
```
|
||||
|
||||
**Feature Enhancements:**
|
||||
|
||||
| Enhancement | Effort | Value | Description |
|
||||
|---|---|---|---|
|
||||
| **Competitive SEO Health Scan** | Medium | High | Crawl top 5 competitors and surface their SEO health side-by-side with yours. "Their meta tags are complete. Yours are missing on 12 pages." Uses the shared audit checker library |
|
||||
| **Competitor AI Visibility Matrix** | Medium | High | How visible are your competitors in ChatGPT, Gemini, Claude, Perplexity, Grok? Which competitors get cited? What queries? The GEO/AEO scan but applied to competitors |
|
||||
| **Combined CI + SEO Report** | Medium | Very High | One report per competitor: funding moves, team changes, product launches, AND their SEO health. "They raised $5M, hired a CMO, and their SEO score dropped 15 points — they're distracted, your window is now" |
|
||||
| **llms.txt Competitive Analysis** | Low | Low | Flag competitors who have/don't have llms.txt. Trend indicator — who's thinking about AI crawlers? |
|
||||
|
||||
---
|
||||
|
||||
## 4. Cross-Product Data Flow
|
||||
|
||||
```
|
||||
SEO Audit Tool IntelSight
|
||||
────────────── ──────────
|
||||
Client site crawl ──────────────→ Competitive SEO baseline
|
||||
(anonymous, aggregated)
|
||||
|
||||
Shared Crawler
|
||||
←────────────── Competitor site crawl
|
||||
|
||||
Client SEO scores ──────────────→ Industry benchmarks
|
||||
(aggregated, anon) "Average SMB scores 62/100.
|
||||
Your clients average 78/100."
|
||||
|
||||
←─────────────── IntelSight alerts
|
||||
"Competitor X launched new site.
|
||||
Run audit comparison?"
|
||||
```
|
||||
|
||||
Key rule: **client data stays in the SEO tool. Competitor data stays in IntelSight.** The shared layer only moves aggregated, anonymous benchmarks between them. No client PII crosses the boundary.
|
||||
|
||||
---
|
||||
|
||||
## 5. Implementation Sequence
|
||||
|
||||
### Phase 1: Foundation (Weeks 1-2)
|
||||
- Build the shared Crawler Engine (Python/requests, configurable depth, politeness)
|
||||
- Build the Audit Checker Library (50+ check functions)
|
||||
- Stand up shared API Gateway (FastAPI)
|
||||
- Separate databases per product
|
||||
|
||||
### Phase 2: SEO Tool Launch (Weeks 3-4)
|
||||
- Launch Audit: auto-runs on CloudPanel deploy webhook
|
||||
- Fix-It Snippets: HTML output per finding
|
||||
- Monthly crawl cron
|
||||
- AI Fix Prompt Generation (low effort, high impact)
|
||||
- llms.txt checking
|
||||
- Impact x Effort matrix in reports
|
||||
- Dashboard at `seo.itpropartner.com`
|
||||
|
||||
### Phase 3: IntelSight Enhancements (Weeks 5-8)
|
||||
- Competitive SEO Health Scan (uses shared audit checker)
|
||||
- Competitor AI Visibility Matrix (uses Super Search GEO queries)
|
||||
- Combined CI + SEO report template
|
||||
- MCP Server for external AI agent access
|
||||
|
||||
### Phase 4: Cross-Product (Weeks 9-12)
|
||||
- Anonymous industry benchmarks from SEO tool data
|
||||
- IntelSight alerts triggering SEO comparisons
|
||||
- White-label dashboard for agency clients
|
||||
- <10s scan speed optimization
|
||||
|
||||
---
|
||||
|
||||
## 6. What Stays Separate
|
||||
|
||||
| Layer | Separate? | Why |
|
||||
|---|---|---|
|
||||
| **Databases** | Separate | Client site data vs competitor intelligence — different access patterns, retention policies, privacy considerations |
|
||||
| **Auth / Tenants** | Separate | SEO tool: ITPP hosting auth (Stack Auth, existing). IntelSight: standalone SaaS auth with Stripe billing |
|
||||
| **Pricing / Billing** | Separate | SEO: bundled with hosting or $49/mo add-on. IntelSight: $199-1,499/mo standalone |
|
||||
| **Branding** | Separate | SEO: ITPP-branded, "your hosting includes this." IntelSight: standalone brand, intelsight.io |
|
||||
| **Go-to-Market** | Separate | SEO: hosting upsell, retention play. IntelSight: SaaS marketing, Product Hunt, direct sales |
|
||||
|
||||
---
|
||||
|
||||
## 7. Key Decisions
|
||||
|
||||
| Decision | Status | Notes |
|
||||
|---|---|---|
|
||||
| Merge products into one platform? | **SETTLED: No** | Different buyers, different pricing, different GTM |
|
||||
| Share crawl engine? | **SETTLED: Yes** | Same underlying tech, no reason to build twice |
|
||||
| Share audit checker library? | **SETTLED: Yes** | Check functions are pure logic — zero product coupling |
|
||||
| Cross-product data sharing? | **OPEN** | Aggregated/anonymous only. Privacy boundary TBD |
|
||||
| SEO Tool MCP Server? | **OPEN** | Pro tier feature. Proximity to agent ecosystem is a differentiator |
|
||||
| IntelSight SEO report depth? | **OPEN** | Full audit (like SEO tool) or summary (top 20 checks)? Depends on crawl politeness and competitor detection risk |
|
||||
|
||||
---
|
||||
|
||||
## 8. Next Actions
|
||||
|
||||
- [ ] Build shared Crawler Engine (`seo-crawler` package)
|
||||
- [ ] Define Audit Checker Library spec (50+ checks, output format)
|
||||
- [ ] Add AI Fix Prompt Generation to seo-audit skill (immediate low-effort win)
|
||||
- [ ] Add llms.txt check to seo-audit skill crawl phase
|
||||
- [ ] Prototype GEO/AEO visibility query via Super Search
|
||||
Reference in New Issue
Block a user