251 lines
13 KiB
Markdown
251 lines
13 KiB
Markdown
# SEO Tool + IntelSight — Shared Architecture
|
||
|
||
**Status:** OPEN
|
||
**Date:** 2026-08-10
|
||
**Products:** SEO Audit Tool (`git.itpropartner.com/ippadmin/seo-tool`), IntelSight (`intelsight.io`)
|
||
|
||
---
|
||
|
||
## 1. Core Principle
|
||
|
||
Two products. One infrastructure layer. Zero product merge.
|
||
|
||
| | SEO Audit Tool | IntelSight |
|
||
|---|---|---|
|
||
| **Buyer** | ITPP hosting client (small business owner) | Marketing VP, Founder, Strategy lead |
|
||
| **Question** | "Why isn't my site getting traffic?" | "What are my competitors doing?" |
|
||
| **Pricing** | Free / $49/mo Pro | $199–$1,499/mo |
|
||
| **Current state** | Proposal (repo created Aug 10) | Proposal (July 25, 2026) |
|
||
|
||
They share the crawl engine, the audit checker library, the report renderer, and the Super Search backend. They sell to different people, at different price points, through different channels.
|
||
|
||
---
|
||
|
||
## 2. Shared Infrastructure Layer
|
||
|
||
```
|
||
┌─────────────────────────────────────────────────────────┐
|
||
│ SHARED INFRASTRUCTURE │
|
||
│ │
|
||
│ ┌──────────┐ ┌───────────┐ ┌──────────┐ ┌────────┐ │
|
||
│ │ Crawler │ │ Audit │ │ Report │ │ Super │ │
|
||
│ │ Engine │ │ Checkers │ │ Renderer │ │ Search │ │
|
||
│ │ (Python/ │ │ (50+ rules│ │ (PDF/HTML│ │ (7 │ │
|
||
│ │ requests)│ │ library) │ │ /email) │ │providers│ │
|
||
│ └────┬─────┘ └─────┬─────┘ └────┬─────┘ └───┬────┘ │
|
||
│ │ │ │ │ │
|
||
│ ┌────┴──────────────┴─────────────┴────────────┴───┐ │
|
||
│ │ API Gateway (FastAPI) │ │
|
||
│ └──────────────────────────────────────────────────┘ │
|
||
│ │
|
||
│ ┌──────────────────┐ ┌──────────────────┐ │
|
||
│ │ IntelSight DB │ │ SEO Audit DB │ │
|
||
│ │ (competitor │ │ (site audits, │ │
|
||
│ │ profiles, │ │ scores, trends,│ │
|
||
│ │ alerts, CI) │ │ client sites) │ │
|
||
│ └──────────────────┘ └──────────────────┘ │
|
||
└─────────────────────────────────────────────────────────┘
|
||
```
|
||
|
||
### Crawler Engine
|
||
|
||
- HTTP crawler that fetches and parses any URL
|
||
- Extracts: HTML structure, meta tags, headings, links, images, structured data, performance metrics
|
||
- Used by both products — SEO tool crawls the client's own site; IntelSight crawls competitor sites
|
||
- Configurable depth, rate limiting, politeness
|
||
|
||
### Audit Checker Library
|
||
|
||
- 50+ reusable check functions: `check_meta_description()`, `check_og_tags()`, `check_canonical()`, `check_structured_data()`, `check_heading_hierarchy()`, `check_image_alts()`, `check_internal_links()`, `check_llms_txt()`
|
||
- Each check returns: `{ status: pass|fail|warn, score: 0-100, fix: "<html snippet>", impact: high|medium|low, effort: low|medium|high }`
|
||
- Used by both products — SEO tool runs all checks on the client's site; IntelSight runs summary checks on competitor sites
|
||
|
||
### Report Renderer
|
||
|
||
- Takes audit results and produces: HTML dashboard, PDF report, email summary
|
||
- SEO tool renders: per-page issues, fix-it snippets, trend charts
|
||
- IntelSight renders: competitive SEO comparison, AI visibility matrix, CI + SEO combined report
|
||
|
||
### Super Search Backend
|
||
|
||
- Already production-hardened: 7 providers, circuit breakers, caching, port 8899
|
||
- Powers IntelSight's competitive intelligence queries
|
||
- Powers SEO tool's AI engine visibility checks (GEO/AEO)
|
||
|
||
---
|
||
|
||
## 3. Product-Specific Architecture
|
||
|
||
### SEO Audit Tool
|
||
|
||
```
|
||
┌─────────────────────────────────────┐
|
||
│ SEO AUDIT TOOL │
|
||
│ │
|
||
│ Launch Audit Monthly Crawl │
|
||
│ (on deploy) (cron, 30 days) │
|
||
│ │ │ │
|
||
│ └──────┬───────┘ │
|
||
│ │ │
|
||
│ ┌────────┴────────┐ │
|
||
│ │ Per-Site Audit │ │
|
||
│ │ Dashboard │ │
|
||
│ │ seo.itpropartner│ │
|
||
│ │ .com/{site} │ │
|
||
│ └────────┬────────┘ │
|
||
│ │ │
|
||
│ ┌───────────┴───────────┐ │
|
||
│ │ Fix-It Snippets │ │
|
||
│ │ (copy-paste HTML │ │
|
||
│ │ per issue, per page)│ │
|
||
│ └───────────────────────┘ │
|
||
│ │
|
||
│ AI Fix Prompts AI Visibility │
|
||
│ (Claude/Cursor/ (GEO/AEO check │
|
||
│ Codex ready-made via Super Search)│
|
||
│ fix instructions) │
|
||
└─────────────────────────────────────┘
|
||
```
|
||
|
||
**Feature Enhancements (from SeoLoupe analysis):**
|
||
|
||
| Enhancement | Effort | Value | Description |
|
||
|---|---|---|---|
|
||
| **AI Fix Prompt Generation** | Low | High | Each finding generates a ready-made prompt for Claude/Cursor/Codex — "Fix the meta description on /pricing. Current: '...' Suggested: '...'" |
|
||
| **llms.txt Checking** | Low | Medium | Check if llms.txt exists and is well-formed. Flag missing or malformed. Zero-cost add to crawl phase |
|
||
| **GEO/AEO Visibility Scan** | Medium | High | Query Super Search providers: "Is {domain} cited in AI search results?" Surface how the site appears in ChatGPT, Gemini, Claude, Perplexity |
|
||
| **Impact x Effort Matrix** | Low | Medium | Formalize priority ranking as a 2x2 matrix (not just P0/P1/P2 list). Makes reports more actionable |
|
||
| **<10s Scan Speed Target** | Medium | High | Automate end-to-end pipeline for instant feedback. Dopamine hit = conversion |
|
||
| **MCP Server (Pro tier)** | Medium | High | Expose audit as MCP tools so external AI agents can trigger scans. First-mover in the agent ecosystem |
|
||
|
||
### IntelSight
|
||
|
||
```
|
||
┌─────────────────────────────────────┐
|
||
│ INTELSIGHT │
|
||
│ │
|
||
│ Competitor Market Analysis │
|
||
│ Profiles (funding, moves, │
|
||
│ (Crunchbase, team changes, │
|
||
│ Hunter.io, product launches) │
|
||
│ OSINT) │
|
||
│ │ │ │
|
||
│ └──────┬───────┘ │
|
||
│ │ │
|
||
│ ┌────────┴────────┐ │
|
||
│ │ CI Dashboard │ │
|
||
│ │ intelsight.io │ │
|
||
│ │ /dashboard │ │
|
||
│ └────────┬────────┘ │
|
||
│ │ │
|
||
│ ┌───────────┴───────────┐ │
|
||
│ │ Competitive SEO Scan │ ← NEW │
|
||
│ │ (competitor site │ │
|
||
│ │ health vs yours) │ │
|
||
│ └───────────────────────┘ │
|
||
│ │
|
||
│ Competitor AI Alert Engine │
|
||
│ Visibility (funding, team, │
|
||
│ (GEO matrix) product changes) │
|
||
└─────────────────────────────────────┘
|
||
```
|
||
|
||
**Feature Enhancements:**
|
||
|
||
| Enhancement | Effort | Value | Description |
|
||
|---|---|---|---|
|
||
| **Competitive SEO Health Scan** | Medium | High | Crawl top 5 competitors and surface their SEO health side-by-side with yours. "Their meta tags are complete. Yours are missing on 12 pages." Uses the shared audit checker library |
|
||
| **Competitor AI Visibility Matrix** | Medium | High | How visible are your competitors in ChatGPT, Gemini, Claude, Perplexity, Grok? Which competitors get cited? What queries? The GEO/AEO scan but applied to competitors |
|
||
| **Combined CI + SEO Report** | Medium | Very High | One report per competitor: funding moves, team changes, product launches, AND their SEO health. "They raised $5M, hired a CMO, and their SEO score dropped 15 points — they're distracted, your window is now" |
|
||
| **llms.txt Competitive Analysis** | Low | Low | Flag competitors who have/don't have llms.txt. Trend indicator — who's thinking about AI crawlers? |
|
||
|
||
---
|
||
|
||
## 4. Cross-Product Data Flow
|
||
|
||
```
|
||
SEO Audit Tool IntelSight
|
||
────────────── ──────────
|
||
Client site crawl ──────────────→ Competitive SEO baseline
|
||
(anonymous, aggregated)
|
||
|
||
Shared Crawler
|
||
←────────────── Competitor site crawl
|
||
|
||
Client SEO scores ──────────────→ Industry benchmarks
|
||
(aggregated, anon) "Average SMB scores 62/100.
|
||
Your clients average 78/100."
|
||
|
||
←─────────────── IntelSight alerts
|
||
"Competitor X launched new site.
|
||
Run audit comparison?"
|
||
```
|
||
|
||
Key rule: **client data stays in the SEO tool. Competitor data stays in IntelSight.** The shared layer only moves aggregated, anonymous benchmarks between them. No client PII crosses the boundary.
|
||
|
||
---
|
||
|
||
## 5. Implementation Sequence
|
||
|
||
### Phase 1: Foundation (Weeks 1-2)
|
||
- Build the shared Crawler Engine (Python/requests, configurable depth, politeness)
|
||
- Build the Audit Checker Library (50+ check functions)
|
||
- Stand up shared API Gateway (FastAPI)
|
||
- Separate databases per product
|
||
|
||
### Phase 2: SEO Tool Launch (Weeks 3-4)
|
||
- Launch Audit: auto-runs on CloudPanel deploy webhook
|
||
- Fix-It Snippets: HTML output per finding
|
||
- Monthly crawl cron
|
||
- AI Fix Prompt Generation (low effort, high impact)
|
||
- llms.txt checking
|
||
- Impact x Effort matrix in reports
|
||
- Dashboard at `seo.itpropartner.com`
|
||
|
||
### Phase 3: IntelSight Enhancements (Weeks 5-8)
|
||
- Competitive SEO Health Scan (uses shared audit checker)
|
||
- Competitor AI Visibility Matrix (uses Super Search GEO queries)
|
||
- Combined CI + SEO report template
|
||
- MCP Server for external AI agent access
|
||
|
||
### Phase 4: Cross-Product (Weeks 9-12)
|
||
- Anonymous industry benchmarks from SEO tool data
|
||
- IntelSight alerts triggering SEO comparisons
|
||
- White-label dashboard for agency clients
|
||
- <10s scan speed optimization
|
||
|
||
---
|
||
|
||
## 6. What Stays Separate
|
||
|
||
| Layer | Separate? | Why |
|
||
|---|---|---|
|
||
| **Databases** | Separate | Client site data vs competitor intelligence — different access patterns, retention policies, privacy considerations |
|
||
| **Auth / Tenants** | Separate | SEO tool: ITPP hosting auth (Stack Auth, existing). IntelSight: standalone SaaS auth with Stripe billing |
|
||
| **Pricing / Billing** | Separate | SEO: bundled with hosting or $49/mo add-on. IntelSight: $199-1,499/mo standalone |
|
||
| **Branding** | Separate | SEO: ITPP-branded, "your hosting includes this." IntelSight: standalone brand, intelsight.io |
|
||
| **Go-to-Market** | Separate | SEO: hosting upsell, retention play. IntelSight: SaaS marketing, Product Hunt, direct sales |
|
||
|
||
---
|
||
|
||
## 7. Key Decisions
|
||
|
||
| Decision | Status | Notes |
|
||
|---|---|---|
|
||
| Merge products into one platform? | **SETTLED: No** | Different buyers, different pricing, different GTM |
|
||
| Share crawl engine? | **SETTLED: Yes** | Same underlying tech, no reason to build twice |
|
||
| Share audit checker library? | **SETTLED: Yes** | Check functions are pure logic — zero product coupling |
|
||
| Cross-product data sharing? | **OPEN** | Aggregated/anonymous only. Privacy boundary TBD |
|
||
| SEO Tool MCP Server? | **OPEN** | Pro tier feature. Proximity to agent ecosystem is a differentiator |
|
||
| IntelSight SEO report depth? | **OPEN** | Full audit (like SEO tool) or summary (top 20 checks)? Depends on crawl politeness and competitor detection risk |
|
||
|
||
---
|
||
|
||
## 8. Next Actions
|
||
|
||
- [ ] Build shared Crawler Engine (`seo-crawler` package)
|
||
- [ ] Define Audit Checker Library spec (50+ checks, output format)
|
||
- [ ] Add AI Fix Prompt Generation to seo-audit skill (immediate low-effort win)
|
||
- [ ] Add llms.txt check to seo-audit skill crawl phase
|
||
- [ ] Prototype GEO/AEO visibility query via Super Search
|