ippadmin/seo-tool
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
master
seo-tool/docs/architecture.md
T

13 KiB
Raw Permalink Blame History

SEO Tool + IntelSight — Shared Architecture

Status: OPEN Date: 2026-08-10 Products: SEO Audit Tool (git.itpropartner.com/ippadmin/seo-tool), IntelSight (intelsight.io)

1. Core Principle

Two products. One infrastructure layer. Zero product merge.

SEO Audit Tool IntelSight
Buyer ITPP hosting client (small business owner) Marketing VP, Founder, Strategy lead
Question "Why isn't my site getting traffic?" "What are my competitors doing?"
Pricing Free / $49/mo Pro $199–$1,499/mo
Current state Proposal (repo created Aug 10) Proposal (July 25, 2026)

They share the crawl engine, the audit checker library, the report renderer, and the Super Search backend. They sell to different people, at different price points, through different channels.

2. Shared Infrastructure Layer

┌─────────────────────────────────────────────────────────┐
│                 SHARED INFRASTRUCTURE                     │
│                                                          │
│  ┌──────────┐  ┌───────────┐  ┌──────────┐  ┌────────┐  │
│  │  Crawler  │  │  Audit    │  │  Report  │  │ Super  │  │
│  │  Engine   │  │  Checkers │  │ Renderer │  │ Search │  │
│  │ (Python/ │  │ (50+ rules│  │ (PDF/HTML│  │  (7    │  │
│  │ requests)│  │  library) │  │ /email)  │  │providers│  │
│  └────┬─────┘  └─────┬─────┘  └────┬─────┘  └───┬────┘  │
│       │              │             │            │        │
│  ┌────┴──────────────┴─────────────┴────────────┴───┐    │
│  │              API Gateway (FastAPI)                │    │
│  └──────────────────────────────────────────────────┘    │
│                                                          │
│  ┌──────────────────┐  ┌──────────────────┐              │
│  │  IntelSight DB   │  │   SEO Audit DB   │              │
│  │  (competitor     │  │  (site audits,   │              │
│  │   profiles,      │  │   scores, trends,│              │
│  │   alerts, CI)    │  │   client sites)  │              │
│  └──────────────────┘  └──────────────────┘              │
└─────────────────────────────────────────────────────────┘

Crawler Engine

  • HTTP crawler that fetches and parses any URL
  • Extracts: HTML structure, meta tags, headings, links, images, structured data, performance metrics
  • Used by both products — SEO tool crawls the client's own site; IntelSight crawls competitor sites
  • Configurable depth, rate limiting, politeness

Audit Checker Library

  • 50+ reusable check functions: check_meta_description(), check_og_tags(), check_canonical(), check_structured_data(), check_heading_hierarchy(), check_image_alts(), check_internal_links(), check_llms_txt()
  • Each check returns: { status: pass|fail|warn, score: 0-100, fix: "<html snippet>", impact: high|medium|low, effort: low|medium|high }
  • Used by both products — SEO tool runs all checks on the client's site; IntelSight runs summary checks on competitor sites

Report Renderer

  • Takes audit results and produces: HTML dashboard, PDF report, email summary
  • SEO tool renders: per-page issues, fix-it snippets, trend charts
  • IntelSight renders: competitive SEO comparison, AI visibility matrix, CI + SEO combined report

Super Search Backend

  • Already production-hardened: 7 providers, circuit breakers, caching, port 8899
  • Powers IntelSight's competitive intelligence queries
  • Powers SEO tool's AI engine visibility checks (GEO/AEO)

3. Product-Specific Architecture

SEO Audit Tool

┌─────────────────────────────────────┐
│         SEO AUDIT TOOL               │
│                                      │
│  Launch Audit    Monthly Crawl        │
│  (on deploy)     (cron, 30 days)      │
│       │              │                │
│       └──────┬───────┘                │
│              │                        │
│     ┌────────┴────────┐               │
│     │  Per-Site Audit  │               │
│     │  Dashboard       │               │
│     │  seo.itpropartner│               │
│     │  .com/{site}     │               │
│     └────────┬────────┘               │
│              │                        │
│  ┌───────────┴───────────┐            │
│  │  Fix-It Snippets      │            │
│  │  (copy-paste HTML     │            │
│  │   per issue, per page)│            │
│  └───────────────────────┘            │
│                                      │
│  AI Fix Prompts     AI Visibility     │
│  (Claude/Cursor/    (GEO/AEO check    │
│   Codex ready-made   via Super Search)│
│   fix instructions)                   │
└─────────────────────────────────────┘

Feature Enhancements (from SeoLoupe analysis):

Enhancement Effort Value Description
AI Fix Prompt Generation Low High Each finding generates a ready-made prompt for Claude/Cursor/Codex — "Fix the meta description on /pricing. Current: '...' Suggested: '...'"
llms.txt Checking Low Medium Check if llms.txt exists and is well-formed. Flag missing or malformed. Zero-cost add to crawl phase
GEO/AEO Visibility Scan Medium High Query Super Search providers: "Is {domain} cited in AI search results?" Surface how the site appears in ChatGPT, Gemini, Claude, Perplexity
Impact x Effort Matrix Low Medium Formalize priority ranking as a 2x2 matrix (not just P0/P1/P2 list). Makes reports more actionable
<10s Scan Speed Target Medium High Automate end-to-end pipeline for instant feedback. Dopamine hit = conversion
MCP Server (Pro tier) Medium High Expose audit as MCP tools so external AI agents can trigger scans. First-mover in the agent ecosystem

IntelSight

┌─────────────────────────────────────┐
│            INTELSIGHT                │
│                                      │
│  Competitor     Market Analysis       │
│  Profiles       (funding, moves,      │
│  (Crunchbase,   team changes,         │
│   Hunter.io,    product launches)     │
│   OSINT)                              │
│       │              │                │
│       └──────┬───────┘                │
│              │                        │
│     ┌────────┴────────┐               │
│     │  CI Dashboard    │               │
│     │  intelsight.io   │               │
│     │  /dashboard      │               │
│     └────────┬────────┘               │
│              │                        │
│  ┌───────────┴───────────┐            │
│  │  Competitive SEO Scan │  ← NEW     │
│  │  (competitor site     │            │
│  │   health vs yours)    │            │
│  └───────────────────────┘            │
│                                      │
│  Competitor AI      Alert Engine      │
│  Visibility         (funding, team,   │
│  (GEO matrix)       product changes)  │
└─────────────────────────────────────┘

Feature Enhancements:

Enhancement Effort Value Description
Competitive SEO Health Scan Medium High Crawl top 5 competitors and surface their SEO health side-by-side with yours. "Their meta tags are complete. Yours are missing on 12 pages." Uses the shared audit checker library
Competitor AI Visibility Matrix Medium High How visible are your competitors in ChatGPT, Gemini, Claude, Perplexity, Grok? Which competitors get cited? What queries? The GEO/AEO scan but applied to competitors
Combined CI + SEO Report Medium Very High One report per competitor: funding moves, team changes, product launches, AND their SEO health. "They raised $5M, hired a CMO, and their SEO score dropped 15 points — they're distracted, your window is now"
llms.txt Competitive Analysis Low Low Flag competitors who have/don't have llms.txt. Trend indicator — who's thinking about AI crawlers?

4. Cross-Product Data Flow

SEO Audit Tool                    IntelSight
──────────────                    ──────────
Client site crawl ──────────────→ Competitive SEO baseline
(anonymous, aggregated)

                    Shared Crawler
                    ←────────────── Competitor site crawl

Client SEO scores   ──────────────→ Industry benchmarks
(aggregated, anon)                  "Average SMB scores 62/100.
                                     Your clients average 78/100."

                    ←─────────────── IntelSight alerts
                                     "Competitor X launched new site.
                                      Run audit comparison?"

Key rule: client data stays in the SEO tool. Competitor data stays in IntelSight. The shared layer only moves aggregated, anonymous benchmarks between them. No client PII crosses the boundary.

5. Implementation Sequence

Phase 1: Foundation (Weeks 1-2)

  • Build the shared Crawler Engine (Python/requests, configurable depth, politeness)
  • Build the Audit Checker Library (50+ check functions)
  • Stand up shared API Gateway (FastAPI)
  • Separate databases per product

Phase 2: SEO Tool Launch (Weeks 3-4)

  • Launch Audit: auto-runs on CloudPanel deploy webhook
  • Fix-It Snippets: HTML output per finding
  • Monthly crawl cron
  • AI Fix Prompt Generation (low effort, high impact)
  • llms.txt checking
  • Impact x Effort matrix in reports
  • Dashboard at seo.itpropartner.com

Phase 3: IntelSight Enhancements (Weeks 5-8)

  • Competitive SEO Health Scan (uses shared audit checker)
  • Competitor AI Visibility Matrix (uses Super Search GEO queries)
  • Combined CI + SEO report template
  • MCP Server for external AI agent access

Phase 4: Cross-Product (Weeks 9-12)

  • Anonymous industry benchmarks from SEO tool data
  • IntelSight alerts triggering SEO comparisons
  • White-label dashboard for agency clients
  • <10s scan speed optimization

6. What Stays Separate

Layer Separate? Why
Databases Separate Client site data vs competitor intelligence — different access patterns, retention policies, privacy considerations
Auth / Tenants Separate SEO tool: ITPP hosting auth (Stack Auth, existing). IntelSight: standalone SaaS auth with Stripe billing
Pricing / Billing Separate SEO: bundled with hosting or $49/mo add-on. IntelSight: $199-1,499/mo standalone
Branding Separate SEO: ITPP-branded, "your hosting includes this." IntelSight: standalone brand, intelsight.io
Go-to-Market Separate SEO: hosting upsell, retention play. IntelSight: SaaS marketing, Product Hunt, direct sales

7. Key Decisions

Decision Status Notes
Merge products into one platform? SETTLED: No Different buyers, different pricing, different GTM
Share crawl engine? SETTLED: Yes Same underlying tech, no reason to build twice
Share audit checker library? SETTLED: Yes Check functions are pure logic — zero product coupling
Cross-product data sharing? OPEN Aggregated/anonymous only. Privacy boundary TBD
SEO Tool MCP Server? OPEN Pro tier feature. Proximity to agent ecosystem is a differentiator
IntelSight SEO report depth? OPEN Full audit (like SEO tool) or summary (top 20 checks)? Depends on crawl politeness and competitor detection risk

8. Next Actions

  • Build shared Crawler Engine (seo-crawler package)
  • Define Audit Checker Library spec (50+ checks, output format)
  • Add AI Fix Prompt Generation to seo-audit skill (immediate low-effort win)
  • Add llms.txt check to seo-audit skill crawl phase
  • Prototype GEO/AEO visibility query via Super Search
Reference in New Issue View Git Blame Copy Permalink