diff --git a/docs/super-search-cf-bypass.md b/docs/super-search-cf-bypass.md new file mode 100644 index 0000000..d3a1571 --- /dev/null +++ b/docs/super-search-cf-bypass.md @@ -0,0 +1,37 @@ +# Super Search — Cloudflare Bypass + +**Added:** July 21, 2026 +**Version:** 2.1.0 + +## Extraction Chain + +| Tier | Provider | What it handles | Fallback Trigger | +|---|---|---|---| +| 1 | Trafilatura | Normal sites | Error OR CF challenge detected | +| 2 | Browserless Chrome | CF-protected sites | Chrome render + Trafilatura | +| 3 | Firecrawl | Everything else | API-based extraction | + +## CF Detection +Nine detection markers for caught challenge pages (from Hound's code + additional): +- cf-turnstile, challenges.cloudflare.com/turnstile +- cf_chl_opt, __cf_chl +- cf-browser-verification, challenge-platform, cf-mitigated +- "Checking your browser", "Just a moment" + +## Infrastructure +- Browserless Chrome on app1 (152.53.36.131), port 3005 +- Caddy proxy: app1:3006 → localhost:3005 +- Firewall: only Core (152.53.192.33) can reach port 3006 +- Super Search: `/root/docker/super-search/server.py` + +## Verify +```bash +# Test CF bypass +cd /root/docker/super-search && source venv/bin/activate +python3 -c " +from server import _extract_one +import asyncio +r = asyncio.run(_extract_one('https://nowsecure.nl')) +print(r['provider']) # Should be 'trafilatura' or 'browserless' +" +```