38 lines
1.1 KiB
Markdown
38 lines
1.1 KiB
Markdown
# Super Search — Cloudflare Bypass
|
|
|
|
**Added:** July 21, 2026
|
|
**Version:** 2.1.0
|
|
|
|
## Extraction Chain
|
|
|
|
| Tier | Provider | What it handles | Fallback Trigger |
|
|
|---|---|---|---|
|
|
| 1 | Trafilatura | Normal sites | Error OR CF challenge detected |
|
|
| 2 | Browserless Chrome | CF-protected sites | Chrome render + Trafilatura |
|
|
| 3 | Firecrawl | Everything else | API-based extraction |
|
|
|
|
## CF Detection
|
|
Nine detection markers for caught challenge pages (from Hound's code + additional):
|
|
- cf-turnstile, challenges.cloudflare.com/turnstile
|
|
- cf_chl_opt, __cf_chl
|
|
- cf-browser-verification, challenge-platform, cf-mitigated
|
|
- "Checking your browser", "Just a moment"
|
|
|
|
## Infrastructure
|
|
- Browserless Chrome on app1 (152.53.36.131), port 3005
|
|
- Caddy proxy: app1:3006 → localhost:3005
|
|
- Firewall: only Core (152.53.192.33) can reach port 3006
|
|
- Super Search: `/root/docker/super-search/server.py`
|
|
|
|
## Verify
|
|
```bash
|
|
# Test CF bypass
|
|
cd /root/docker/super-search && source venv/bin/activate
|
|
python3 -c "
|
|
from server import _extract_one
|
|
import asyncio
|
|
r = asyncio.run(_extract_one('https://nowsecure.nl'))
|
|
print(r['provider']) # Should be 'trafilatura' or 'browserless'
|
|
"
|
|
```
|