2.2 KiB
2.2 KiB
Web Research Tools Stack
Configured Jul 7, 2026 for IT Pro Partner infrastructure.
Active Tools
| Tool | Type | Cost | Credits | Used for |
|---|---|---|---|---|
| Firecrawl | Web search + content extraction | Free tier | 1,000/mo | Quick web lookups, web_search and web_extract tools |
| ScrapingAnt | JS rendering via headless Chrome | Free tier | 10,000 free (no card) | JS-heavy sites, government pages, Texas Legislature, LinkedIn public pages |
| SearXNG | Self-hosted meta search (Docker) | $0 (runs on Core) | Unlimited | Private search, fallback when other tools unavailable |
Configuration
Firecrawl
- API key saved in
.env:FIRECRAWL_API_KEY=fc-4f9...63c1 - Hermes config:
web.backend: firecrawl,web.use_gateway: true - Activate on next session /reset after adding key
- Free tier: 1,000 credits/mo, upgrade to 5K credits planned when needed
- Usage tracking: cron job
firecrawl-usage-checkruns daily at 9 AM, no_agent
ScrapingAnt
- API key saved in
.env:SCRAPINGANT_API_KEY=8ab7...af6 - Used via direct API calls from terminal (
curlto scrapingant.com/v2/general with browser=true) - 10,000 free credits/mo, no credit card required
- $19/mo for 100K credits (recommended upgrade when free tier runs out)
SearXNG
- Deployed as Docker container on Core at port 8888 (localhost only)
- Accessed via
curl "http://127.0.0.1:8888/search?q=...&format=json" - JSON API returns structured results with titles, URLs, snippets
- Self-hosted, unlimited, no API key needed
- Auto-starts with Docker (compose file at ~/docker/searxng/)
Multi-Profile Setup
When a second Hermes profile (e.g. Anita's anita profile) needs web tools:
- Copy API keys to the profile's
.envfile - Enable web backend in the profile's
config.yaml - Restart the profile's gateway to pick up changes
- The user needs to /reset their session to see the new tools
Why Both Firecrawl and SearXNG?
SearXNG handles unlimited search (finding the right article/result at no cost). Firecrawl handles content extraction (converting specific page URLs to clean markdown). Without SearXNG, Firecrawl would handle both search + extraction but burn credits on searches that SearXNG can do for free.