Initial skills documentation — 25 categories, all SKILL.md + references + scripts

2026-07-15 17:42:20 -04:00
commit 1b4fcd4427
976 changed files with 188344 additions and 0 deletions
@@ -0,0 +1,84 @@
+# Super Search — MCP Server Build Reference
+
+**Built:** July 11, 2026 · **Updated:** July 15, 2026 (v2.0.0)
+**Server:** Core (152.53.192.33)
+**Service:** `super-search.service` (systemd, auto-restart)
+**Endpoint:** http://127.0.0.1:8899/mcp
+
+## Architecture
+
+```
+Super Search (FastMCP v2.0.0) — 10 tools, 7 providers
+  ├── web_search(query, limit, category, time_range, site_filter)
+  │   └── SearXNG → Exa → OpenCorporates → CourtListener → DuckDuckGo → Firecrawl
+  ├── web_extract(url, char_limit)
+  │   └── Trafilatura → Firecrawl
+  ├── web_search_premium(query, limit, include_domains, exclude_domains, date_start, date_end)
+  │   └── Exa direct (DRE-grade)
+  ├── web_search_news(query, limit, days)          ★ NEW
+  │   └── SearXNG (news) → Exa
+  ├── web_search_academic(query, limit)            ★ NEW
+  │   └── SearXNG (science) → Exa
+  ├── web_extract_batch(urls, char_limit)          ★ NEW
+  │   └── Parallel Trafilatura → Firecrawl (max 10 URLs)
+  ├── web_suggest_queries(query, limit)            ★ NEW
+  │   └── SearXNG suggestions
+  ├── health_check()                               ★ NEW
+  │   └── Provider statuses + cache stats + rate limit buckets
+  ├── web_lookup(query, limit)                     ★ NEW
+  │   └── DuckDuckGo Instant Answers + Wikipedia
+  └── web_search_images(query, limit)              ★ NEW
+      └── SearXNG (images) → general fallback
+```
+
+## File Layout
+
+```
+/root/docker/super-search/
+├── server.py              ← FastMCP server (192 lines)
+├── super-search.service   ← systemd unit file (copied to /etc/systemd/system/)
+├── venv/                  ← Python virtualenv
+├── exa.env                ← Exa API key (chmod 600)
+├── check_api.py           ← test artifacts (cleanup)
+└── check_api2.py          ← test artifacts (cleanup)
+```
+
+## Key Design Decisions
+
+1. **Async I/O everywhere** — httpx.AsyncClient for all HTTP calls. The searxng, exa, firecrawl, duckduckgo, and wikipedia search functions are all async, allowing the server to handle concurrent requests without blocking.
+
+2. **Seven-provider fallback chain** — SearXNG (free, privacy-respecting) → Exa (premium neural) → OpenCorporates (company records) → CourtListener (case law) → DuckDuckGo (instant answers) → Wikipedia (encyclopedia) → Firecrawl (last resort). Each tier fails independently with full telemetry reporting.
+
+3. **Extraction fallback** — Trafilatura (local, fast, free) → Firecrawl scrape (remote, paid). Trafilatura handles most HTML pages; Firecrawl handles JavaScript-heavy and paywalled sites.
+
+4. **Batch extraction** — `web_extract_batch` extracts up to 10 URLs in parallel using `asyncio.gather`, dramatically faster than sequential extraction.
+
+5. **Domain-specific tools** — News, academic, image search, and factual lookups each get dedicated tools with appropriate fallback chains and defaults (date ranges for news, science category for academic).
+
+6. **Credentials from .env** — FIRECRAWL_API_KEY and EXA_API_KEY loaded from ~/.hermes/.env via python-dotenv. The systemd service uses EnvironmentFile to pass them.
+
+7. **Rate limiting** — Token bucket per provider (7 providers), TTL caching per provider, retry with exponential backoff on 429/5xx.
+
+8. **Provider health telemetry** — Every search tool returns which providers were tried, their status, result counts, and upstream engine health (SearXNG unresponsive engines with reasons).
+
+## Hermes Integration
+
+```yaml
+mcp_servers:
+  super-search:
+    url: http://127.0.0.1:8899/mcp
+    enabled: true
+```
+
+After deploying a new version, the Hermes gateway must be restarted to pick up new tools:
+```bash
+systemctl restart hermes.service
+# or from outside the gateway: hermes gateway restart
+```
+
+## Version History
+
+- v1.0 — Initial build (SearXNG → Firecrawl, Trafilatura → Firecrawl) — 3 tools
+- v1.1 — Added Exa as premium tier (SearXNG → Exa → Firecrawl) — 3 tools
+- v1.2 — Added OpenCorporates + CourtListener to fallback chain + rate limiting — 3 tools
+- v2.0.0 — Major expansion: 7 new tools (10 total), 2 new providers (DuckDuckGo, Wikipedia), enhanced params on all existing tools — July 15, 2026