Initial skills documentation — 25 categories, all SKILL.md + references + scripts

2026-07-15 17:42:20 -04:00
commit 1b4fcd4427
976 changed files with 188344 additions and 0 deletions
@@ -0,0 +1,184 @@
+# Browser MCP Pattern (Playwright + Browserless)
+
+A fully headless browser MCP server that provides web navigation, screenshots, JavaScript execution, element extraction, and click interactions. Uses **browserless/chrome** as the rendering engine and **Playwright** for the MCP integration.
+
+## Architecture
+
+```
+Open WebUI / Hermes
+  │ MCP (Streamable HTTP)
+  ↓
+mcp-browser (Python 3.13, FastMCP, Playwright)
+  │ CDP WebSocket
+  ↓
+browserless/chrome (headless Chromium)
+```
+
+## Docker Compose (add to existing LiteLLM stack)
+
+```yaml
+services:
+  browserless:
+    image: browserless/chrome:latest
+    container_name: browserless
+    restart: unless-stopped
+    environment:
+      - CONNECTION_TIMEOUT=600000
+      - MAX_CONCURRENT_SESSIONS=5
+      - PREBOOT_CHROME=true
+    ports:
+      - '127.0.0.1:3005:3000'   # 3000 conflicts with Open WebUI host mapping
+    networks:
+      - litellm-net
+    deploy:
+      resources:
+        limits: {cpus: '2', memory: '1G'}
+        reservations: {cpus: '0.5', memory: '256M'}
+
+  mcp-browser:
+    build: ../mcp-browser
+    container_name: mcp-browser
+    restart: unless-stopped
+    environment:
+      - BROWSER_WS=ws://browserless:3000   # Docker internal network name
+      - PYTHONUNBUFFERED=1
+    ports:
+      - '127.0.0.1:8901:8901'
+    networks:
+      - litellm-net
+    depends_on:
+      - browserless
+```
+
+## Dockerfile
+
+```dockerfile
+FROM python:3.13-slim
+WORKDIR /app
+RUN apt-get update && apt-get install -y --no-install-recommends \
+    curl ca-certificates && rm -rf /var/lib/apt/lists/*
+COPY requirements.txt .
+RUN pip install --no-cache-dir -r requirements.txt
+RUN python3 -m playwright install chromium
+RUN python3 -m playwright install-deps chromium
+COPY server.py ./
+ENV PYTHONUNBUFFERED=1
+EXPOSE 8901
+CMD ["python3", "server.py"]
+```
+
+## Requirements
+
+```
+fastmcp>=3.0.0
+playwright>=1.45.0
+httpx>=0.27.0
+```
+
+## Server Template (server.py)
+
+```python
+from fastmcp import FastMCP
+from playwright.async_api import async_playwright
+import base64, os
+
+BROWSER_WS = os.getenv('BROWSER_WS', 'ws://browserless:3000')
+mcp = FastMCP('Browser', version='1.0.0')
+_browser = None
+
+async def get_browser():
+    global _browser
+    if _browser is None:
+        pw = await async_playwright().start()
+        _browser = await pw.chromium.connect_over_cdp(BROWSER_WS)
+    return _browser
+
+@mcp.tool(description="Navigate to a URL and return page text + title")
+async def browse(url: str) -> str:
+    browser = await get_browser()
+    page = await browser.new_page()
+    try:
+        await page.goto(url, wait_until='networkidle', timeout=30000)
+        title = await page.title()
+        text = await page.evaluate('document.body?.innerText || ""')
+        return f"Title: {title}\n\n{text[:10000]}"
+    finally:
+        await page.close()
+
+@mcp.tool(description="Take a full-page screenshot, return as base64 PNG")
+async def screenshot(url: str) -> str:
+    browser = await get_browser()
+    page = await browser.new_page()
+    try:
+        await page.goto(url, wait_until='networkidle', timeout=30000)
+        data = await page.screenshot(full_page=True, type='png')
+        return f"data:image/png;base64,{base64.b64encode(data).decode()}"
+    finally:
+        await page.close()
+
+@mcp.tool(description="Execute JavaScript on a page, return result")
+async def execute_js(url: str, code: str) -> str:
+    browser = await get_browser()
+    page = await browser.new_page()
+    try:
+        await page.goto(url, wait_until='networkidle', timeout=30000)
+        result = await page.evaluate(code)
+        return str(result)[:5000]
+    finally:
+        await page.close()
+
+@mcp.tool(description="Extract content by CSS selector")
+async def extract_element(url: str, selector: str) -> str:
+    browser = await get_browser()
+    page = await browser.new_page()
+    try:
+        await page.goto(url, wait_until='networkidle', timeout=30000)
+        elements = await page.query_selector_all(selector)
+        texts = [await el.inner_text() for el in elements if (await el.inner_text()).strip()]
+        return f"Found {len(texts)} elements matching '{selector}':\n" + "\n---\n".join(texts[:10])
+    finally:
+        await page.close()
+
+@mcp.tool(description="Click an element by text or CSS selector, return new page state")
+async def click(url: str, target: str, is_selector: bool = False) -> str:
+    browser = await get_browser()
+    page = await browser.new_page()
+    try:
+        await page.goto(url, wait_until='networkidle', timeout=30000)
+        if is_selector:
+            await page.click(target)
+        else:
+            await page.click(f'text={target}')
+        await page.wait_for_load_state('networkidle', timeout=10000)
+        title = await page.title()
+        text = await page.evaluate('document.body?.innerText || ""')
+        return f"Title: {title}\nURL: {page.url}\n\n{text[:8000]}"
+    finally:
+        await page.close()
+
+if __name__ == '__main__':
+    mcp.run(transport='http', host='0.0.0.0', port=8901, path='/mcp')
+```
+
+## Verifying
+
+```bash
+docker exec mcp-browser python3 -c "
+from fastmcp import Client, asyncio
+async def test():
+    async with Client('http://127.0.0.1:8901/mcp') as client:
+        tools = await client.list_tools()
+        print(f'Tools: {len(tools)}')
+        for t in tools:
+            print(f'  - {t.name}: {t.description[:50]}...')
+asyncio.run(test())
+"
+```
+
+## Pitfalls
+
+- **Port 3000 conflict** — Open WebUI maps host port 3000. Browserless also defaults to 3000. Use `127.0.0.1:3005:3000` to remap.
+- **Playwright deps install is slow** — `playwright install-deps chromium` pulls apt packages, 2-3 min build time. Cache the Docker layer.
+- **Browserless TOKEN env** — Set `TOKEN=` (empty) to disable token auth. The default requires no auth for internal networks.
+- **Slim images lack curl** — The Dockerfile adds `curl ca-certificates` explicitly. Health checks using `curl` still won't work in slim containers without this.
+- **CDP connection reuse** — The server reuses one browser instance across all requests via `connect_over_cdp`. Each tool opens/closes a page (tab), not a whole browser.
@@ -0,0 +1,157 @@
+# Remote H2 Database Query via SSH
+
+When an MCP server needs to read from an H2 database on a remote host (e.g., Traccar on app2), use this pattern:
+
+## Architecture
+
+```
+MCP tool call
+  → SSH into remote host (key-based, no password prompt)
+  → scp the .mv.db file to a local temp file
+  → Java H2 Shell query (org.h2.tools.Shell)
+  → Parse pipe-delimited output
+  → Delete temp file
+  → Return structured JSON
+```
+
+## Why this pattern
+
+- **H2 databases lock.** You cannot query a live H2 database from two processes simultaneously. Copying with `scp` creates a read-only snapshot — the copy succeeds even while the DB is in use, and you query the copy.
+- **Java is needed.** H2's file format is proprietary. Python libraries like `jaydebeapi` exist but are brittle; the H2 Shell jar is self-contained and reliable.
+- **30-second cache.** The SCP + Java query takes 3-5 seconds. Caching avoids this overhead on repeated calls.
+
+## Prerequisites
+
+On the local host:
+```bash
+apt-get install -y default-jre-headless
+# Copy H2 jar from the remote host or the container
+scp root@remote:/tmp/h2.jar /tmp/h2.jar
+# Test: java -cp /tmp/h2.jar org.h2.tools.Shell --help
+```
+
+On the remote host (the H2 jar must exist):
+```bash
+# If inside a Docker container:
+docker cp container_name:/opt/app/lib/h2-*.jar /tmp/h2.jar
+```
+
+## Implementation Template
+
+```python
+import subprocess, tempfile, os, time
+
+HOST = "152.53.xxx.xxx"
+DB_PATH = "/root/docker/app/data/database"
+CACHE_TTL = 30
+
+_cache = {"data": None, "ts": 0}
+
+def query_devices():
+    if _cache["data"] and (time.time() - _cache["ts"]) < CACHE_TTL:
+        return _cache["data"]
+
+    tmp_db = tempfile.mktemp(suffix=".mv.db")
+    # 1. Copy DB snapshot from remote
+    subprocess.run(
+        ["scp", "-i", "/root/.ssh/itpp-infra", "-o", "StrictHostKeyChecking=no",
+         f"root@{HOST}:{DB_PATH}.mv.db", tmp_db],
+        capture_output=True, timeout=15, check=True
+    )
+
+    # 2. Query with Java H2 Shell
+    base = tmp_db.replace(".mv.db", "")
+    result = subprocess.run(
+        ["java", "-cp", "/tmp/h2.jar", "org.h2.tools.Shell",
+         "-url", f"jdbc:h2:{base}", "-user", "sa", "-password", "",
+         "-sql", "SELECT id, name, latitude, longitude FROM tc_devices;"],
+        capture_output=True, text=True, timeout=20
+    )
+    os.unlink(tmp_db)
+
+    # 3. Parse pipe-delimited output (H2 Shell format)
+    # Output: "ID | NAME | LATITUDE | LONGITUDE\n1  | foo  | 10.4     | -75.5"
+    rows = []
+    for line in result.stdout.strip().split('\n'):
+        parts = [p.strip() for p in line.split('|')]
+        if len(parts) >= 4 and parts[0].strip().isdigit():
+            rows.append({
+                "id": int(parts[0]),
+                "name": parts[1] if parts[1] != "null" else None,
+            })
+
+    data = {"rows": rows, "count": len(rows)}
+    _cache["data"] = data
+    _cache["ts"] = time.time()
+    return data
+```
+
+## Traccar Schema Reference (GPS Tracking)
+
+These are the live table schemas used by Traccar 6.x. All queries go through the H2 database at `/root/docker/traccar/traccar-data/database.mv.db` on app2 (152.53.39.202).
+
+### tc_devices
+
+| Column | Type | Notes |
+|---|---|---|
+| id | INT | Primary key |
+| name | VARCHAR | User-assigned device name |
+| uniqueid | VARCHAR | Device identifier (IMEI, serial) |
+| lastupdate | TIMESTAMP | Last position received |
+| status | VARCHAR | "online", "offline", or "unknown" |
+
+### tc_positions
+
+| Column | Type | Notes |
+|---|---|---|
+| id | BIGINT | Auto-increment |
+| deviceid | INT | FK → tc_devices.id |
+| latitude | DOUBLE | Decimal degrees |
+| longitude | DOUBLE | Decimal degrees |
+| speed | DOUBLE | Knots (× 1.15078 for mph) |
+| devicetime | TIMESTAMP | Device-reported time |
+
+### Common Queries
+
+```sql
+-- All devices with latest position (one query)
+SELECT d.id, d.name, d.uniqueid, d.lastupdate, d.status,
+       p.latitude, p.longitude, p.speed, p.devicetime
+FROM tc_devices d
+LEFT JOIN tc_positions p ON p.id = (
+    SELECT MAX(id) FROM tc_positions WHERE deviceid = d.id
+)
+ORDER BY d.lastupdate DESC;
+
+-- Position history for one device
+SELECT latitude, longitude, speed, devicetime
+FROM tc_positions WHERE deviceid = 1
+ORDER BY devicetime ASC;
+
+-- Stats: distance (haversine), max speed, active time
+SELECT COUNT(*) as positions,
+       MAX(speed) * 1.15078 as max_speed_mph
+FROM tc_positions
+WHERE deviceid = 1
+  AND devicetime >= NOW() - INTERVAL '24' HOUR;
+```
+
+## FastAPI Proxy Integration
+
+When the MCP server is slow (3-5s per SCP+Java query), serve data through a FastAPI proxy with 30-second caching. The ops portal at `/opt/ops-portal/server.py` uses this pattern:
+
+```python
+# /api/ft360/status endpoint proxies Traccar data
+@app.get("/api/ft360/status")
+async def traccar_status():
+    return _get_traccar_data()  # cached 30s, SCP+Java underneath
+```
+
+The JavaScript dashboard fetches from `/api/ft360/status` every 30 seconds — same TTL as the cache. No authentication needed; the proxy handles the SSH and Java complexity.
+
+- **Java must be on PATH.** Install `default-jre-headless` before using. The `java` binary in Docker containers (e.g., `/opt/traccar/jre/bin/java`) isn't on PATH by default.
+- **H2 jar must be on the same host and compatible.** Copy it from the container or remote host — don't download a different version from Maven Central.
+- **H2 Shell output format.** The first line is column headers (`ID | NAME | ...`), subsequent lines are data. Filter with `.isdigit()` on the first column.
+- **SCP key must work.** The SSH key path must be absolute, and `StrictHostKeyChecking=no` avoids prompt-freezing on first connection.
+- **Temp file cleanup.** Always `os.unlink(tmp_db)` in a `finally` block or immediately after the query. A 176KB file won't cause issues if left, but it's sloppy.
+- **Cache invalidation.** If you stop+start the remote container (restarting the DB), the cache may return stale data until the TTL expires. For write operations, skip the cache.
@@ -0,0 +1,33 @@
+# Immich photo map MCP pattern
+
+Use this when Germaine asks to extract photo metadata from Immich and show where photos were taken.
+
+## Immich API endpoints
+
+Relevant stable endpoints:
+
+- `POST /search/metadata` with `withExif: true` to search assets and include EXIF/location data.
+- `GET /assets/{id}/metadata` to fetch full key-value metadata for a single asset.
+
+`/search/metadata` can filter by album IDs, taken date, city/state/country, camera make/model, favorite state, asset type, etc.
+
+## Recommended MCP tools
+
+- `immich_search_geo_assets(taken_after?, taken_before?, album_id?, limit=500)` — return assets with latitude/longitude, taken time, filename, city/state/country, camera model, thumbnail URL.
+- `immich_get_asset_metadata(asset_id)` — return full Immich metadata for one asset.
+- `immich_geojson(...)` — return a GeoJSON FeatureCollection suitable for Leaflet/MapLibre.
+- `immich_map_summary(...)` — summarize counts by city/country/date range and count missing-location assets.
+
+## Map UI
+
+The MCP should return data; a small UI can live in ops portal or a private subdomain. Use Leaflet + OpenStreetMap tiles + marker clustering. Marker click should show thumbnail, filename, taken date, and location labels.
+
+## Security
+
+GPS photo data is sensitive. Keep the MCP localhost-only, store `IMMICH_BASE_URL` and `IMMICH_API_KEY` in a chmod 600 `.env`, and put any map UI behind auth/Cloudflare Access. Prefer thumbnails, not full-resolution assets.
+
+## Implementation notes
+
+- Count missing GPS as `missing_location`, not an error.
+- Return GeoJSON coordinates as `[longitude, latitude]`.
+- Build read-only first. Metadata update/fix workflows are a later explicit feature.
@@ -0,0 +1,108 @@
+# Open WebUI Configuration via SQLite
+
+Open WebUI v0.10+ stores persistent config as JSON key/value pairs in its SQLite database at `/app/backend/data/webui.db` (Docker volume: `/var/lib/docker/volumes/openwebui_data/_data/webui.db`).
+
+After the first launch, Docker `ConfigVar` environment variables are **ignored** — the DB is the single source of truth. All configuration changes after initial setup must be made directly in SQLite.
+
+## Schema
+
+```sql
+CREATE TABLE config (
+    "key" TEXT NOT NULL, 
+    value JSON NOT NULL, 
+    updated_at BIGINT, 
+    PRIMARY KEY ("key")
+);
+```
+
+## Essential config keys
+
+### Model capabilities (vision, image generation)
+
+```sql
+-- Enable vision for all models (required for image upload in chat)
+UPDATE config SET value = '{"capabilities": {"vision": true}}', updated_at = <unix_ts>
+WHERE key = 'models.default_metadata';
+```
+
+Without this, image upload is disabled in Open WebUI regardless of which vision-capable models are connected.
+
+### Image generation
+
+```sql
+-- Enable image generation feature
+UPDATE config SET value = 'true', updated_at = <unix_ts> WHERE key = 'image_generation.enable';
+-- Engine/model is stored in separate keys
+-- Engine: openai, automatic1111, comfyui, gemini
+UPDATE config SET value = '"openai"', updated_at = <unix_ts> WHERE key = 'image_generation.engine';
+UPDATE config SET value = '"gemini/imagen-4.0-generate-001"', updated_at = <unix_ts> WHERE key = 'image_generation.model';
+```
+
+### Web search
+
+```sql
+-- Enable web search
+UPDATE config SET value = 'true', updated_at = <unix_ts> WHERE key = 'web.search.enable';
+-- Set engine: searxng, google_pse, brave, serpapi, etc.
+UPDATE config SET value = '"google_pse"', updated_at = <unix_ts> WHERE key = 'web.search.engine';
+```
+
+Public SearXNG instances (searx.be, search.sapti.me, etc.) are heavily rate-limited (403/429). Google PSE is the reliable fallback but uses API quota. Self-hosted SearXNG is ideal.
+
+### MCP tool server connections
+
+```python
+import sqlite3, json, time
+db = sqlite3.connect('/var/lib/docker/volumes/openwebui_data/_data/webui.db')
+
+row = db.execute("SELECT value FROM config WHERE key='tool_server.connections'").fetchone()
+current = json.loads(row[0]) if row else []
+
+# Remove duplicates by port
+current = [c for c in current if '<port>' not in c.get('url', '')]
+
+current.append({
+    'url': 'http://host.docker.internal:<PORT>/mcp',
+    'type': 'mcp',              # CRITICAL: not 'openapi'
+    'auth_type': 'none',
+    'key': '',
+    'headers': None,
+    'config': {
+        'enable': True,
+        'function_name_filter_list': '',
+        'access_grants': [{
+            'principal_type': 'group',
+            'principal_id': '<group-id>',  # from DB: SELECT value FROM config WHERE key='ui.default_group_id'
+            'permission': 'read'
+        }]
+    },
+    'info': {'id': '', 'name': 'Server Name', 'description': 'What it does'}
+})
+
+db.execute("UPDATE config SET value = ?, updated_at = ? WHERE key = ?",
+    (json.dumps(current), int(time.time()), 'tool_server.connections'))
+db.commit()
+db.close()
+# Then: docker restart openwebui
+```
+
+### User features/permissions
+
+```sql
+-- Check image_generation and other feature flags
+SELECT value FROM config WHERE key = 'user.permissions';
+-- Look for JSON path: $.features.image_generation
+```
+
+## Change workflow
+
+1. Update the DB: `python3 -c "..."` (sqlite3 write)
+2. Verify: `SELECT value FROM config WHERE key = '...'`
+3. Restart: `docker restart openwebui`
+4. Wait 10-15 seconds for health check
+
+## Pitfalls
+
+- **ConfigVar env vars are dead on restart** — After first launch, `DEFAULT_MODEL_METADATA` and other ConfigVar env vars in docker-compose are silently ignored. The DB wins.
+- **Quoting in shell** — When running Python SQLite updates through SSH, use single-quoted Python strings and double-quoted SQL identifiers to avoid escaping hell. Better: write a .py script, scp it, run it.
+- **Group ID discovery** — The principal_id for access grants is found at `SELECT value FROM config WHERE key='ui.default_group_id'`. This changes per installation.
@@ -0,0 +1,69 @@
+# OSINT Person Intelligence MCP Pattern
+
+Build a dedicated entity-resolution layer above generic search MCPs when the task is person/skip-trace intelligence rather than raw search.
+
+## Architecture
+
+```
+Hermes
+  ↓
+osint-person-mcp        # entity resolution, scoring, dossier logic
+  ↓
+super-search MCP        # raw search + extraction providers
+  ↓
+SearXNG / Exa / OpenCorporates / CourtListener / Firecrawl
+```
+
+Keep `super-search` generic. Put identity-specific logic in a separate MCP so the generic search engine does not become a grab bag of scoring and compliance behavior.
+
+## Tool set
+
+Recommended tools:
+
+| Tool | Purpose |
+|---|---|
+| `person_search` | Query generation, search, source scoring, entity clustering, compliance notes |
+| `phone_search_variants` | Normalize phone and emit common variants |
+| `phone_search` | Search all phone variants and cluster matches |
+| `email_search` | Search email address and cluster matches |
+| `business_affiliation_search` | Search person + organization/business/location anchors |
+| `court_business_search` | Search court/business/public-record query variants |
+| `score_source` | Score URL/title/description reliability |
+| `dossier_report` | Convert structured JSON into concise markdown report |
+
+## Source scoring
+
+Use durable source tiers:
+
+| Tier | Examples | Treatment |
+|---|---|---|
+| High | official school/government/court/publication sources | Strong evidence only when identity anchors also match |
+| Medium-high | LinkedIn, RocketReach, TheOrg, Tradeloop, ZoomInfo | Useful but needs corroboration |
+| Medium-low | Whitepages, 411, BeenVerified, NPD, FastBackgroundCheck | Lead only; mark unverified |
+| Low | obituaries/name-only pages | Usually weak/name-collision |
+| Reject | random blob/CDN/search-spam pages | Exclude from useful clusters |
+
+Do not let organization-only pages rank as likely person matches. An official SCAD page with no subject name is context, not identity evidence.
+
+## Matching pitfalls
+
+- Ignore one-letter middle initials for alias matching. A lone `m` from `Germaine M. Brown` matches almost every page.
+- Normalize phone numbers into all common forms: digits, dashed, dotted, `(xxx) xxx-xxxx`, `+1 xxx xxx xxxx`, `+1-xxx-xxx-xxxx`.
+- Separate likely target, possible target, weak/different, and rejected results.
+- Preserve source URL, query, provider, matched fields, score, and timestamp for every lead.
+- Treat Atlanta/Savannah/location-only matches as insufficient without identity anchors.
+
+## Compliance guardrails
+
+- OSINT results are leads, not verified facts.
+- Do not auto-contact from scraped or data-broker leads without human review.
+- Preserve source URL and timestamp for every claim.
+- Mark data-broker information as unverified until corroborated.
+
+## DRE integration path
+
+1. Add `Run OSINT` button on claim/debtor page.
+2. Call `person_search` with debtor anchors.
+3. Save JSON packet + markdown dossier to debtor record.
+4. Require human review before any collection action uses a lead.
+5. Later add paid providers only where MVP gaps are proven (People Data Labs, Trestle/Ekata-style phone intelligence, Pipl).
@@ -0,0 +1,84 @@
+# Super Search — MCP Server Build Reference
+
+**Built:** July 11, 2026 · **Updated:** July 15, 2026 (v2.0.0)
+**Server:** Core (152.53.192.33)
+**Service:** `super-search.service` (systemd, auto-restart)
+**Endpoint:** http://127.0.0.1:8899/mcp
+
+## Architecture
+
+```
+Super Search (FastMCP v2.0.0) — 10 tools, 7 providers
+  ├── web_search(query, limit, category, time_range, site_filter)
+  │   └── SearXNG → Exa → OpenCorporates → CourtListener → DuckDuckGo → Firecrawl
+  ├── web_extract(url, char_limit)
+  │   └── Trafilatura → Firecrawl
+  ├── web_search_premium(query, limit, include_domains, exclude_domains, date_start, date_end)
+  │   └── Exa direct (DRE-grade)
+  ├── web_search_news(query, limit, days)          ★ NEW
+  │   └── SearXNG (news) → Exa
+  ├── web_search_academic(query, limit)            ★ NEW
+  │   └── SearXNG (science) → Exa
+  ├── web_extract_batch(urls, char_limit)          ★ NEW
+  │   └── Parallel Trafilatura → Firecrawl (max 10 URLs)
+  ├── web_suggest_queries(query, limit)            ★ NEW
+  │   └── SearXNG suggestions
+  ├── health_check()                               ★ NEW
+  │   └── Provider statuses + cache stats + rate limit buckets
+  ├── web_lookup(query, limit)                     ★ NEW
+  │   └── DuckDuckGo Instant Answers + Wikipedia
+  └── web_search_images(query, limit)              ★ NEW
+      └── SearXNG (images) → general fallback
+```
+
+## File Layout
+
+```
+/root/docker/super-search/
+├── server.py              ← FastMCP server (192 lines)
+├── super-search.service   ← systemd unit file (copied to /etc/systemd/system/)
+├── venv/                  ← Python virtualenv
+├── exa.env                ← Exa API key (chmod 600)
+├── check_api.py           ← test artifacts (cleanup)
+└── check_api2.py          ← test artifacts (cleanup)
+```
+
+## Key Design Decisions
+
+1. **Async I/O everywhere** — httpx.AsyncClient for all HTTP calls. The searxng, exa, firecrawl, duckduckgo, and wikipedia search functions are all async, allowing the server to handle concurrent requests without blocking.
+
+2. **Seven-provider fallback chain** — SearXNG (free, privacy-respecting) → Exa (premium neural) → OpenCorporates (company records) → CourtListener (case law) → DuckDuckGo (instant answers) → Wikipedia (encyclopedia) → Firecrawl (last resort). Each tier fails independently with full telemetry reporting.
+
+3. **Extraction fallback** — Trafilatura (local, fast, free) → Firecrawl scrape (remote, paid). Trafilatura handles most HTML pages; Firecrawl handles JavaScript-heavy and paywalled sites.
+
+4. **Batch extraction** — `web_extract_batch` extracts up to 10 URLs in parallel using `asyncio.gather`, dramatically faster than sequential extraction.
+
+5. **Domain-specific tools** — News, academic, image search, and factual lookups each get dedicated tools with appropriate fallback chains and defaults (date ranges for news, science category for academic).
+
+6. **Credentials from .env** — FIRECRAWL_API_KEY and EXA_API_KEY loaded from ~/.hermes/.env via python-dotenv. The systemd service uses EnvironmentFile to pass them.
+
+7. **Rate limiting** — Token bucket per provider (7 providers), TTL caching per provider, retry with exponential backoff on 429/5xx.
+
+8. **Provider health telemetry** — Every search tool returns which providers were tried, their status, result counts, and upstream engine health (SearXNG unresponsive engines with reasons).
+
+## Hermes Integration
+
+```yaml
+mcp_servers:
+  super-search:
+    url: http://127.0.0.1:8899/mcp
+    enabled: true
+```
+
+After deploying a new version, the Hermes gateway must be restarted to pick up new tools:
+```bash
+systemctl restart hermes.service
+# or from outside the gateway: hermes gateway restart
+```
+
+## Version History
+
+- v1.0 — Initial build (SearXNG → Firecrawl, Trafilatura → Firecrawl) — 3 tools
+- v1.1 — Added Exa as premium tier (SearXNG → Exa → Firecrawl) — 3 tools
+- v1.2 — Added OpenCorporates + CourtListener to fallback chain + rate limiting — 3 tools
+- v2.0.0 — Major expansion: 7 new tools (10 total), 2 new providers (DuckDuckGo, Wikipedia), enhanced params on all existing tools — July 15, 2026