ippadmin/hermes-skills

Fork 0

Files

T

root 1b4fcd4427 Initial skills documentation — 25 categories, all SKILL.md + references + scripts

2026-07-15 17:42:20 -04:00

17 KiB

Raw Permalink Blame History

name, description, version, author, tags

name

description

version

author

MCP Servers

Build, deploy, and wire MCP servers into Hermes. MCP servers provide additional tools via the Model Context Protocol — think of them as plugin toolkits your agent can call natively.

Architecture

Hermes Agent
  │ native MCP client
  ├── mcp_servers.mysql     → python3 /root/.hermes/scripts/mcp-mysql.py
  ├── mcp_servers.super-search → http://127.0.0.1:8899/mcp
  └── mcp_servers.exa       → https://mcp.exa.ai/mcp

Two transport types:

Command-based — Hermes spawns a Python process that speaks MCP over stdio
HTTP-based — A persistent server runs at an HTTP endpoint, Hermes connects via SSE

HTTP-Based MCP Server (FastMCP Pattern)

This is the preferred pattern for complex, persistent toolkits. The server runs as a systemd service and Hermes connects via HTTP.

Build Steps

Install dependencies in a virtualenv:

pip install fastmcp httpx trafilatura firecrawl-py

Create the server — a single Python file with FastMCP tools:

from fastmcp import FastMCP
mcp = FastMCP("Server Name", version="1.0.0")

@mcp.tool(description="Search the web")
async def web_search(query: str, limit: int = 5) -> str:
    # ... implementation
    return results

if __name__ == "__main__":
    mcp.run(transport="http", host="127.0.0.1", port=8899, path="/mcp")

Create a systemd service:

[Unit]
Description=Super Search MCP Server
After=network.target

[Service]
Type=simple
User=root
WorkingDirectory=/root/docker/super-search
Environment="PATH=/root/docker/super-search/venv/bin:/usr/local/sbin:..."
Environment="PYTHONUNBUFFERED=1"
EnvironmentFile=/root/.hermes/.env
ExecStart=/root/docker/super-search/venv/bin/python3 /root/docker/super-search/server.py
Restart=always
RestartSec=3

[Install]
WantedBy=multi-user.target

Wire into Hermes config:

hermes config set mcp_servers.<name>.url 'http://127.0.0.1:<port>/mcp'
hermes config set mcp_servers.<name>.enabled 'true'

Or edit ~/.hermes/config.yaml directly:

mcp_servers:
  super-search:
    url: http://127.0.0.1:8899/mcp
    enabled: true

Restart gateway or reload MCP:

# Need to restart from outside the gateway session
systemctl restart hermes.service

Testing

# List all MCP servers
hermes mcp list

# Test connection and discover tools
hermes mcp test <server-name>
# Expected output:
#   ✓ Connected (XXms)
#   ✓ Tools discovered: N
#     tool_name1     Description...
#     tool_name2     Description...

Performance

Local HTTP MCP servers: ~50-60ms connection time
Remote HTTP MCP servers (e.g., Exa): ~1-2s connection time (includes OAuth/TLS)

Troubleshooting

Server won't start — port in use: A leftover process from a subagent's test run often holds the port. The fuser command alone may not suffice:

# Find what's on the port
ss -tlnp | grep <port>   # shows PID
kill <PID>               # then restart
systemctl restart <service>

If the process won't die (orphaned): fuser -k <port>/tcp then restart.

Subagent timeout during builds: The default 600s child timeout can be tight for multi-step MCP server builds (install deps + write code + systemd service + Hermes config + gateway restart). The subagent usually produces the code and service file correctly before timing out — just the final wiring steps (systemctl enable, testing) are incomplete. Do those manually rather than re-dispatching.

MCP endpoint returns HTTP 406: The client must accept both application/json and text/event-stream. Add headers:

Accept: application/json, text/event-stream

MCP endpoint returns "Missing session ID": The HTTP transport uses SSE for session negotiation. Simple curl tests won't work. Use hermes mcp test instead, which handles the MCP protocol correctly.

Tools discovered but not showing in Hermes: Restart the gateway: systemctl restart hermes.service (from outside the gateway session).

Immich Photo Map MCP Pattern

For Immich/photo-location workflows, build a dedicated read-only MCP that uses Immich POST /search/metadata with withExif: true and returns geotagged assets plus GeoJSON for a private Leaflet map. Keep GPS data private and expose thumbnails only. See references/immich-photo-map-mcp.md.

OSINT Person Intelligence Layer

When raw search becomes a person/skip-trace workflow, do not overload the generic search MCP with identity logic. Build a dedicated entity-resolution MCP above Super Search. Pattern reference: references/osint-person-intelligence.md.

Use this pattern for: person search, phone/email variant search, source scoring, identity clustering, dossier generation, DRE debtor enrichment, and compliance-preserving lead packets.

Command-Based MCP Server Pattern

For simpler, stateless MCP servers that don't warrant a persistent daemon:

mcp_servers:
  mysql:
    command: python3
    args:
      - /root/.hermes/scripts/mcp-mysql.py
    enabled: true

Hermes spawns the process per-request. No systemd service needed.

Remote MCP Server Pattern (OAuth)

Some services (Exa, agent-data.dev) provide hosted MCP servers with OAuth:

mcp_servers:
  exa:
    url: https://mcp.exa.ai/mcp
    enabled: true

First connection opens a browser for OAuth sign-in. Heremes handles this via hermes mcp login <name>.

Rate Limiting & TTL Caching Layer

When integrating multiple third-party APIs with rate limits (free tiers, govt databases, people-search APIs), add a token-bucket rate limiter and TTL cache between the MCP tool and the provider calls.

Architecture

Query → TTL Cache (dict, per-provider TTL)
          ↓ cache miss
       Token Bucket (per-provider tokens/interval)
          ↓ tokens available
       Provider API → success? → cache + return
          ↓ 429 / rate limited
       Exponential Backoff (respects Retry-After header)
          ↓ exhausted
       Fallback → next provider in chain

Implementation — `ratelimit.py`

Create a standalone module that the MCP server imports:

"""ratelimit.py — Token bucket rate limiter + TTL cache for MCP providers."""

import time, hashlib, json, asyncio

# Per-provider rate limit config
RATE_LIMITS = {
    "searxng":        {"tokens": 30, "interval": 1.0},   # local — high limit
    "exa":            {"tokens": 10, "interval": 1.0},   # paid tier
    "firecrawl":      {"tokens": 5,  "interval": 60.0},  # 1k/mo budget
    "opencorporates": {"tokens": 1,  "interval": 1.0},  # 1/sec free tier
    "courtlistener":  {"tokens": 10, "interval": 6.0},  # ~100/min free
}

# Per-provider cache TTL in seconds
CACHE_TTL = {
    "searxng":        120,    # 2 min — web results change
    "exa":            300,    # 5 min
    "opencorporates": 3600,  # 1 hr — company records are stable
    "courtlistener":  3600,  # 1 hr — court cases are infrequent
    "firecrawl":      300,   # 5 min
}

class TokenBucket:
    """Async-safe token bucket for per-provider rate limiting."""
    def __init__(self, tokens, interval):
        self.max_tokens = tokens
        self.tokens = tokens
        self.interval = interval  # seconds between refills
        self.last_refill = time.time()

    async def acquire(self):
        now = time.time()
        if now - self.last_refill >= self.interval:
            self.tokens = self.max_tokens
            self.last_refill = now
        if self.tokens <= 0:
            return False
        self.tokens -= 1
        return True

_buckets = {}
_cache = {}

async def rate_limited_search(provider: str, cache_key: str, fn, *args, **kwargs):
    """Wrapper: check cache → rate limit → call → cache result."""
    # 1. Cache check
    cache_key_full = f"{provider}:{cache_key}"
    if cache_key_full in _cache:
        entry = _cache[cache_key_full]
        if time.time() - entry["ts"] < CACHE_TTL.get(provider, 300):
            return entry["data"]

    # 2. Rate limit check
    rl = RATE_LIMITS.get(provider)
    if rl:
        if provider not in _buckets:
            _buckets[provider] = TokenBucket(rl["tokens"], rl["interval"])
        allowed = await _buckets[provider].acquire()
        if not allowed:
            raise RateLimitedError(f"{provider}: token bucket empty")

    # 3. Call with retry
    result = await _retry_with_backoff(fn, *args, **kwargs)

    # 4. Cache result
    _cache[cache_key_full] = {"data": result, "ts": time.time()}
    return result

async def _retry_with_backoff(fn, *args, retries=3, base_delay=1.0):
    """Exponential backoff on 429/5xx, respects Retry-After."""
    for attempt in range(retries):
        try:
            return await fn(*args, **kwargs)
        except Exception as e:
            if attempt == retries - 1:
                raise
            delay = base_delay * (2 ** attempt)
            # Check for Retry-After in response headers
            await asyncio.sleep(delay)

Key decisions

Cache aggressive for stable data — company records and court cases get 1-hour TTLs since they rarely change. Web search results get 2-5 minutes.
Token bucket per provider — each provider has its own limit so one fast provider can't starve another.
Fall through on rate limit — don't fail the whole tool. The MCP server's fallback chain should catch the RateLimitedError and try the next provider.

Multi-Provider Search Routing Pattern

When building a search/extraction MCP server that uses multiple backends, use this fallback chain pattern (incorporating rate limiting):

async def web_search(query, limit=5):
    # 1. Try SearXNG (free, local)
    try:
        results = await _searxng_search(query, limit)
        if results:
            return {"provider": "searxng", "results": results}
    except:
        pass
    
    # 2. Fallback to Exa (premium, API key)
    try:
        results = await _exa_search(query, limit)
        if results:
            return {"provider": "exa", "results": results}
    except:
        pass
    
    # 3. Last resort: Firecrawl (paid, most resilient)
    results = await _firecrawl_search(query, limit)
    return {"provider": "firecrawl", "results": results, "telemetry": telemetry}

This ensures the service degrades gracefully — free first, premium second, paid last resort.

Telemetry rule: Multi-provider MCP tools must return provider telemetry, not just final results. Include providers attempted, status/error, result counts, and upstream health details (for example, SearXNG unresponsive_engines with CAPTCHA/rate-limit reasons). Do not silently hide a provider that is reachable but returning zero useful results because upstream engines are blocked.

Remote Database MCP Data Source

When an MCP server needs to read from a database on a remote host (e.g., Traccar H2 on app2), use the SSH+SCP+query pattern: copy the DB as a read-only snapshot, query locally with Java H2 Shell, parse pipe-delimited output, return structured data. Avoids API auth, CORS, and credential management for read-only queries. Full reference: references/h2-remote-db-query.md.

This pattern works for H2, SQLite, and any file-based database accessible via SSH.

Docker-Based MCP Deployment (alongside Open WebUI)

When deploying MCP servers on app1 to serve Open WebUI (and Hermes), use Docker Compose instead of systemd. Add the MCP service to the existing LiteLLM compose at /root/docker/litellm/docker-compose.yml.

Pattern

services:
  super-search:
    build: ../super-search          # Directory with server.py + Dockerfile
    container_name: super-search
    restart: unless-stopped
    environment:
      - EXA_API_KEY=<key>
      - FIRECRAWL_API_KEY=<key>
      - PYTHONUNBUFFERED=1
    ports:
      - '127.0.0.1:8899:8899'       # Bound to 127.0.0.1 — services on same host
    networks:
      - litellm-net                  # Share LiteLLM's network

Dockerfile (slim Python)

FROM python:3.13-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY server.py ratelimit.py ./
ENV PYTHONUNBUFFERED=1
EXPOSE 8899
CMD ["python3", "server.py"]

Health check pitfall — slim containers

Python slim images lack curl, ss, pgrep, and other common health check tools. Do NOT use shell-based health checks. Options:

Skip it — Remove the health check block entirely. Docker will still restart on crash.
Python socket test — python3 -c "import socket; s=socket.socket(); s.connect(('127.0.0.1',8899)); s.close()"
Docker exec test from host — docker exec super-search python3 -c "..."

Addressing

Open WebUI (Docker → host): http://host.docker.internal:8899/mcp
Hermes (host → Docker): http://127.0.0.1:8899/mcp (since port is mapped to localhost)
Process test from inside container: http://127.0.0.1:8899/mcp

Verifying deployment

Use docker exec with FastMCP client to verify tools:

docker exec super-search python3 -c "
from fastmcp import Client
import asyncio
async def test():
    async with Client('http://127.0.0.1:8899/mcp') as client:
        tools = await client.list_tools()
        print(f'Tools: {len(tools)}')
        for t in tools:
            print(f'  - {t.name}')
asyncio.run(test())
"

Wiring MCP into Open WebUI

Open WebUI stores tool server connections as a JSON blob in its SQLite DB at key tool_server.connections. Add MCP servers programmatically:

import sqlite3, json, time
db = sqlite3.connect('/var/lib/docker/volumes/openwebui_data/_data/webui.db')

row = db.execute("SELECT value FROM config WHERE key='tool_server.connections'").fetchone()
current = json.loads(row[0]) if row else []

# Remove duplicates by URL
current = [c for c in current if '8899' not in c.get('url', '')]

current.append({
    'url': 'http://host.docker.internal:8899/mcp',
    'type': 'mcp',                      # CRITICAL: not 'openapi'
    'auth_type': 'none',
    'config': {
        'enable': True,
        'function_name_filter_list': '',
        'access_grants': [{
            'principal_type': 'group',
            'principal_id': '<group-id>',
            'permission': 'read'
        }]
    },
    'info': {'name': 'My MCP', 'description': 'Tools description'}
})

db.execute("UPDATE config SET value = ?, updated_at = ? WHERE key = ?",
    (json.dumps(current), int(time.time()), 'tool_server.connections'))
db.commit()

Then restart Open WebUI: docker restart openwebui

Users enable MCP tools per-chat via the ➕ → Integrations → Tools menu.

Pitfalls

yaml.dump drops top-level networks: section — When editing docker-compose.yml with Python's yaml.dump(), the networks: block is silently removed on every write. This happened 2x in one session (Jul 15). Verify with grep '^networks:' docker-compose.yml after every YAML edit. If missing, restore from .bak and append new services via cat >> instead. You can also append the networks block manually: echo -e '\nnetworks:\n litellm-net:\n driver: bridge' >> docker-compose.yml
Slim Python images lack health check tools — curl, ss, pgrep, and wget are NOT in python:3.13-slim. Skip Docker health checks or use pgrep -f server.py.
Port conflicts on app1 — Port 3000 is bound by Open WebUI. Check ss -tlnp | grep <port> before adding new MCP service ports.
Open WebUI stores MCP connections in SQLite — not in env vars or docker-compose. Adding an MCP server requires a DB update + restart at /var/lib/docker/volumes/openwebui_data/_data/webui.db, key tool_server.connections.
Open WebUI ConfigVar env vars are ignored after first launch — values persist in SQLite. Use UPDATE config SET value=... WHERE key=... for persistent config.
MCP connection type must be \"mcp\", not \"openapi\" — using OpenAPI type for MCP URLs causes infinite loading spinner.
host.docker.internal for Docker→host — when Open WebUI (Docker) needs to reach an MCP server on the same host, use http://host.docker.internal:<port>/mcp, not localhost.
Hermes config changes to mcp_servers require gateway restart — tools won't be available until restart.
HTTP transport needs SSE session negotiation — curl POST won't work. Use docker exec <name> python3 -c \"from fastmcp import Client...\" for verification.
Subagent timeout — default 600s is tight for MCP server builds. Subagents often return "next steps" without deploying — verify yourself or rebuild.

See references/browser-puppeteer-mcp.md for the full Playwright + browserless/chrome MCP pattern. See references/openwebui-config.md for Open WebUI SQLite configuration reference.

17 KiB Raw Permalink Blame History Unescape Escape

MCP Servers

Architecture

HTTP-Based MCP Server (FastMCP Pattern)

Build Steps

Testing

Performance

Troubleshooting

Immich Photo Map MCP Pattern

OSINT Person Intelligence Layer

Command-Based MCP Server Pattern

Remote MCP Server Pattern (OAuth)

Rate Limiting & TTL Caching Layer

Architecture

Implementation — ratelimit.py

Key decisions

Multi-Provider Search Routing Pattern

Remote Database MCP Data Source

Docker-Based MCP Deployment (alongside Open WebUI)

Pattern

Dockerfile (slim Python)

Health check pitfall — slim containers

Addressing

Verifying deployment

Wiring MCP into Open WebUI

Pitfalls

17 KiB

Raw Permalink Blame History

Implementation — `ratelimit.py`