72hr audit: 4 new docs, port fixes, project-log updated

2026-08-08 12:55:30 -04:00
parent a269a17b1f
commit 11a1110b81
6 changed files with 438 additions and 3 deletions
@@ -0,0 +1,76 @@
+# Super Search v2.4.0 -- Client-ID Metrics Tracking
+
+**Created:** 2026-08-08
+**Service:** Super Search MCP (Core, port 8899)
+**Feature:** Client-ID tracking via Prometheus metrics + Grafana dashboard
+
+---
+
+## Overview
+
+Super Search v2.4.0 adds per-client usage tracking. A Starlette middleware intercepts the `X-Client-Id` header on every MCP call and increments Prometheus counters per client and per tool. Metrics are exposed at `:8899/metrics` and scraped by Prometheus every 30s. A Grafana dashboard visualizes usage.
+
+## Architecture
+
+```
+Client (hermes/intelsight/dre-osint/verdicttank) 
+    -> X-Client-Id header
+    -> Super Search Middleware (intercepts, increments Prometheus counter)
+    -> MCP tool handler
+    -> :8899/metrics (Prometheus endpoint)
+    -> Prometheus (Docker, scrapes 172.17.0.1:8899/metrics every 30s)
+    -> Grafana (Dashboard: "Super Search - Client Tracking" at /d/ffuktvmgcpkhse)
+```
+
+## Clients Tracked
+
+| Client | Purpose |
+|--------|---------|
+| `hermes` | Hermes Agent's own Super Search usage |
+| `intelsight` | IntelSight product backend |
+| `dre-osint` | Debt Recovery Experts skip tracing |
+| `verdicttank` | VerdictTank research |
+
+Fallback: calls without `X-Client-Id` header are logged as `anonymous`.
+
+## Key Changes
+
+### Super Search (server.py)
+- Middleware added: intercepts `X-Client-Id` header on `/mcp` POST
+- Prometheus counters: `ss_tool_calls_total{client, tool}`, `ss_tool_duration_seconds{client, tool}`
+- `/metrics` endpoint exposed on port 8899
+
+### Prometheus (prometheus.yml)
+- Job: `super-search`
+- Target: `172.17.0.1:8899` (Docker bridge to host)
+- Scrape interval: 30s
+- Config: `/root/docker/monitoring/prometheus/prometheus.yml`
+
+### Grafana
+- Dashboard UID: `ffuktvmgcpkhse`
+- Title: "Super Search - Client Tracking"
+- Access: `https://core:3002/d/ffuktvmgcpkhse`
+- Panels: tool calls per client, duration distribution, top tools
+
+### Firewall (UFW)
+- Rule: allow 172.17.0.0/16 to port 8899/tcp
+- Reason: Prometheus Docker container needs host access
+
+### Super Search Binding
+- Changed from `127.0.0.1:8899` to `0.0.0.0:8899`
+- Required because Docker containers (Prometheus) cannot reach 127.0.0.1 on the host
+
+## Verification
+
+```
+ss -tlnp | grep 8899   -> 0.0.0.0:8899 (bound to all interfaces)
+curl -s 172.17.0.1:8899/metrics | grep ss_tool  -> counters present
+ufw status | grep 8899  -> ALLOW 172.17.0.0/16
+```
+
+## Related Docs
+
+- Super Search Enhancement Plan: `/root/projects/itpp-infrastructure/docs/super-search-enhancement-plan.md`
+- Server: `/root/docker/super-search/server.py`
+- Systemd: `super-search.service`
+- Prometheus config: `/root/docker/monitoring/prometheus/prometheus.yml`