# Systematic AI Provider Key Testing

## Test protocol (2026-07-15 session)

When given multiple provider API keys to verify, test each one systematically:

### Step 1 — List models (lightweight, no cost)
```bash
curl -sS https://api.<provider>.com/v1/models \
  -H "Authorization: Bearer <key>" --max-time 10
```
Confirms: key is valid, provider is reachable, models are available.

### Step 2 — Chat completion (real cost)
```bash
curl -sS -X POST https://api.<provider>.com/v1/chat/completions \
  -H "Authorization: Bearer <key>" \
  -H "Content-Type: application/json" \
  -d '{"model":"<smallest-model>","messages":[{"role":"user","content":"Say OK if working"}],"max_tokens":10}'
```
Use the smallest/cheapest model. Confirms: completions actually work, no billing/tier blocks.

### Step 3 — Classify result
| Result | Meaning |
|---|---|
| Models 200 + Completions 200 | ✅ Fully operational |
| Models 200 + Completions 4xx | ⚠️ Key valid but billing/tier blocks completions |
| Models 401 | ❌ Key invalid/expired |
| Completions 200 but model wrong | ⚠️ List models first, use exact model ID |

### Gotcha — model name mismatch
Some APIs return model IDs that differ from documentation. Always `GET /models` first and use the EXACT `id` field, never assume model names.

### Storage
All keys → `/root/.hermes/.env` (chmod 600), one `PROVIDER_API_KEY=<value>` per line.

## AI21-specific gotcha (Jul 2026)
AI21 API keys come in two formats — the user key prefix may differ from what works. The original key returned "Authentication required" while a different key format worked. Always test immediately after receiving a key, don't assume one format works.