---
name: hermes-vision-backup
description: "Backup and restore Hermes auxiliary vision provider configuration so vision_analyze can be recovered after provider changes or profile resets."
version: 1.0.0
author: ShoNuff
platforms: [linux]
tags: [hermes, vision, backup, recovery, auxiliary]
---

# Hermes Vision Backup & Recovery

Hermes uses an **auxiliary vision model** for image analysis when the primary conversation model doesn't support native vision. This is configured via `auxiliary.vision.*` in `config.yaml`. If the vision provider breaks (API key change, provider outage, config reset), `vision_analyze` and `browser_vision` fail.

## What to Back Up

The vision configuration lives in two places:

### 1. Config.yaml — auxiliary.vision section

```yaml
auxiliary:
  vision:
    provider: "custom:name"     # or openrouter, google, nous, etc.
    model: "model-name"         # e.g. "gpt-4o", "claude-sonnet-4", etc.
```

### 2. The API key for the vision provider

The vision provider uses the same credential pools as the main provider — but if the vision provider is DIFFERENT from the main provider, its API key must be set separately.

## Current Setup (as of July 2026)

**Profile:** default
**Main provider:** admin-ai (custom, deepseek-chat via LiteLLM proxy)
**Vision provider:** admin-ai (same proxy, different model)
**Vision model:** `gemini-flash-latest` (via admin-ai — free tier, fast, vision-capable)
**Status:** ✅ Working (fixed: api_key had to be set explicitly — see pitfalls below)

### Recommended: Gemini Flash via admin-ai (Jul 12, 2026)

The best current vision setup uses Gemini Flash through the admin-ai LiteLLM proxy. Gemini has a generous free tier (60 req/min) and is already configured in admin-ai:

```bash
hermes config set auxiliary.vision.provider admin-ai
hermes config set auxiliary.vision.model gemini-flash-latest
```

No additional API key needed — admin-ai already has the Gemini key. Verify with:
```bash
curl -s <admin-ai-url>/v1/models | grep -i gemini
# Should show: gemini-flash-latest, gemini-pro-latest, etc.
```

**Note:** `vision.provider` (top-level) is NOT the same as `auxiliary.vision.provider`. The `vision_analyze` tool reads from `auxiliary.vision.*`. Setting only the top-level `vision.*` has no effect on image analysis.

### Verify vision is working

```python
# Test that the configured vision model accepts image inputs:
curl -s <admin-ai-url>/v1/chat/completions \
  -H "Authorization: Bearer <key>" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gemini-flash-latest",
    "messages": [{"role":"user","content":[
      {"type":"text","text":"describe this image"},
      {"type":"image_url","image_url":{"url":"https://example.com/test.png"}}
    ]}]
  }'
```

Look for a successful response containing `choices[0].message.content` to confirm the model supports vision through the proxy.

## Recovery Steps

### Quick fix — set a working vision provider

```bash
# Option A: Use a free/trial vision provider
hermes config set auxiliary.vision.provider openrouter
hermes config set auxiliary.vision.model openai/gpt-4o

# Option B: Use Anthropic (requires ANTHROPIC_API_KEY)
hermes config set auxiliary.vision.provider anthropic
hermes config set auxiliary.vision.model claude-sonnet-4

# Option C: Use Google Gemini (requires GOOGLE_API_KEY)
hermes config set auxiliary.vision.provider google
hermes config set auxiliary.vision.model gemini-2.0-flash
```

After setting: `/reset` (new session) for the change to take effect.

### Custom provider pattern (LiteLLM / Open WebUI proxy)

When vision is routed through a custom provider (e.g. admin-ai.itpropartner.com running LiteLLM), you MUST set ALL four fields:

```bash
hermes config set auxiliary.vision.provider admin-ai
hermes config set auxiliary.vision.base_url https://admin-ai.itpropartner.com/v1
hermes config set auxiliary.vision.model openrouter/openai/gpt-4o
hermes config set auxiliary.vision.api_key sk-...your-key...
```

The `api_key` field does NOT inherit from the main provider automatically — even if the main provider uses the same proxy. It must be set explicitly on the auxiliary config.

**Verifying the custom provider has a vision model available:**
```bash
KEY=<your-api-key>
curl -s <base-url>/models -H "Authorization: Bearer $KEY"
# Look for models containing: gpt-4o, gpt-4.1, claude-sonnet-4, gemini-2.0-flash, qwen-vl-plus, etc.
# Text-only models (deepseek-chat, etc.) will NOT work for vision.
```

### LiteLLM proxy: empty model list

If `/v1/models` returns an empty `data` array, the API key is wrong or the LiteLLM proxy isn't configured with model routing. Check:
1. The API key has access to the `/v1/models` endpoint
2. LiteLLM config has models listed in `model_list`
3. The vision model is exposed under the expected model ID (e.g. `openrouter/openai/gpt-4o`)

### Verify

```bash
hermes config check
hermes doctor --fix
```

Then test with a simple image in a new session.

## Backup Commands

```bash
# Save current vision config
hermes config | grep -A4 "auxiliary" > /root/.hermes/.backups/vision-config.txt

# Include in daily Hermes backup (already covered by hermes-backup.sh)
# Vision config is part of config.yaml which is already in the backup tarball
```

The `hermes-backup.sh` script already backs up the entire `~/.hermes/` config directory — so the vision config IS already backed up to Wasabi S3 daily. Recovery is just: restore config.yaml from S3, restart Hermes.

## Pitfalls

- **`auxiliary.vision.provider: auto`** means Hermes tries to auto-detect. If no vision provider is configured, `auto` returns nothing — silent failure.
- **Setting a vision provider requires a `/reset` or new session** — mid-conversation changes don't take effect.
- **If vision model is incompatible** with the provider (e.g. putting a text-only model in `auxiliary.vision.model`), it fails silently. Test after every change.
- **The vision provider can be different from the main provider** — main could be deepseek (no vision) while auxiliary vision is gpt-4o (has vision). This is the expected pattern.
- **`auxiliary.vision.api_key` does NOT inherit from the main provider**, even when both use the same provider name. The LiteLLM proxy sends `no-****required` / blank key if `api_key` is left empty on the auxiliary config. Must set it explicitly.
- **LiteLLM proxy models** require the full model string as passed from the proxy (e.g. `openrouter/openai/gpt-4o`, not just `gpt-4o`). Check `/v1/models` for the exact ID.