ippadmin/hermes-skills

Fork 0

Files

T

root 1b4fcd4427 Initial skills documentation — 25 categories, all SKILL.md + references + scripts

2026-07-15 17:42:20 -04:00

12 KiB

Raw Blame History

name, description, version, author, platforms, metadata

name

description

version

author

platforms

metadata

hetzner-server-provisioning

Provision Hetzner Cloud servers via API — create CPX/CX instances, inject SSH keys, choose datacenters, set up Ubuntu, and deliver credentials. Part of the IT Pro Partner infrastructure automation umbrella.

1.2.0

Sho'Nuff

linux

hermes

Hetzner Server Provisioning

Provision new Hetzner Cloud servers using the REST API. For IT Pro Partner infrastructure expansion (app3, Tony's VPS, etc.).

Prerequisites

Hetzner API token in ~/.hermes/.env as HETZNER_API_TOKEN
SSH key(s) registered in Hetzner project: itpp-infra (ID 114709791)
curl + python3 for API calls

API endpoint

POST https://api.hetzner.cloud/v1/servers
GET  https://api.hetzner.cloud/v1/ssh_keys
GET  https://api.hetzner.cloud/v1/images?type=system
GET  https://api.hetzner.cloud/v1/datacenters
GET  https://api.hetzner.cloud/v1/locations
GET  https://api.hetzner.cloud/v1/servers/{id}

Server types (common)

Type	vCPU	RAM	Disk	Former price	Current API price (Jul 2026)
CPX11	2	2 GB	40 GB	~€3.79	~€17.49/mo
CPX21	3	4 GB	80 GB	~€7.85	~€31.99/mo
CPX31	4	8 GB	160 GB	~€15.59	~€62.49/mo
CPX32	4	8 GB	160 GB	—	~€35.49/mo
CPX41	8	16 GB	240 GB	—	~€120.49/mo
CPX42	8	16 GB	320 GB	—	~€69.49/mo

⚠️ Prices shown are from live API — significantly inflated vs. historical rates. For new servers, strongly consider netcup as a cheaper alternative. The consolidation plan (move Hetzner workloads to netcup) avoids inflated pricing entirely.

Image IDs (Ubuntu)

161547269 — Ubuntu 24.04 (current)
161547270 — Ubuntu 24.04 (fallback)
103908130 — Ubuntu 22.04

Locations

Code	City	Latency to US East
ash	Ashburn, VA	~10ms
hil	Hillsboro, OR	~60ms
nbg	Nuremberg, DE	~90ms
fsn	Falkenstein, DE	~100ms
hel	Helsinki, FI	~120ms
sin	Singapore	~200ms

For US-based users, prefer ash (Ashburn). For European proximity, prefer fsn or nbg.

Provision payload

curl -s -X POST "https://api.hetzner.cloud/v1/servers" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "server-name",
    "server_type": "cpx11",
    "image": 161547269,
    "location": "ash",
    "ssh_keys": [114709791]
  }'

Important: Use "location": "ash" not "datacenter": "ash-dc1". The API rejects datacenter as a field. Use location with the two-letter city code.

Server swap (delete + recreate)

To upgrade from one server type to another (e.g. CPX11 -> CPX21):

Call DELETE /v1/servers/{id} and note the response status
Wait at least 15 seconds — the IP release is asynchronous and the Hetzner API enforces a per-project primary IP limit
If the create call returns "Primary IP limit exceeded", wait another 10 seconds and retry. The old IP may still be "in-flight"
The same IP address is typically reused once released — convenient for DNS, but SSH won't connect for another 20-30s while the new server boots

Pitfall — timing is tight. The "Primary IP limit exceeded" error is NOT an actual limit — it's a race condition where the old IP hasn't been released yet. Wait longer between delete and create. Do not change the IP or request a new one — reusing the freed IP is the goal.

Post-provision steps

Wait ~20s for the server to initialize (up to 45s for the first SSH connection to succeed)
SSH in with ssh -o StrictHostKeyChecking=no -i ~/.ssh/itpp-infra root@<ip>
Verify hostname and uptime

Run initial setup:

hostnamectl set-hostname <name>
echo "<name>" > /etc/hostname
sed -i "s/127.0.1.1.*/127.0.1.1 <name>/" /etc/hosts
apt-get update -qq && apt-get upgrade -y -qq
timedatectl set-timezone America/New_York
ufw --force enable
ufw default deny incoming
ufw default allow outgoing
ufw allow ssh
useradd -m -s /bin/bash <username> 2>/dev/null || true

Verify: hostnamectl, free -h, df -h /, ufw status verbose

Hermes deployment for a new user

When provisioning a VPS for someone else to use as a personal Hermes agent:

Server spec considerations

CPX11 (2C/2GB) is the cheapest option but has NO room for local models. Hermes + gateway fits, but any additional service (Docker, local LLM) will push it over.
CPX21 (3C/4GB) is the correct choice if there's any chance of running a local fallback model or Docker services. The extra $4.50/mo removes the "no room for anything else" constraint.
If in doubt, choose CPX21 over CPX11. The 4GB headroom is the difference between "runs Hermes" and "can experiment."

Model architecture for a new user

Primary model: An admin-ai API key provisioned by the main user (DeepSeek Chat for conversation, GPT-4o or Gemini for vision if needed)
Local fallback: A quantized Llama 3.2 1B (Q4_K_M, ~771MB) running as a systemd service on 127.0.0.1:8080
Failover: A heartbeat script (system cron, every 60s) that checks admin-ai health. If down, switches Hermes' provider to local. When admin-ai comes back, switches back and alerts.

Installing Hermes

curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash

Then configure ~/.hermes/config.yaml with the admin-ai provider pointing at https://admin-ai.itpropartner.com/v1 and the user's API key.

Local Llama model setup

# Download model
mkdir -p /root/models
curl -sL "https://huggingface.co/bartowski/Llama-3.2-1B-Instruct-GGUF/resolve/main/Llama-3.2-1B-Instruct-Q4_K_M.gguf" -o /root/models/llama-3.2-1b-q4.gguf

# Install Python deps (venv is required — Ubuntu 24 has PEP 668)
apt-get install -y python3.12-venv
python3 -m venv /opt/llama-venv
/opt/llama-venv/bin/pip3 install llama-cpp-python --break-system-packages

# Create server script at /usr/local/bin/llama-server.py
# (OpenAI-compatible HTTP server on :8080 using llama-cpp-python)

# Create systemd service
cat > /etc/systemd/system/llama-server.service << 'EOF'
[Unit]
Description=Local LLM fallback (Llama 3.2 1B)
After=network.target

[Service]
Type=simple
ExecStart=/opt/llama-venv/bin/python3 /usr/local/bin/llama-server.py
Restart=on-failure
RestartSec=5
User=root

[Install]
WantedBy=multi-user.target
EOF

systemctl daemon-reload
systemctl enable --now llama-server

Pitfall — PEP 668: Ubuntu 24 prevents pip install outside a venv. Either create a venv or pass --break-system-packages. The venv approach is cleaner. The python3.12-venv package IS required on minimal Hetzner Ubuntu images — it is NOT installed by default.

Pitfall — llama-server binary: Do NOT download the pre-built llama-server binary from GitHub releases. It often fails silently (downloads a corrupt file). Use pip install llama-cpp-python instead and run a Python wrapper script (/usr/local/bin/llama-server.py) that creates an HTTP server using the llama_cpp.Llama class. This is more reliable, easier to restart as a systemd service, and simpler to debug.

Heartbeat failover script

cat > /usr/local/bin/hermes-heartbeat.sh << 'EOF'
#!/bin/bash
# Checks admin-ai health. Falls back to local model if down.
CONFIG="$HOME/.hermes/config.yaml"
CURRENT=$(grep "^  provider:" "$CONFIG" | head -1 | awk "{print \$2}")

if curl -sf -o /dev/null --connect-timeout 5 https://admin-ai.itpropartner.com/v1/models -H "Authorization: Bearer $TOKEN"; then
  if [ "$CURRENT" != "admin-ai" ]; then
    sed -i "s/^  provider:.*/  provider: admin-ai/" "$CONFIG"
    echo "Switched back to admin-ai"
  fi
else
  if [ "$CURRENT" != "admin-ai" ]; then
    exit 0  # already on fallback
  fi
  if curl -sf -o /dev/null --connect-timeout 5 http://127.0.0.1:8080/v1/models; then
    sed -i "s/^  provider:.*/  provider: local/" "$CONFIG"
    echo "admin-ai down, switched to local Llama"
  else
    echo "BOTH models unreachable!"
  fi
fi
EOF
chmod +x /usr/local/bin/hermes-heartbeat.sh
echo "* * * * * /usr/local/bin/hermes-heartbeat.sh" | crontab -

Access model for other-user instances

When provisioning a Hermes instance for someone OTHER than the main user:

Their Hermes does NOT connect back to the main user's Hermes
The main user CAN connect to theirs for emergency SSH access only
They interact ONLY through Telegram — they never SSH in
Their server credentials are stored for emergency-only use by the main user
The welcome email should NOT include SSH IP, local model port, or server details
Their server details file goes in /root/.hermes/references/<name>-vps.txt (main user's reference only)

Welcome email pattern

When the main user asks you to draft and send a welcome email to a new Hermes user:

Send a draft to the main user for approval FIRST (BCC them)
After approval, send to the new user's email
Follow the email composition style guide from the email-workflows skill

The email should cover:

What Hermes is (personal AI agent on their own server)
Real examples of what the main user has accomplished with it
Email section (ask if they want their email set up; explicit "permission required" statement)
Backup & Safety (multi-layer, tested)
Three-step Telegram setup guide (BotFather, userinfobot, reply with info)
Commands reference (/queue, /retry, /undo, /clear, /title, /goal, /compress)
Minimal server details (primary model provided by main user, backup built in)

Verification

ssh -o StrictHostKeyChecking=no -i ~/.ssh/itpp-infra root@<ip> "hostname && uptime && echo OK"

Pitfalls

datacenter is not a valid field — Use location with the two-letter code. The API docs show datacenter but the create endpoint rejects it. Sending "datacenter": "ash-dc1" returns "Error: invalid input in field 'datacenter'". Fix: use "location": "ash".
Primary IP limit on re-provision — If you delete a server and immediately recreate with the same project, you may hit "Primary IP limit exceeded" for a few seconds while the old IP is released. The Hetzner API enforces a hard limit on primary IPs per project. Deleting frees the IP, but the release and recreation cycle has a ~5-10s delay. Wait at least 15 seconds (tested: 5s was too short, 10s was enough on retry) before retrying the create call, or the same IP may still be "in-flight" from the deletion. After a successful wait the same IP is typically reused.
Same IP reuse — When you delete and recreate, Hetzner reuses the freed IP within seconds. The new server gets the same IP address, which is convenient for DNS but confusing during the initialization window (SSH will refuse connection for ~10-20s while the new server boots).
SSH key injection: The API auto-injects keys by ID. No need for user-data scripts or post-provision key copying.
Server name uniqueness: Hetzner enforces unique names within a project. Choose something that won't collide.
Image IDs change with new Ubuntu releases — Verify the latest Ubuntu 24.04 image ID with GET /v1/images?type=system before provisioning.
No file command on fresh Ubuntu: Use ls -l or stat instead.
First SSH attempt may fail: Even after API says "running," the server may need 20-30s to boot and accept connections. Retry with a sleep loop.
python3.12-venv NOT installed on minimal Hetzner Ubuntu images — This package is NOT present by default on Hetzner Cloud Ubuntu 24.04 images. Attempting python3 -m venv without it fails silently. Always install it: apt-get install -y python3.12-venv. Otherwise the venv is created without pip and the bin/ directory has no pip3 or python3 symlinks.

12 KiB Raw Blame History