# Hetzner Snapshot Automation

Weekly snapshots of all running servers in a Hetzner Cloud account. Part of the provider-level backup layer (layer 1 in the three-layer model).

## Script

File: `snapshot-hetzner.py` at `/root/.hermes/scripts/snapshot-hetzner.py`

Reads the API token from a companion file `.hetzner_token` in the same directory (avoids shell escaping issues with special characters in the token).

## API Token Handling

The Hetzner Cloud API token can contain special characters (`$`, `!`, etc.) that break shell-based environment variable passing. **Do not use `export TOKEN=...` in bash** — the shell will mangle it.

**Pattern:** Store the token in a dedicated file, have the script read it directly:

```python
script_dir = os.path.dirname(os.path.abspath(__file__))
token_file = os.path.join(script_dir, ".hetzner_token")
with open(token_file) as f:
    TOKEN = f.read().strip()
```

Token file permissions: `chmod 600`.

## Setup

1. Generate a Hetzner Cloud API token (Read & Write) from the Hetzner console
2. Write it to `/root/.hermes/scripts/.hetzner_token` (one line, no trailing newline via Python write)
3. `chmod 600 /root/.hermes/scripts/.hetzner_token`
4. Test: `python3 /root/.hermes/scripts/snapshot-hetzner.py`
5. Cron job handles weekly runs (Monday at 5:00 UTC = 1:00 AM ET)

## What it does

- Lists all servers in the account via GET `/v1/servers`
- For each running server, POSTs a `create_image` action with `type=snapshot`
- Snapshot description: `auto-weekly-{server-name}-{date}`
- Skips stopped/offline servers

## Cron schedule

```
0 5 * * 1  (Mondays at 5:00 UTC / 1:00 AM ET)
```

Via no_agent cron job, silent output (watchdog pattern).

## Pitfalls

- **Token expires** — Hetzner tokens can have expiry dates. If snapshots start failing with 401, regenerate the token and update the file.
- **Writing the token file via echo truncates it** — Bash `echo "$TOKEN" > file` can silently truncate tokens containing `$`, `!`, backticks, or other shell-special characters. The file may contain fewer characters than the actual token, causing silent auth failures. Always write the token via Python:
  ```python
  token = 'paste-the-full-token-here'
  with open('/path/to/.hetzner_token', 'w') as f:
      f.write(token)
  print(f'Wrote {len(token)} chars')
  ```
  Verify the length matches the token in the Hetzner console.
- **Snapshot names are unique** — A snapshot with description `auto-weekly-{name}-{date}` allows easy identification in the Hetzner console. Hetzner allows up to 10 snapshots per server.
- **Billing** — Hetzner charges for snapshot storage (based on disk size). CPX11 (40GB) snapshots cost ~$0.01-0.02 each. Full account backup of 9 servers = negligible monthly cost.