# Service Health Check (no_agent watchdog)

A reusable pattern for no_agent cron jobs that monitor ALL critical services and only report on failure.

## Design

```bash
#!/bin/bash
# silent on success, report failures only

CHECKS=()
FAILED=0

# Each check:
command || { CHECKS+=("Service Name"); FAILED=1; }

if [ $FAILED -eq 1 ]; then
    echo "⚠️  Service Health Alert — $(date)"
    for c in "${CHECKS[@]}"; do echo "  ❌ $c"; done
    exit 1
fi
exit 0
```

## What to check

- Systemd services (caddy, mysql-tunnel)
- Docker containers (docker ps --format)
- Local HTTP endpoints (curl -sf)
- Remote reachability (ping through tunnel, SSH)
- Database connectivity (mysql -e "SELECT 1")
- API token validity (curl + verify response)

## Pitfalls

- **Must be no_agent: true** in cron config — LLM processing is wasted on binary health checks and causes typing indicators
- **Include the exit code** — exit 0 = silent, exit 1 = output delivered. Without the right exit code, cron can't tell success from failure
- **Test with `bash script.sh` and check `echo $?`** before deploying
- **MySQL checks need `--skip-ssl`** for local tunnel connections
- **Don't check too many things** — 10-12 checks is the sweet spot. More than 15 and the script gets brittle
- **SSH checks need `-o BatchMode=yes -o ConnectTimeout=5`** to avoid hanging on unreachable hosts