# Caddy Proxy Port Mismatch — Deployment Tracking Pitfall

## Problem
A Caddy reverse proxy entry points to a different port than the service is actually running on. The service has been running for days (sometimes via systemd), but the site returns 502 or ERR_CONNECTION_REFUSED because Caddy is proxying to a dead port.

## Root Cause
When a service is deployed, three things must stay in sync:
1. The **systemd service / Docker container** port (e.g., `--port 8083`)
2. The **systemd unit file** ExecStart line (e.g., `uvicorn server:app --host 0.0.0.0 --port 8083`)
3. The **Caddyfile** `reverse_proxy` target (e.g., `reverse_proxy 127.0.0.1:8083`)

If one of these is updated without updating the others, the port drifts. The most common failure: Caddy proxies to port A (from an old config or manual edit), but the actual service runs on port B.

## Example (shark.iamgmb.com, Jul 12, 2026)
- Caddy had: `reverse_proxy 127.0.0.1:8081`
- Actual backend: `systemd service → port 8083` (confirmed by `journalctl` showing "Uvicorn running on http://0.0.0.0:8083")
- Result: shark.iamgmb.com returned 502 for 23+ hours despite the backend being active

## Diagnosis
```bash
# Check what Caddy proxies to
grep -A2 'shark\.iamgmb\.com' /etc/caddy/Caddyfile

# Check what port the service actually binds
journalctl -u shark-game.service --no-pager | grep -i 'running on\|listening on\|port'

# Check if anything responds on Caddy's target port
ss -tlnp | grep ':8081'    # empty = no one listening
ss -tlnp | grep ':8083'    # should show python3
```

## Fix
```bash
sed -i 's/reverse_proxy 127.0.0.1:8081/reverse_proxy 127.0.0.1:8083/' /etc/caddy/Caddyfile
caddy fmt --overwrite /etc/caddy/Caddyfile
caddy validate --config /etc/caddy/Caddyfile
systemctl reload caddy
```

## Prevention
When deploying or updating any service behind Caddy:
1. Verify the port in the systemd unit file matches `ExecStart`
2. Verify the port in Caddyfile matches the service port
3. After any change, confirm with `ss -tlnp | grep ':PORT'` that the service is listening
4. Use `curl -sS -o /dev/null -w '%{http_code}' https://domain.com/` to confirm the site responds