46 lines
2.1 KiB
Markdown
46 lines
2.1 KiB
Markdown
# Caddy Proxy Port Mismatch — Deployment Tracking Pitfall
|
|
|
|
## Problem
|
|
A Caddy reverse proxy entry points to a different port than the service is actually running on. The service has been running for days (sometimes via systemd), but the site returns 502 or ERR_CONNECTION_REFUSED because Caddy is proxying to a dead port.
|
|
|
|
## Root Cause
|
|
When a service is deployed, three things must stay in sync:
|
|
1. The **systemd service / Docker container** port (e.g., `--port 8083`)
|
|
2. The **systemd unit file** ExecStart line (e.g., `uvicorn server:app --host 0.0.0.0 --port 8083`)
|
|
3. The **Caddyfile** `reverse_proxy` target (e.g., `reverse_proxy 127.0.0.1:8083`)
|
|
|
|
If one of these is updated without updating the others, the port drifts. The most common failure: Caddy proxies to port A (from an old config or manual edit), but the actual service runs on port B.
|
|
|
|
## Example (shark.iamgmb.com, Jul 12, 2026)
|
|
- Caddy had: `reverse_proxy 127.0.0.1:8081`
|
|
- Actual backend: `systemd service → port 8083` (confirmed by `journalctl` showing "Uvicorn running on http://0.0.0.0:8083")
|
|
- Result: shark.iamgmb.com returned 502 for 23+ hours despite the backend being active
|
|
|
|
## Diagnosis
|
|
```bash
|
|
# Check what Caddy proxies to
|
|
grep -A2 'shark\.iamgmb\.com' /etc/caddy/Caddyfile
|
|
|
|
# Check what port the service actually binds
|
|
journalctl -u shark-game.service --no-pager | grep -i 'running on\|listening on\|port'
|
|
|
|
# Check if anything responds on Caddy's target port
|
|
ss -tlnp | grep ':8081' # empty = no one listening
|
|
ss -tlnp | grep ':8083' # should show python3
|
|
```
|
|
|
|
## Fix
|
|
```bash
|
|
sed -i 's/reverse_proxy 127.0.0.1:8081/reverse_proxy 127.0.0.1:8083/' /etc/caddy/Caddyfile
|
|
caddy fmt --overwrite /etc/caddy/Caddyfile
|
|
caddy validate --config /etc/caddy/Caddyfile
|
|
systemctl reload caddy
|
|
```
|
|
|
|
## Prevention
|
|
When deploying or updating any service behind Caddy:
|
|
1. Verify the port in the systemd unit file matches `ExecStart`
|
|
2. Verify the port in Caddyfile matches the service port
|
|
3. After any change, confirm with `ss -tlnp | grep ':PORT'` that the service is listening
|
|
4. Use `curl -sS -o /dev/null -w '%{http_code}' https://domain.com/` to confirm the site responds
|