Initial skills documentation — 25 categories, all SKILL.md + references + scripts

2026-07-15 17:42:20 -04:00
commit 1b4fcd4427
976 changed files with 188344 additions and 0 deletions
@@ -0,0 +1,89 @@
+# Caddy / Tailscale Port 443 Conflict
+
+## Symptom
+
+`systemctl status caddy` shows:
+```
+listen tcp :443: bind: address already in use
+```
+
+## Root Cause
+
+`tailscaled` (the Tailscale daemon) binds port 443 on **its Tailnet IP** (e.g. `100.71.155.7:443`). When Caddy tries to bind `:443` (all interfaces — `0.0.0.0:443`), the kernel blocks it because `100.71.155.7:443` is already taken. The public IP (`152.53.192.33:443`) is completely free — Caddy just can't reach it via the blanket `:443` bind.
+
+## Diagnosis
+
+```bash
+# Check what's on port 443
+ss -tlnp | grep 443
+# → tailscaled  PID 897  bound to 100.71.155.7:443 (NOT 0.0.0.0:443)
+
+# Confirm Caddy config is trying to use all-interfaces :443
+grep ':443' /etc/caddy/Caddyfile
+
+# Confirm Tailscale serve is active
+tailscale serve status
+```
+
+## Resolution — Use `default_bind` in Caddyfile
+
+Tailscale binds to its specific Tailnet interface IP. Caddy can bind to the **public IP** instead, and both coexist:
+
+Add a global options block at the **top** of `/etc/caddy/Caddyfile`:
+
+```caddy
+{
+    default_bind 152.53.192.33
+}
+```
+
+This tells Caddy to bind ALL site blocks to the public IP only, avoiding the Tailscale interface entirely. Leases port 443 for public HTTPS without fighting Tailscale.
+
+**Verify Caddy parses the config before restarting:**
+
+```bash
+caddy validate --config /etc/caddy/Caddyfile
+# → "Valid configuration"
+
+# Format it
+caddy fmt --overwrite /etc/caddy/Caddyfile
+
+# Start it
+systemctl start caddy
+
+# Confirm it's listening
+systemctl is-active caddy
+# → active
+
+# Test a site
+curl -sS -o /dev/null -w 'HTTP %{http_code}' https://sign.itpropartner.com/
+# → HTTP 200
+```
+
+**After the fix, verify the health check passes:**
+```bash
+bash /root/.hermes/scripts/service-health-check.sh
+echo "Exit code: $?"  # Should be 0 (silent = all healthy)
+```
+
+## How It Works
+
+- Tailscale binds `100.71.155.7:443` (its Tailnet IP, interface `tailscale0`)
+- Caddy with `default_bind 152.53.192.33` binds `152.53.192.33:443` (the public IP, interface `eth0`)
+- No overlap. Both serve HTTPS on port 443 to their respective audiences.
+
+Without `default_bind`, Caddy tries `0.0.0.0:443` (all interfaces), which includes the Tailscale IP → conflict.
+
+## Resolution Options (ranked)
+
+| Option | Effort | Effect |
+|--------|--------|--------|
+| **`default_bind <public-ip>`** (recommended) | Add 3 lines to Caddyfile | Full Caddy + Tailscale coexistence on :443 |
+| **iptables redirect** from Tailscale IP | Moderate | Works if Tailscale isn't actively serving |
+| **Disable Tailscale :443** | `tailscale serve --https=443 off` | Loses Tailscale Serve HTTPS on that box |
+| **Run Caddy on alt port** (e.g., 8443) | Update Caddy config + DNS | `sign.itpropartner.com` needs custom port |
+| **Let Caddy fail** | Update health check to skip Caddy | Acceptable if services reachable another way |
+
+## Current State (Jul 11, 2026)
+
+On **Core** (netcup, 152.53.192.33): **Fixed — Caddy running with `default_bind 152.53.192.33`.** Both Caddy (public HTTPS) and Tailscale (tailnet HTTPS) coexist cleanly.