Initial skills documentation — 25 categories, all SKILL.md + references + scripts

skills/devops/cloudflare-dns-and-domains/SKILL.md

@@ -0,0 +1,255 @@
+---
+name: cloudflare-dns-and-domains
+description: "Manage Cloudflare DNS zones, DNS records, and domain registrations (registrar) through the Cloudflare API. Integration with IT Pro Partner portal for customer DNS management and domain lifecycle."
+version: 1.1.0
+author: Sho'Nuff
+tags: [cloudflare, dns, registrar, api, portal]
+related_skills: []
+---
+
+# Cloudflare DNS & Domain Management
+
+Manage Cloudflare DNS zones, DNS records, and domain registrations via the Cloudflare API v4. Used for the IT Pro Partner operations portal and Hermes integrations.
+
+## Related Reference
+
+**Before registering: research domain availability** — see `references/domain-availability-research.md` for the full workflow (WHOIS lookups, DNS NXDOMAIN checks, creative name generation, pricing across registrars, and pitfalls). This covers the **pre-registration** phase; this skill covers what happens **after** you own a domain.
+
+## API Authentication
+
+All requests use `Authorization: Bearer <token>` header. Tokens are scoped per-permission — never use a global API key.
+
+### Token scoping rules
+
+| Feature | Required Permission | Example token template |
+|---|---|---|
+| DNS record management | Zone → DNS → Edit | "Edit zone DNS" template |
+| View registered domains | Account → Registrar → Read | Add to existing DNS token |
+| Renew domains | Account → Registrar → Admin | Includes read + write + renew |
+| Register new domains | Account → Registrar → Admin | Also needs `can_register: true` on TLD |
+| Transfer domains | Account → Registrar → Admin | Includes transfer-in operations |
+| Access/Zero Trust | Account → Access: Apps and Policies → Edit | Add to existing DNS token at dashboard |
+
+**Start with DNS Edit + Registrar Read.** Upgrade to Admin only when purchasing/transferring domains is needed. For Access operations, edit the existing token from the dashboard to add the Access permission — no need to create a separate token.
+
+**PITFALL — Token generation produces invalid keys via Telegram:** Cloudflare API tokens contain mixed-case and numeric characters that can be mangled when pasted through Telegram. If `curl -s "https://api.cloudflare.com/client/v4/user/tokens/verify"` returns `"Invalid API Token"` after pasting, the token was corrupted in transit. Options: (a) have the user email the token to shonuff@ instead of Telegram, or (b) have them regenerate from the dashboard.
+
+### Verify token
+
+```bash
+curl -s -X GET "https://api.cloudflare.com/client/v4/user/tokens/verify" \
+  -H "Authorization: Bearer $TOKEN"
+```
+
+Returns `{"success": true}` with token ID, status, and permissions list.
+
+The token is stored in `~/.hermes/.env` as `CLOUDFLARE_API_TOKEN`.
+
+**PITFALL — service-health-check.sh uses literal `Bearer ***`:** The health check script (`/root/.hermes/scripts/service-health-check.sh`) previously used `$TOKEN` to capture the real token but then sent `Authorization: Bearer ***` (literal asterisks) instead of `Bearer $TOKEN`. Every 5 minutes it was hitting Cloudflare with literal asterisks and reporting the token as "unhealthy" — but the actual token was fine. If a user reports the Cloudflare token as "unhealthy" in health alerts, check this script first: ensure the variable name used in the `Authorization` header matches what was captured on the line above. Use a distinct variable name like `CLOUD_TOKEN` to avoid shadowing.
+
+## DNS Zone Management
+
+### List all zones
+
+```bash
+curl -s "https://api.cloudflare.com/client/v4/zones" \
+  -H "Authorization: Bearer $TOKEN" | python3 -c "
+import sys, json
+d = json.load(sys.stdin)
+print(f'Count: {d[\"result_info\"][\"total_count\"]}')
+for z in d['result']:
+    print(f'  {z[\"name\"]} ({z[\"plan\"][\"name\"]}) - {z[\"status\"]}')"
+```
+
+### List DNS records for a zone
+
+```bash
+ZONE_ID=$(curl -s "https://api.cloudflare.com/client/v4/zones" \
+  -H "Authorization: Bearer $TOKEN" | python3 -c "import sys,json;d=json.load(sys.stdin);z=[x['id'] for x in d['result'] if x['name']=='yourdomain.com'][0];print(z)")
+
+curl -s "https://api.cloudflare.com/client/v4/zones/$ZONE_ID/dns_records?per_page=5" \
+  -H "Authorization: Bearer $TOKEN" | python3 -c "
+import sys, json
+d = json.load(sys.stdin)
+for r in d['result']:
+    print(f'  {r[\"type\"]} {r[\"name\"]} -> {r[\"content\"]}')"
+```
+
+## Domain Registration (Registrar API)
+
+### Pitfall — Caddy + Cloudflare proxy creates 308 redirect loop (Jul 12, 2026)
+
+When Cloudflare proxy (orange cloud) is enabled for a subdomain pointing to a Caddy origin server, Cloudflare connects to the origin on HTTP (port 80), Caddy responds with a 308 redirect to HTTPS, Cloudflare follows the redirect back to its own address, and the browser sees an infinite redirect loop. The response header is `HTTP/1.1 308` with `Location: https://<same-domain>/`.
+
+**Fix options (in order of reliability):**
+1. **Path-based proxy** (most reliable) — Serve the service behind an existing working Caddy domain as a sub-path. E.g., `core.itpropartner.com` has `handle_path /vault/* { reverse_proxy localhost:8080 }`. No Cloudflare issues because the parent domain already has a valid cert and Cloudflare handles the path transparently.
+2. **DNS-only (gray cloud)** — Disable Cloudflare proxy for the record so traffic goes direct to the origin IP. Requires origin to have public ports open and a valid cert.
+3. **Cloudflare Origin Certificate** — Generate an origin cert in the Cloudflare dashboard and configure Caddy to use it (not tested on this infrastructure).
+
+**What does NOT work:** Caddy's automatic Let's Encrypt behind Cloudflare proxy. Let's Encrypt can't reach the origin through the proxy for the HTTP-01 challenge, and the ACME DNS-01 challenge requires Caddy to have the `dns.providers.cloudflare` module installed (not included in the default Caddy binary).
+
+### List all registered domains with expiry
+
+```bash
+ACCT_ID=$(curl -s "https://api.cloudflare.com/client/v4/accounts" \
+  -H "Authorization: Bearer $TOKEN" | python3 -c "import sys,json;print(json.load(sys.stdin).get('result',[{}])[0].get('id',''))")
+
+curl -s "https://api.cloudflare.com/client/v4/accounts/$ACCT_ID/registrar/domains" \
+  -H "Authorization: Bearer $TOKEN" | python3 -c "
+import sys, json
+from datetime import datetime, timezone
+d = json.load(sys.stdin)
+for dom in sorted(d.get('result',[]), key=lambda x: x.get('expires_at','')):
+    name = dom['name']
+    exp = dom.get('expires_at','?')
+    renew = dom.get('auto_renew','?')
+    if exp != '?':
+        exp_dt = datetime.fromisoformat(exp.replace('Z','+00:00'))
+        days = (exp_dt - datetime.now(timezone.utc)).days
+        print(f'{name:35s} {exp_dt.strftime("%b %d, %Y")} {days:3d}d  renew:{renew}')
+    else:
+        print(f'{name:35s} expires ?')"
+```
+
+### Single domain detail
+
+```bash
+curl -s "https://api.cloudflare.com/client/v4/accounts/$ACCT_ID/registrar/domains/germainebrown.com" \
+  -H "Authorization: Bearer $TOKEN" | python3 -c "
+import sys, json
+d = json.load(sys.stdin)
+dom = d.get('result',{})
+print(f'Name: {dom[\"name\"]}')
+print(f'Registered: {dom.get(\"registered_at\",\"?\")}')
+print(f'Expires: {dom.get(\"expires_at\",\"?\")}')
+print(f'Registrar: {dom.get(\"current_registrar\",\"?\")}')
+print(f'Auto-renew: {dom.get(\"auto_renew\",\"?\")}')
+print(f'Permissions: {dom.get(\"permissions\",[\"none\"])}')
+print(f'DNS zones: {dom.get(\"cloudflare_dns\",\"?\")}, registered: {dom.get(\"cloudflare_registration\",\"?\")}')"
+```
+
+### Check domain availability for purchase (requires Admin)
+
+**CORRECT endpoint (verified Jul 2026):**
+```bash
+curl -s "https://api.cloudflare.com/client/v4/accounts/$ACCT_ID/registrar/domains/example.com/check" \
+  -H "Authorization: Bearer $TOKEN"
+```
+
+Returns: `name`, `available`, `can_register`, `premium`, `supported_tld`, `fees.registration_fee`, `fees.renewal_fee`, `fees.transfer_fee`.
+
+**PITFALLS (all tested Jul 6, 2026):**
+- POST body `{"domain_name":"..."}` returns "Page not found" — do NOT use POST
+- Query param `?name=example.com` returns "Page not found" — do NOT use query params
+- The domain goes in the URL path like a resource lookup: `/domains/example.com/check`
+- This is a GET, not POST — cloudflare treats it as a resource lookup, not an RPC call
+
+**Verified pricing:** .com TLD costs $10.46/yr (registration, renewal, and transfer all same price). Cloudflare sells domains at cost with no markup.
+
+### Register a new domain
+
+**Endpoint (untested — from Cloudflare API docs):**
+```bash
+curl -s -X POST "https://api.cloudflare.com/client/v4/accounts/$ACCT_ID/registrar/domains/example.com/register" \
+  -H "Authorization: Bearer $TOKEN" \
+  -H "Content-Type: application/json" \
+  -d '{"registrant_contact": {...}, "period": 1}'
+```
+Requires Admin permission and contact details for WHOIS registrant.
+
+### Renew a domain
+
+Available with permissions: `['contact_read', 'contact_write', 'domain_renew', 'domain_transfer_out', 'nameserver_write', 'domain_delete']`
+
+```bash
+curl -s -X POST "https://api.cloudflare.com/client/v4/accounts/$ACCT_ID/registrar/domains/example.com/renew" \
+  -H "Authorization: Bearer $TOKEN" \
+  -H "Content-Type: application/json" \
+  -d '{"period": 1}'  # years
+```
+
+## Permissions reference
+
+A Registrar Read token shows per-domain `permissions`:
+
+```json
+['contact_read', 'contact_write', 'domain_renew', 'domain_transfer_out', 'nameserver_write', 'domain_delete']
+```
+
+Notable absence: no `domain_create` or `domain_register`. These require Admin level. The domain-level check (`/domains/{name}/check`) also requires Admin.
+
+## Cloudflare Access (Zero Trust)
+
+Cloudflare Access provides an authentication layer that sits in front of subdomains. Requests are checked for valid identity (email + optional 2FA) before reaching the origin server.
+
+### When to use
+- Replace Caddy basic_auth for internal dashboards
+- Add 2FA to existing services (CRM, DocuSeal)
+- Single sign-on across multiple services
+
+### Setup requirements
+- **Token scope:** The API token MUST include `Access: Apps and Policies:Edit` permission. The standard DNS-only token (Zone:DNS:Edit) returns "Authentication error" when trying to create Access applications.
+- **DNS proxy:** The subdomain must have Cloudflare proxy enabled (orange cloud) — Access cannot intercept traffic sent directly to the origin IP.
+
+### Quick setup path (when you have the right token)
+1. Enable orange cloud on the subdomain
+2. POST to `https://api.cloudflare.com/client/v4/accounts/{account_id}/access/apps` with name, domain, type: self_hosted, session_duration: 24h
+3. Add identity policies per user by email
+4. Optionally require 2FA in policy advanced rules
+
+**PITFALL — Setting require TOTP fails via API:** The `"require":[{"totp":{}}]` policy setting returns `access.api.error.invalid_request` when sent via the API — it expects a different require format that isn't documented in the API reference. **Drop the 2FA requirement from the API call** and configure it manually from the dashboard if needed.
+
+**PITFALL — Custom domain for Access login page:** Setting a custom domain (e.g. `auth.debtrecoveryexperts.com`) cannot be done via API — the `/access/custom_pages` endpoint returns "Method not allowed for this authentication scheme". Must be done via the dashboard at https://one.dash.cloudflare.com/ → Settings → Customization → Custom domain. Create a CNAME record pointing to `cloudflareaccess.com` first so Cloudflare can verify the domain.
+
+**PITFALL — Token refresh after permission update:** When a user adds permissions to an existing Cloudflare token from the dashboard, the token value regenerates and the old value becomes invalid. The new value must be saved to `~/.hermes/.env` under `CLOUDFLARE_API_TOKEN`. Like token creation, the value can be mangled in Telegram paste — if `curl -s "https://api.cloudflare.com/client/v4/user/tokens/verify"` returns "Invalid API Token" after the user pastes it, have them email it to shonuff@germainebrown.com instead.
+
+**PITFALL — Access apps need proxy (orange cloud) enabled:** Access cannot intercept traffic sent directly to the origin IP. The DNS record for the subdomain MUST have Cloudflare proxy enabled (orange cloud) before Access will work. Without it, users reach the origin server directly and bypass authentication entirely.
+
+### Manual setup (Cloudflare dashboard)
+Go to https://one.dash.cloudflare.com/ → Access → Applications → Add application → Self-hosted. Enter domain, set session duration, add email-based policies.
+
+### Debugging
+
+#### Verify policy membership
+
+Check who is authorized for a specific Access app:
+
+```bash
+ACCT="<account_id>"
+# Find the app ID by name
+curl -s "https://api.cloudflare.com/client/v4/accounts/$ACCT/access/apps" \
+  -H "Authorization: Bearer $TOKEN" | python3 -c "
+import sys, json
+d = json.load(sys.stdin)
+for app in d.get('result', []):
+    print(f'{app[\"name\"]:30s} {app[\"domain\"]:45s} {app[\"id\"]}')
+"
+
+# List policies for a specific app
+APP_ID="<access_app_id>"
+curl -s "https://api.cloudflare.com/client/v4/accounts/$ACCT/access/apps/$APP_ID/policies" \
+  -H "Authorization: Bearer $TOKEN" | python3 -c "
+import sys, json
+d = json.load(sys.stdin)
+for pol in d.get('result', []):
+    print(f'Policy: {pol.get(\"name\",\"\")}  Decision: {pol.get(\"decision\",\"\")}')
+    for inc in pol.get('include', []):
+        for k,v in inc.items():
+            if isinstance(v, dict):
+                print(f'  Include → {k}: {list(v.values())[0]}')
+            else:
+                print(f'  Include → {k}: {v}')
+"
+```
+
+The output shows which email domains or specific emails are allowed. DRE example: `email_domain: germainebrown.com` + `email_domain: yahoo.com` grants access to anyone @germainebrown.com or @yahoo.com.
+Free up to 50 users. No server-side config needed — works with any origin server (Caddy, nginx, Docker).
+
+The IT Pro Partner portal mockup at `/root/portal-mockup/dns.html` has two tabs:
+1. **DNS Zones** — zone listing with record editor
+2. **Registered Domains** — domain list with expiry dates, auto-renew status, days remaining, and customer links
+
+The portal displays real data from the Cloudflare API. Color coding:
+- Green: 90+ days to expiry
+- Amber: 30-90 days to expiry  
+- Red: Under 30 days or expired

skills/devops/cloudflare-dns-and-domains/references/dns-provider-map.md

@@ -0,0 +1,32 @@
+# DNS Management by Domain
+
+Not all domains use Cloudflare DNS. Some use SiteGround, which has no programmatic API access.
+
+## SiteGround-Managed Domains
+
+`itpropartner.com` DNS is managed at **SiteGround** (nameservers: `ns1.siteground.net`, `ns2.siteground.net`).
+
+**Implications:**
+- DNS updates must be done manually through the SiteGround control panel (Germaine's account)
+- Cloudflare API will return no results for these domains even with a valid API token
+- The `CLOUDFLARE_API_TOKEN` in .env has DNS permissions but only applies to Cloudflare-hosted zones
+
+**Discovery:** Confirmed Jul 10, 2026 — `host -t NS itpropartner.com` returned SiteGround nameservers. Previous attempts to use Cloudflare API for DNS record management on this domain silently failed.
+
+## Cloudflare-Managed Domains (verified)
+From zone listing Jul 10, 2026 — 13 zones. Notable:
+- `germainebrown.com` (MXroute for email, Cloudflare for DNS)
+- `forefrontwireless.com` / `forefrontwireless.net`
+- `iamgmb.com`
+- `debtrecoveryexperts.com`
+- `apextrackexperience.com`
+- `itpropartner.com` ← DNS at SiteGround, NOT Cloudflare despite having a zone entry
+
+## DNS Provider Detection Pattern
+
+Always verify nameservers before using the Cloudflare API:
+```bash
+host -t NS <domain>
+```
+
+If nameservers don't include `cloudflare.com`, the Cloudflare API won't work for DNS record management.

skills/devops/cloudflare-dns-and-domains/references/domain-availability-research.md

@@ -0,0 +1,145 @@
+# Domain Availability Research
+
+**When to use:** Before registering a domain — research which names are available, check WHOIS/DNS, generate creative alternatives, and compare pricing.
+
+---
+
+## Workflow: 3-Prong Validation
+
+Do NOT rely on a single check. The registry WHOIS database can lag or return stale data. Always run all three checks:
+
+### Check 1 — VeriSign WHOIS (authoritative for .com/.net/.edu)
+
+```bash
+whois <domain> 2>&1 | grep "No match for domain"
+```
+
+- **Available:** `No match for domain "EXAMPLE.COM"` — domain is not registered
+- **Taken:** Returns registry domain ID, creation date, expiry, name servers
+- **Rate note:** On fresh installs, `apt-get install -y whois` first
+- **Limitation:** Only covers .com, .net, .edu domains at the VeriSign registry level. For other TLDs (.io, .org, .co, .app, etc.), use a registrar's check endpoint or DNS lookup instead.
+
+### Check 2 — DNS NXDOMAIN verification
+
+```bash
+nslookup <domain> 2>&1 | grep "NXDOMAIN"
+```
+
+- **Available:** Returns `NXDOMAIN` — no DNS records exist
+- **Taken:** Returns one or more IP addresses
+- Works for ANY TLD, not just .com
+- A DNS lookup returning `NXDOMAIN` but WHOIS showing "No match" confirms availability from two angles
+
+### Check 3 — HTTP redirect/resolve check
+
+```bash
+curl -sk -o /dev/null -w '%{http_code} %{redirect_url}' https://<domain> 2>&1
+```
+
+- Sometimes a domain is registered but parked (HTTP 200/302 lander page) — means it's taken but possibly for sale
+- NXDOMAIN from Check 2 + timeout/slow from Check 3 = definitely available
+- A fast 200/302 response means someone owns it or a registrar has a landing page up
+
+### Important: DNS vs WHOIS discrepancies
+
+A domain can show "No match" in WHOIS yet resolve in DNS. This happens with:
+- Very recent registrations (WHOIS database hasn't propagated yet — up to 24h lag)
+- Domain was just registered but the WHOIS caching layer hasn't refreshed
+- **Resolution:** If WHOIS says "No match" but DNS resolves, the domain was likely registered within the last few hours. Mark as "likely taken — verify again in 24h."
+
+---
+
+## Generating Creative Domain Names
+
+### Formula patterns
+
+| Pattern | Example | Why It Works |
+|---------|---------|-------------|
+| `{theme}{keyword}.com` | `sharkfantasy.com` | Direct, brandable |
+| `{theme}ff.com` | `sharkattackff.com` | "FF" = fantasy football shorthand |
+| `{theme}draft.com` | `jawsdraft.com` | Highlights the draft mechanic |
+| `{theme}league.com` | `chumleague.com` | Wordplay: chum = shark bait |
+| `{theme}szn.com` | `jawsszn.com` | Sports seasons slang (SZN) |
+| `{wordplay}league.com` | `toothyleague.com` | Fun, playful, memorable |
+| `{compound}league.com` | `finomenalleague.com` | "Fin" + phenomenal |
+| `{anatomy}picks.com` | `dorsalpicks.com` | Shark anatomy + draft picks |
+| `{slang}league.com` | `sharkattackseason.com` | "Season" = sports league |
+| `{chum-themed}{keyword}.com` | `chumdraft.com`, `predapick.com` | Short, wordplay, edgy |
+
+### Criteria for good names
+- ≤15 characters ideal (fits URLs, social handles, easy to type)
+- No hyphens if possible (people forget them, look spammy)
+- Avoid numbers unless part of brand (e.g., Web3 names with 3)
+- Easy to spell after hearing it once (the "radio test")
+- Avoid trademarked terms (e.g., "Feeding Frenzy" — EA owns it)
+- Avoid names that sound like existing products or confuse the brand
+
+---
+
+## Registrar Pricing Reference (as of Jul 2026)
+
+| Registrar | .com (1yr) | Notes |
+|-----------|-----------|-------|
+| **Cloudflare** | **$10.46** | At-cost, no markup — best value. Registration/renewal/transfer all same price. |
+| Namecheap | ~$10–$14 | Often has first-year promos, reliable API |
+| Porkbun | ~$10–$12 | Good mid-tier, simple interface |
+| GoDaddy | ~$12–$18 | Renewals higher; good for premium/auction domains |
+| HugeDomains | Premium | Marketplace for already-registered domains, often $1K+ |
+
+**Best practice:** Use Cloudflare Registrar for new registrations if the site will use Cloudflare DNS anyway. If another registrar, transfer to Cloudflare later.
+
+### Premium/aftermarket domains
+- `fantasyfrenzy.com` → HugeDomains (likely $1K+)
+- `apexshark.com` → GoDaddy Auctions ($2,500 or $834/mo lease-to-own)
+- Before pursuing a premium domain, check if an available creative alternative exists first (usually does at $10.46)
+
+---
+
+## Venue-Specific Domain Availability Checks
+
+### Cloudflare Registrar API (requires token with Registrar Admin permission)
+
+```bash
+ACCT_ID=$(curl -s "https://api.cloudflare.com/client/v4/accounts" \
+  -H "Authorization: Bearer $TOKEN" | python3 -c "import sys,json;print(json.load(sys.stdin).get('result',[{}])[0].get('id',''))")
+
+curl -s "https://api.cloudflare.com/client/v4/accounts/$ACCT_ID/registrar/domains/example.com/check" \
+  -H "Authorization: Bearer $TOKEN"
+```
+
+Returns: `available`, `can_register`, `premium`, `supported_tld`, `fees.registration_fee`, `fees.renewal_fee`, `fees.transfer_fee`.
+
+**Pitfalls:**
+- This is GET, not POST. POST returns "Page not found"
+- Domain name goes in URL path, not as query param or request body
+- Requires `Account → Registrar → Admin` permission token (standard DNS-only token won't work)
+- See `cloudflare-dns-and-domains` umbrella skill for full token setup
+
+---
+
+## Deliverable Format
+
+After researching, compile a ranked report with:
+
+1. **SECTION A: Initial List** — the domains the user asked about, each with status + note
+2. **SECTION B: Creative Alternatives** — original ideas generated by the research, checked for availability
+3. **SECTION C: Top Recommendations** — 3-5 ranked picks with rationale
+4. **SECTION D: Strategy** — which to register now vs. consider vs. skip
+5. **Pricing** — Cloudflare at-cost as baseline ($10.46/.com), note premiums
+
+### Rating scale
+- ⭐⭐⭐⭐⭐ = Short (≤12 chars), brandable, descriptive, no issues
+- ⭐⭐⭐⭐ = Good option, slightly longer or less obvious
+- ⭐⭐⭐ = Decent but long or less catchy
+- ⭐⭐ = Functional but not ideal
+- ⭐ = Skip (taken, premium, confusing)
+
+---
+
+## Pitfalls
+
+- **WHOIS not installed** — on minimal Linux installs, `whois` may not be present. Run `apt-get install -y whois` first.
+- **WHOIS rate limits** — VeriSign applies rate limiting at high query volumes (exact threshold undocumented). If queries start failing with connection errors, space them out or use DNS as a lighter alternative. For 20+ domains, batch with a loop + 0.5s delay.
+- **Premium domain price shock** — a "taken" domain at a marketplace like GoDaddy or HugeDomains can cost $500-$5,000+. Always check creative alternatives first at $10.46.
+- **"No match" with DNS resolution** — domain was likely registered in the last few hours. Cross-check again in 24h.
+- **New TLDs (.app, .dev, .io, etc.)** — WHOIS check via VeriSign only covers .com/.net/.edu. For other TLDs, use DNS NXDOMAIN + Cloudflare check API + registrar's own lookup. Many new TLDs require HSTS preloading (.app, .dev) which can cause redirect issues with http-only test pages.