diff --git a/CHANGELOG.md b/CHANGELOG.md index b48118c..d260a02 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,7 +2,42 @@ --- -## 2026-07-25 — Project Inception & API Connection +## 2026-07-25 (PM) — Multi-Source Architecture Scoped + +**Scope expansion:** The portal at my.forefrontwireless.com will aggregate data from four sources, not just Splynx: + +### Data Sources + +| Source | Purpose | Auth | Location | +|--------|---------|------|----------| +| **Splynx v4.2** | Billing, CRM, customers, invoices, services | HMAC-SHA256 signature | Self-hosted at portal.forefrontwireless.com | +| **UISP (UNMS)** | Network management, tower/AP inventory, link quality | API token | app2 Docker (unms.forefrontwireless.com) | +| **MikroTik RouterOS** | Real-time bandwidth, DHCP leases, queue stats | REST API or SSH | 5 × CCR2004 routers on network | +| **Zabbix / SNMP** | Infrastructure monitoring, SNMP for non-UISP devices, historical graphs | Zabbix API | Currently on-prem; may move to cloud | + +### Architecture Decision +The FastAPI backend is now an **aggregation layer** — read-only adapters for each source, normalizing data into a unified customer view. The backend never writes to Splynx/UISP/MikroTik/Zabbix directly; all mutations stay in the native tools. + +### Build Phases + +| Phase | Sources | V1? | +|:---:|---|---| +| 1 | Splynx (billing + CRM) | Yes — core self-service | +| 2 | MikroTik (router telemetry) | Yes — data usage, device lists | +| 3 | UISP (network management) | V2 — network status, outage awareness | +| 4 | Zabbix/SNMP (infra monitoring) | V2+ — SLA dashboards, historical graphs | + +### Zabbix Migration Options +- **Recommended:** Cloud Zabbix + on-prem proxy — SNMP stays local, server moves to managed infra +- **Alternative:** Keep on-prem, API tunnel via Tailscale — zero cost, simple +- **For later:** Decision deferred until portal reaches Phase 4 + +### Page-to-Source Mapping +Every customer-facing feature mapped to its backend data provider — see README Data Aggregation Model section. + +--- + +## 2026-07-25 (AM) — Splynx API Connection **Context:** Germaine shared Splynx API credentials for Forefront Wireless, his WISP company. The self-hosted Splynx instance (v4.2) runs at `portal.forefrontwireless.com/admin/`. The goal is to build a branded customer portal at `my.forefrontwireless.com`. @@ -37,11 +72,3 @@ Key: 5a8dc490716424f435306f2c18b4fdcb Secret: 5773f779e5ae5aac965845a66bd857a0 Base: https://portal.forefrontwireless.com/api/2.0/ ``` - -### Next Steps (When Ready) - -1. Register `my.forefrontwireless.com` DNS -2. Build FastAPI backend with Splynx proxy layer -3. Build React dashboard -4. Stripe integration for payments -5. Deploy on app3 via CloudPanel diff --git a/README.md b/README.md index 2f40e47..19e3c04 100644 --- a/README.md +++ b/README.md @@ -5,71 +5,206 @@ > **Domain:** my.forefrontwireless.com > **Companion project:** IntelSight (shared auth — same Super Search/deepseek infra) -Self-service customer portal for Forefront Wireless, a WISP operating in Texas. Replaces or augments the existing Splynx customer portal with a modern, branded experience that directly consumes the Splynx API. +Self-service customer portal for Forefront Wireless, a WISP operating in Texas. Aggregates data from four distinct sources — Splynx (billing/CRM), UISP/UNMS (network management), MikroTik (router telemetry), and Zabbix/SNMP (infrastructure monitoring) — into a single branded customer experience. --- -## Architecture +## Architecture — Multi-Source Data Aggregation ``` -┌─────────────────┐ ┌──────────────────┐ ┌────────────────────────┐ -│ my.forefront │────▶│ FastAPI backend │────▶│ Splynx API v2.0 │ -│ wireless.com │ │ (api subdomain) │ │ portal.forefront... │ -│ (React SPA) │ │ Flask or FastAPI │ │ /api/2.0/ │ -└─────────────────┘ └──────────────────┘ └────────────────────────┘ - │ - ┌─────────▼──────────┐ - │ Splynx self-hosted │ - │ v4.2 │ - │ Forefront Wireless │ - └────────────────────┘ +┌──────────────────────────────────────────────────────────────────────────┐ +│ my.forefrontwireless.com │ +│ (React SPA) │ +└──────────────────────────────┬───────────────────────────────────────────┘ + │ + ▼ +┌──────────────────────────────────────────────────────────────────────────┐ +│ FastAPI Aggregation Backend │ +│ (api.forefrontwireless.com) │ +│ │ +│ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────────────┐ │ +│ │ Splynx │ │ UISP │ │ MikroTik │ │ Zabbix / SNMP │ │ +│ │ Adapter │ │ Adapter │ │ Adapter │ │ Adapter │ │ +│ └────┬─────┘ └────┬─────┘ └────┬─────┘ └────────┬─────────┘ │ +└───────┼──────────────┼──────────────┼─────────────────┼──────────────────┘ + │ │ │ │ + ▼ ▼ ▼ ▼ +┌──────────────┐ ┌──────────┐ ┌──────────────┐ ┌──────────────┐ +│ Splynx v4.2 │ │ UISP │ │ MikroTik │ │ Zabbix │ +│ Self-hosted │ │ (UNMS) │ │ RouterOS │ │ (existing │ +│ portal.ffw │ │ app2 │ │ REST API │ │ on-prem or │ +│ .com/admin/ │ │ Docker │ │ 5 × CCR2004 │ │ cloud) │ +└──────────────┘ └──────────┘ └──────────────┘ └──────────────┘ + billing/CRM network router stats infra health + customers devices bandwidth SNMP data + invoices links DHCP leases device uptime + services outages WiFi clients alerts ``` -**Stack:** React SPA → FastAPI → Splynx API v2.0 -**Auth:** Signature-based HMAC-SHA256 (nonce + key, secret-driven) -**API Key:** `5a8dc490716424f435306f2c18b4fdcb` (full permissions) -**Hosting:** CloudPanel on app3 (same as other ITPP portals) +**Stack:** React SPA → FastAPI aggregation backend → 4 data-source adapters +**Hosting:** CloudPanel on app3 (152.53.241.111) +**Auth pattern:** Each adapter uses the native auth for its source; backend handles user auth separately --- -## Splynx API — Proven Endpoints +## Data Source 1: Splynx (Billing & CRM) + +**URL:** https://portal.forefrontwireless.com/api/2.0/ +**Version:** v4.2, self-hosted +**Auth:** Signature-based HMAC-SHA256 (key + nonce, secret-driven) + +### What It Provides +- Customer accounts (43 total, 33 active) +- Service plans & active subscriptions +- Invoices, payments, transaction history +- Internet tariffs (9 plans, $59.95–$299.95/mo) +- Support tickets +- Router inventory (5 × MikroTik CCR2004) ### Authentication ``` -POST /api/2.0/admin/auth/tokens -Body: {"auth_type":"api_key","key":"...","nonce":,"signature":"..."} -→ Returns access_token (30-min expiry) + refresh_token - -OR: Direct signature header (preferred for server-to-server) Authorization: Splynx-EA (key=K&nonce=N&signature=S) Nonce: round(microtime * 100) — hundredths of a second Signature: strtoupper(hash_hmac('sha256', nonce . key, secret)) ``` -### Working Endpoints +### Proven Endpoints | Method | Path | Description | Status | |--------|------|-------------|:---:| -| GET | `/admin/customers/customer` | List all customers (43 total, 33 active) | ✓ | -| GET | `/admin/customers/customer/{id}` | Single customer detail | ✓ | -| GET | `/admin/customers/customer-internet-services/{customer_id}` | Services for customer | ✓ | -| POST | `/admin/customers/customer-internet-services` | List with filters (needs `customer_id` in body) | ⚠ 405 on GET | -| GET | `/admin/networking/routers` | All routers (5 CCR2004s) | ✓ | -| GET | `/admin/tariffs/internet` | Internet plans (9 plans, $59.95–$299.95) | ✓ | -| GET | `/admin/finance/invoices` | Invoice list | ✓ | -| GET | `/admin/finance/payments` | Payment records | ⚠ untested | -| GET | `/admin/finance/transactions` | Transaction history | ⚠ untested | -| GET | `/admin/support/tickets` | Support tickets | ⚠ untested | +| GET | `/admin/customers/customer` | List all customers | ✓ | +| GET | `/admin/customers/customer/{id}` | Single customer | ✓ | +| GET | `/admin/customers/customer-internet-services/{customer_id}` | Customer services | ✓ | +| GET | `/admin/networking/routers` | Router inventory | ✓ | +| GET | `/admin/tariffs/internet` | Internet plans | ✓ | +| GET | `/admin/finance/invoices` | Invoices | ✓ | +| GET | `/admin/finance/payments` | Payments | ⚠ untested | +| GET | `/admin/finance/transactions` | Transactions | ⚠ untested | +| GET | `/admin/support/tickets` | Tickets | ⚠ untested | -### Inventory Snapshot (July 2026) +--- -| Resource | Count | -|----------|:---:| -| Total customers | 43 | -| Active subscribers | 33 | -| Disabled accounts | 9 | -| Routers | 5 × MikroTik CCR2004 | -| Internet tariffs | 9 plans | +## Data Source 2: UISP / UNMS (Network Management) + +**Current location:** app2 (152.53.39.202), Docker deployment +**Notes:** UNMS was renamed to UISP. Project docs use UISP for new work, UNMS for legacy references and config paths. + +### What It Provides +- Tower/AP inventory and status +- Wireless link quality (signal, capacity, uptime) +- Device firmware versions +- Network topology +- Outage/alert history +- Client device associations + +### API Access Options + +| Method | Auth | Use Case | +|--------|------|----------| +| UISP REST API | API token (Settings → Users → API Access) | Read-only dashboard data | +| Local WebSocket | Internal, via Nginx 443 | Device communication only | +| Direct MongoDB | `mongo --port 27117 ace` | Emergency read-only inventory | + +### Key UISP Resources +- **Backup sync:** Daily 6 AM via `unms-backup-sync.sh` → `s3://hermes-vps-backups/unms-backups/` +- **Port architecture:** Internal 443 = web UI (proxy externally), 8089 = WS tunnel (NEVER proxy — returns 426) +- **Container recovery:** `device-ws` containers may need manual start after backup restore +- **Vault key:** Destroy/recreate on migration; key is NOT stored in any config file + +--- + +## Data Source 3: MikroTik RouterOS (Direct Device API) + +**Devices:** 5 × MikroTik CCR2004 (known from Splynx router list) +**Auth:** SSH key or RouterOS REST API credentials +**Protocol:** MikroTik REST API (port 443/8443) or SSH command line + +### What It Provides +- Real-time bandwidth per interface +- DHCP lease table (which customer has which IP) +- Connected WiFi clients (via CAPsMAN or built-in wireless) +- Queue/simple queue stats (per-customer bandwidth limits) +- Interface status, errors, uptime +- ARP table +- RouterOS version and firmware + +### Integration Approach +Each router needs REST API enabled and a read-only API user. The backend adapter polls all 5 routers on a schedule and caches results. Key endpoints: + +| REST Resource | What It Returns | +|---------------|-----------------| +| `/rest/interface` | All interfaces + current RX/TX rates | +| `/rest/ip/dhcp-server/lease` | Active DHCP leases (MAC ↔ IP ↔ hostname) | +| `/rest/queue/simple` | Per-customer bandwidth queues | +| `/rest/system/resource` | CPU, memory, uptime | +| `/rest/ip/arp` | ARP table | + +**Pitfall:** MikroTik REST API returns data in RouterOS-specific data structures (not standard JSON). Lists may be nested, IDs are hex strings, timestamps are RouterOS epoch. The adapter must normalize all of this. + +--- + +## Data Source 4: Zabbix / SNMP (Infrastructure Monitoring) + +**Current state:** Zabbix server running on-premises within the Forefront network +**Future direction:** Move to cloud (netcup/app3) or run remote Zabbix proxy + +### What It Provides +- SNMP data from devices NOT in UISP (switches, power equipment, environmental sensors) +- Historical bandwidth graphs (interface counters via SNMP) +- Device uptime, CPU, temperature +- Custom alert triggers (tower power loss, temp thresholds) +- SLA reporting data + +### Integration Options + +| Option | Pros | Cons | +|--------|------|------| +| **Zabbix API** | Direct access to all data, triggers, graphs | Requires Zabbix reachable from portal backend | +| **Remote Zabbix proxy** | Proxy polls on-prem, reports to cloud server | Extra component to manage | +| **Direct SNMP from backend** | No Zabbix dependency, simpler | Have to reimplement alerting logic | +| **Zabbix → InfluxDB → Portal** | Time-series optimized, Grafana-compatible | Additional database, more moving parts | + +### Recommended Architecture (when ready) +``` +Forefront Network Cloud (netcup/app3) +┌─────────────┐ ┌──────────────┐ ┌────────────────┐ +│ SNMP devices│────▶│ Zabbix Proxy │────▶│ Zabbix Server │ +│ (switches, │ │ (lightweight │ │ (full instance)│ +│ PDUs, etc) │ │ collector) │ │ │ +└─────────────┘ └──────────────┘ └───────┬────────┘ + │ Zabbix API + ▼ + ┌────────────────┐ + │ Portal Backend │ + │ (read-only) │ + └────────────────┘ +``` + +**Zabbix proxy** stays on-prem collecting SNMP; Zabbix server moves to cloud. Portal queries the cloud server's API. No firewall holes needed inbound — proxy initiates outbound connection to server. + +--- + +## Data Aggregation Model + +The FastAPI backend acts as a **read-only aggregation layer** — it never writes to Splynx, UISP, MikroTik, or Zabbix directly. All mutations go through the respective native tools. + +``` +Customer-facing page Data sources behind it +───────────────────── ────────────────────── +Dashboard Splynx (account + bill) + ├─ Current bill → Splynx invoices + ├─ Data usage → MikroTik queues + ├─ Network status → UISP device state + └─ Any outages? → Zabbix triggers + +Services Splynx + MikroTik + ├─ Plan → Splynx tariff + ├─ IP address → MikroTik DHCP lease + └─ Equipment → UISP device list + +Support Splynx tickets + UISP alerts +Network Status UISP topology + MikroTik interfaces +``` --- @@ -80,29 +215,60 @@ Signature: strtoupper(hash_hmac('sha256', nonce . key, secret)) - [ ] Dashboard: active services, current bill, data usage - [ ] View & pay invoices (Stripe integration) - [ ] Update contact info, phone, email -- [ ] Change WiFi password (via MikroTik API → router) -### V2 — Enhanced +### V2 — Network Visibility +- [ ] Real-time bandwidth usage (MikroTik queue stats) +- [ ] Connected devices (MikroTik DHCP + ARP) +- [ ] Network status indicator (UISP device health) +- [ ] Outage notifications (Zabbix triggers → email/SMS) +- [ ] Support ticket submission & tracking (Splynx) + +### V3 — Enhanced - [ ] Service upgrade/downgrade with proration -- [ ] Support ticket submission & tracking -- [ ] Outage notifications (SMS opt-in) -- [ ] Referral program - -### V3 — Operational -- [ ] Usage graphs (daily/weekly/monthly bandwidth) -- [ ] Auto-pay setup +- [ ] Usage graphs — daily/weekly/monthly (MikroTik + SNMP) +- [ ] Auto-pay setup (Stripe) - [ ] Seasonal hold / vacation pause -- [ ] Equipment return workflow - [ ] Speed test integration +### V4 — Operational +- [ ] Equipment return workflow +- [ ] Referral program +- [ ] SLA dashboard (Zabbix data) +- [ ] Network health map (UISP topology) + +--- + +## Integration Dependencies & Build Order + +Because each data source has its own setup complexity, the build should be phased: + +| Phase | Sources | Complexity | Dependencies | +|:---:|---|:---:|---| +| **1. Splynx** | Billing + CRM | Low | API already proven, key configured | +| **2. MikroTik** | Router telemetry | Medium | Need REST API user on each CCR2004 | +| **3. UISP** | Network management | Medium | Need UISP API token; app2 is live | +| **4. Zabbix/SNMP** | Infrastructure monitoring | High | Decision needed: cloud migration vs proxy | + +Phases 1–2 can be V1. Phases 3–4 unlock V2 and beyond. + --- ## Splynx Quirks (Lessons Learned) 1. **Signature auth is required** — Basic auth returns 401 unless "Unsecure access" is enabled on the key. Signature uses HMAC-SHA256 with a microtime nonce. 2. **Services need customer_id in URL** — `GET /admin/customers/customer-internet-services/{id}` works, but `GET /admin/customers/customer-internet-services?customer_id=2` returns 405. -3. **Tariff names may need separate resolution** — The list endpoint may return tariff IDs only; names require a follow-up call to `/admin/tariffs/internet/{id}`. -4. **Data shown is filtered** — Only what the API key has permission for. Current key has full permissions. +3. **Tariff names may need separate resolution** — The list endpoint may return tariff IDs only; names require a follow-up call. +4. **Data is filtered by permissions** — Current API key has full permissions on all resources. + +## Zabbix Migration Considerations (When Ready) + +| Option | Cost | Complexity | Latency | +|--------|:---:|:---:|:---:| +| Cloud Zabbix + on-prem proxy | +$30–50/mo VPS | Medium — proxy + server setup | Proxy-local (SNMP), API over WAN | +| Full cloud, direct SNMP | +$30–50/mo VPS | High — VPN to each device | WAN-dependent | +| Keep on-prem, API tunnel | $0 incremental | Low — just expose API | Need secure tunnel (Tailscale) | + +**Recommendation:** Cloud Zabbix + on-prem proxy. Keeps SNMP polling local (low latency, no firewall holes), moves the heavy server to managed infra, and gives the portal a stable API endpoint. --- @@ -110,5 +276,7 @@ Signature: strtoupper(hash_hmac('sha256', nonce . key, secret)) - **Splynx v2 docs:** https://api-doc.splynx.com/ - **Splynx v4.2 wiki:** https://wiki.splynx.com/4_2/ -- **Admin panel:** https://portal.forefrontwireless.com/admin/ +- **UISP docs:** https://help.ui.com/hc/en-us/categories/360001181833-UISP +- **MikroTik REST API:** https://help.mikrotik.com/docs/display/ROS/REST+API +- **Zabbix API:** https://www.zabbix.com/documentation/current/en/manual/api - **Git:** https://git.itpropartner.com/ippadmin/forefront-wireless-portal