162 lines
12 KiB
Markdown
162 lines
12 KiB
Markdown
# US Provider Evaluation for the DR Standby Replacement
|
|
|
|
**Date:** 2026-09-15
|
|
**Status:** OPEN, decision required from Germaine
|
|
**Owner:** Sho'Nuff
|
|
**Trigger:** app1-bu (Hetzner CPX21, Ashburn) retires Saturday 2026-09-19
|
|
|
|
---
|
|
|
|
## 1. Why this decision exists
|
|
|
|
`app1-bu` is currently the **only non-netcup box in the fleet**. It is the warm standby for Core.
|
|
|
|
Two separate failure modes are in play, and they need two different kinds of separation:
|
|
|
|
| Failure mode | What protects against it | Current state |
|
|
|---|---|---|
|
|
| netcup provider outage takes down Core and its standby together | **Provider diversity** | **UNMET.** core-bu, app4, and anita-mnz are all netcup. app1-bu (Hetzner) is the only non-netcup box, and it is being retired. |
|
|
| Regional outage (power, network, DC-level) takes down Core and its standby together | **Geographic separation** | **UNMET.** Core sits in netcup's Manassas VA location. app1-bu is in Hetzner Ashburn, roughly 30 miles away in the same corridor and power region. |
|
|
|
|
So the replacement box has to satisfy **both**: not netcup, not Hetzner, and not in the Mid-Atlantic.
|
|
|
|
---
|
|
|
|
## 2. Hard requirements (the gate)
|
|
|
|
These are non-negotiable and were enforced against every candidate:
|
|
|
|
1. **KVM virtualization.** Not OpenVZ, not LXC. A container-backed VPS cannot run the standby's kernel-level needs and is not a real VM.
|
|
2. **Debian 13 (Trixie) available at order time**, or a defensible path to boot it.
|
|
3. **IPv4 included**, not a surprise add-on.
|
|
4. **Monthly billing.** No annual-only pricing dressed up as a monthly rate.
|
|
5. **Geographically far from NOVA** (Manassas VA), to fix the regional-separation gap above.
|
|
6. **DPA available** (data processing agreement). The standby holds client data at rest, so this matters.
|
|
7. **Judged on cart price, not marketing price.** Every figure below was pulled from an order form or a first-party API.
|
|
8. **Benchmark before trust.** Whatever gets bought gets a burn-in before it takes standby duty.
|
|
|
|
---
|
|
|
|
## 3. What the box actually has to do
|
|
|
|
The standby is not a second Core. Its job is narrow:
|
|
|
|
- Run the Hermes gateway and its cron jobs if Core goes down.
|
|
- Pull the checkpointed `state.db` from S3 every 10 minutes.
|
|
- Run the failover watchdog every 5 minutes.
|
|
- Take over messaging when Core is fenced.
|
|
|
|
`app1-bu` does all of this today on **3 vCPU / 4 GB RAM / 80 GB disk**. That is the **proven floor**, not a guess.
|
|
|
|
**8 GB is headroom, not a requirement.** The only moment the box needs more is during a failover, when it is restoring state and running syncs concurrently. 4 GB has handled that role since August. The 8 GB option exists if you want margin and are willing to pay roughly double.
|
|
|
|
---
|
|
|
|
## 4. What was evaluated
|
|
|
|
Nine providers, three tiers, three parallel teams. All figures checked **2026-09-15**.
|
|
|
|
### Tier 1: mainstream US cloud
|
|
|
|
| Provider | 4 GB | 8 GB | Debian 13 | IPv4 | SFO/Seattle/LA region | Reputation | DPA |
|
|
|---|---|---|---|---|---|---|---|
|
|
| **DigitalOcean** | **$24** | **$48** | Yes (`debian-13-x64`) | Included | SFO3 full availability | **4.6/5** (2,555 reviews) | **Published** |
|
|
| Linode / Akamai | $24 | $48 | Yes (`linode/debian13`) | Included | us-west, us-sea, us-lax | 2.5/5 TP, 8.3/10 TrustRadius | Available |
|
|
| Vultr | $20 | $40 | Yes | Included | sea, lax, sjc | **1.6/5** (580 reviews) | Available |
|
|
|
|
### Tier 2: budget and mid-tier
|
|
|
|
| Provider | Best-fit plan | Monthly (cart) | Debian 13 | US locations | Verdict |
|
|
|---|---|---|---|---|---|
|
|
| **RackNerd** | KVM-4GB, 4 vCPU / 4 GB / **130 GB** | **$24.59** | Yes | Dallas, San Jose, LA, Chicago, Atlanta, NY | Solid, closest runner-up |
|
|
| **ServerCheap** | NVMe-KVM-10GB, 4 vCPU / 10 GB / 80 GB | **$6.80** | Yes | **Dallas only** | Cheapest spec match, thin track record |
|
|
| BuyVM / Frantech | Ryzen KVM 4GB / 80 GB NVMe | $20.00 | **No** (max Debian 11) | Las Vegas | Best trust, fails the OS gate |
|
|
| SSD Nodes | Standard 8GB / 160 GB | $19 ($15 + $2 IPv4 + $2 snapshots) | Yes | Dallas, ATL, LA, Chicago, Seattle, NY | Oversell pattern, default-on add-ons |
|
|
| VPSDime | Premium8GB / 120 GB | $40.00 | Yes | LA, Dallas | Cheap tier is **LXC, disqualified** |
|
|
| HostHatch | NVMe 8GB / **35 GB** | $9.00 | **Unconfirmed** | LA, NY, Chicago | **Eliminated** |
|
|
|
|
### Distance from Manassas VA (straight-line estimates)
|
|
|
|
- New York / Piscataway NJ, ~200 mi. **Too close, do not use.**
|
|
- Atlanta ~550 mi, Chicago ~600 mi. Acceptable.
|
|
- Miami ~900 mi, Dallas ~1,200 mi. Good.
|
|
- Las Vegas, Los Angeles, Seattle ~2,100 to 2,300 mi, San Francisco ~2,400 mi. Best.
|
|
|
|
---
|
|
|
|
## 5. Recommendation
|
|
|
|
### Primary: DigitalOcean, SFO3 (San Francisco), 4 GB Basic Droplet, $24/month
|
|
|
|
**Drop-in spec match against app1-bu:** 2 vCPU / 4 GB / 80 GB SSD / 4 TB transfer. Same RAM, same disk, one fewer core than the box it replaces, and it costs **less than the Hetzner CPX21 it is replacing** (EUR 31.99/mo, roughly $35).
|
|
|
|
Every gate item is satisfied and independently verified:
|
|
|
|
- KVM. DigitalOcean Droplets are KVM-based.
|
|
- Debian 13 confirmed as first-party image slug `debian-13-x64`, image ID 234194767.
|
|
- IPv4 included at no extra cost on bundled plans.
|
|
- Monthly billing, moving to per-second billing January 2026.
|
|
- **SFO3 has full availability for Basic Droplets.** This was checked, because SFO2 is capacity-limited and Atlanta, Richmond, and Kansas City only offer limited availability. SFO3 is the correct target.
|
|
- ~2,400 mi from Manassas. Fixes the regional-separation gap that app1-bu never fixed.
|
|
- Not netcup, not Hetzner. Fixes the provider-diversity gap.
|
|
- DPA published, last updated 2026-02-06, explicitly covering GDPR, UK, Swiss, and CCPA data.
|
|
|
|
**Why DO over the two providers at the same price:** RackNerd is $0.59 more per month for double the disk, so price is not the differentiator. At effectively identical cost, DigitalOcean wins on the two things that matter for a box you depend on in a disaster: **reputation** (4.6/5 against RackNerd's 4.1/5, and RackNerd's negative chatter clusters on oversubscription) and **DPA** (published and current, against nothing found for RackNerd).
|
|
|
|
Add **weekly backups at +20%, $4.80/mo**, bringing it to **$28.80/mo**. Note that this is provider-level image backup, not a substitute for the existing S3 pipeline. It is cheap insurance, not the primary backup.
|
|
|
|
### Alternate at the same price: Linode / Akamai, 4 GB, $24/month
|
|
|
|
Identical price and specs, confirmed from Linode's own public API. Regions `us-west` (Fremont), `us-sea`, or `us-lax` all satisfy distance. Backup add-on is $5/mo. Pick this instead of DO only if you prefer Akamai's network or want more US region choices. Its knock is a softer reputation signal since the Akamai acquisition.
|
|
|
|
### If cost is the binding constraint: two options, stated honestly
|
|
|
|
- **ServerCheap, $6.80/mo** for 4 vCPU / 10 GB / 80 GB in Dallas. Debian 13 confirmed, IPv4 and one snapshot included, no default-on paid add-ons, clean $0.00 setup fee, operates its own AS62633. **The risk is real:** single datacenter, only 31 Trustpilot reviews, and no independent forum scrutiny (LowEndTalk, r/VPS) specific to this brand could be found. A positive review I did find turned out to reference `servercheap.net`, a **different, largely inactive legacy brand**. At 3.6x cheaper than DO it is genuinely tempting for an idle box, but it is the option most likely to be the problem when you actually need it.
|
|
- **RackNerd, $24.59/mo** for 4 vCPU / 4 GB / 130 GB in Dallas or San Jose. Well-documented business (Inc. 5000 for multiple years, Trustpilot 4.1/5), Debian 13 confirmed in the order form, IPv4 shown as a $0.00 line item, no forced add-ons. The caution is LowEndTalk chatter about oversubscription, which appears to concentrate on their loss-leader annual specials rather than the standard line.
|
|
|
|
**Vultr is the cheapest of the mainstream three at $20, and is technically fully compliant. I recommend against it anyway.** A 1.6/5 Trustpilot score across 580 reviews, dominated by support and abuse-handling complaints, plus a 2024 ToS controversy over content licensing rights. For a box whose entire purpose is to be there when something has gone wrong, saving $4/month against DigitalOcean is the wrong trade.
|
|
|
|
---
|
|
|
|
## 6. Eliminated, and why
|
|
|
|
- **HostHatch.** Worst reputation of the nine (Trustpilot 2.7/5 across 91 reviews, with recent public complaints of unresponsive support and abrupt suspensions, which is precisely the failure mode a DR standby cannot have). Discounting that: their tiers never cleanly match 4 to 8 GB with 80 GB disk. The 8 GB tier ships 35 GB of disk, and you have to go to the 24 GB tier to clear 80 GB. Their order form also sits behind account login, so neither Debian 13 availability nor IPv4 inclusion could be confirmed from the cart.
|
|
- **SSD Nodes.** Meets the checkboxes on paper, but there is a consistent multi-year pattern of oversubscription complaints (Trustpilot 3.0/5), and the cart defaults two paid add-ons on with the IPv4 and snapshot un-check not sticking during testing. That is the wrong signal for a box that needs honest resource allocation. Deprioritized, not banned.
|
|
- **VPSDime.** Their headline cheap tier is **LXC, which fails the KVM gate outright**. Their KVM Premium tier is compliant but costs $40/mo for 8 GB, double the equivalent elsewhere, and only sells in LA and Dallas. Not worth it against DO at $24.
|
|
- **BuyVM / Frantech.** The best raw trust signal of anything evaluated: founded 2007, Trustpilot 3.8/5, consistently positive long-term community sentiment, and true flat monthly billing with no annual trap. It fails only on Debian 13, which is absent from the order form (the list stops at Debian 11 and Ubuntu 20.04). Workable via custom ISO upload, but that is permanent operational overhead against a one-click template. **Rejected on the gate, not on quality.**
|
|
|
|
---
|
|
|
|
## 7. The DPA gap, stated plainly
|
|
|
|
| Provider | DPA status |
|
|
|---|---|
|
|
| DigitalOcean | **Verified.** Published, updated 2026-02-06, covers GDPR, UK, Swiss, CCPA |
|
|
| Linode, Vultr | Available |
|
|
| RackNerd | **Nothing found.** No published DPA located |
|
|
| ServerCheap, BuyVM | **Nothing found** |
|
|
|
|
Absence of a published DPA is not proof they will refuse one, but budget hosts frequently will not sign one. If the DPA requirement is enforced literally, **this alone eliminates RackNerd, ServerCheap, and BuyVM**, and the decision collapses to DO or Linode.
|
|
|
|
---
|
|
|
|
## 8. Next steps once you pick
|
|
|
|
1. Open the account and order from the live cart. Confirm the price on the final invoice screen before paying.
|
|
2. Burn-in benchmark before it is trusted with anything. This is the standing rule and it applies to a mainstream provider too.
|
|
3. Rebuild the standby on the new box: fleet sshd convention, `itpp-infra` key, ufw, fail2ban, unattended-upgrades, node_exporter, S3 credentials.
|
|
4. Deploy the watchdog and the sync job, then **prove a full failover and a failback**, not just that the scripts run.
|
|
5. Only then retire app1-bu, and disarm it first so two standbys are never armed at once.
|
|
6. Register the new host in monitoring and add the Vaultwarden credential entry.
|
|
|
|
---
|
|
|
|
## 9. Method and limitations
|
|
|
|
- Every price is a cart or first-party API figure, not a marketing headline. Where a headline price was annual-only, that is flagged in the table.
|
|
- DigitalOcean, Vultr, and Linode figures were **independently re-verified by the parent agent** against `api.linode.com/v4/linode/types`, `api.vultr.com/v2/plans`, and DigitalOcean's pricing and image documentation. The Linode, Vultr, and DO numbers in section 4 all match.
|
|
- DigitalOcean SFO3 Basic Droplet availability was verified against DO's regional availability documentation, last updated 2026-08-10.
|
|
- RackNerd, BuyVM, SSD Nodes, VPSDime, and ServerCheap figures come from live WHMCS cart pages.
|
|
- **Unverified gaps:** HostHatch's OS list and IPv4 policy (login-gated cart), and DPA availability for RackNerd, ServerCheap, and BuyVM. These are flagged rather than assumed.
|
|
- Trustpilot and forum reputation figures are as published on 2026-09-15 and are directional, not precise measurements.
|