Files
hermes-recovery/references/server-provisioning-standard-v1.md

138 lines
3.8 KiB
Markdown

# IT Pro Partner — Server Provisioning Standard v1.0
**Author:** Sho'Nuff / Network Services Team
**Date:** July 9, 2026
**Status:** DRAFT — pending review
---
## 1. Server Tier Definitions
| Tier | Spec | OS | Use Case | Monthly Cost |
|------|------|----|----------|-------------|
| **Standard** | RS 4000 G12 (12C EPYC, 32GB RAM, 1TB NVMe) | Debian 13 | app1, app2, app3 | ~$44/mo |
| **Light** | RS 2000 G12 (8C EPYC, 16GB RAM, 512GB NVMe) | Debian 13 | Core (existing) | ~$24/mo |
| **Standby** | CPX11 (2C/2G/40GB) | Debian 12 | app1-bu (warm standby) | ~$8/mo Hetzner |
| **Legacy** | CPX21 (3C/4G/80GB) or CPX41 | Debian 12 | Existing Hetzner boxes | ~$14-44/mo |
---
## 2. OS Provisioning
### OS Selection
- **netcup servers:** Debian 13 (bookworm) — latest stable
- **Hetzner servers:** Debian 12 (existing, migrate to 13 on rebuild)
- **Reasoning:** Debian over Ubuntu for stability, no forced upgrades, smaller footprint
### Base Install
```bash
# Set hostname
hostnamectl set-hostname <server-name>
echo "<server-name>" > /etc/hostname
# Update everything
apt update && apt upgrade -y
# Timezone
timedatectl set-timezone America/New_York
# Fail2ban
apt install -y fail2ban
# Monitoring agent
apt install -y prometheus-node-exporter
# Firewall
apt install -y ufw
ufw default deny incoming
ufw default allow outgoing
ufw allow ssh
ufw --force enable
```
### Standard Users
```bash
# ippadmin — standard admin user for all servers
adduser ippadmin
usermod -aG sudo ippadmin
mkdir -p /home/ippadmin/.ssh
# Deploy itpp-infra SSH key (Hetzner project-wide key)
echo "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAII4dxTH11aJkBqCY8lXl1kTfZ8yXWhTcthHnt1MtAuIE itpp-infra" \
>> /home/ippadmin/.ssh/authorized_keys
# Disable password auth for SSH
sed -i 's/^#PasswordAuthentication yes/PasswordAuthentication no/' /etc/ssh/sshd_config
systemctl restart sshd
```
### Key Deployment
- **itpp-infra** — standard admin key, deployed to all servers (Hetzner project-wide via SSH vault)
- **Root SSH disabled** after initial setup — use `ippadmin` with `sudo`
---
## 3. Standard Services
Every server gets:
| Service | Purpose | Installed By |
|---------|---------|-------------|
| fail2ban | Brute force protection | Base install |
| prometheus-node-exporter | Metrics for monitoring | Base install |
| ufw | Host firewall | Base install |
| Docker (if needed) | Container runtime | Per-server |
| Caddy (if serving web) | Reverse proxy + TLS | Per-server |
---
## 4. Docker Standard
### Installation
```bash
curl -fsSL https://get.docker.com | bash
apt install -y docker-compose-plugin
```
### Docker Compose
- All stacks at `/opt/<service>/docker-compose.yml`
- Standard network: bridge (default)
- Volume mounts under `/opt/<service>/data/`
- .env files at `/opt/<service>/.env` (chmod 600)
---
## 5. Backup Standard
| Data | Frequency | Location | Retention |
|------|-----------|----------|-----------|
| System config | Daily | s3://hermes-vps-backups/<server>/ | 90 days |
| Docker volumes | Daily | s3://hermes-vps-backups/<server>/ | 90 days |
| Database dumps | Daily | s3://hermes-vps-backups/<server>/ | 90 days |
| Application data | Per-application | s3://hermes-vps-backups/<server>/ | 90 days |
| Hermes state | Every 15 min | s3://hermes-vps-backups/live/ | 48h (snapshots) |
---
## 6. DR Restore Procedure (Standard)
### Standard Restore
```bash
# 1. Provision new server (same tier or better)
# 2. Run base install steps (Section 2)
# 3. Deploy SSH key
# 4. Restore from S3:
aws s3 cp s3://hermes-vps-backups/<server>/latest/ /opt/restore/ \
--endpoint-url https://s3.us-east-1.wasabisys.com --recursive
# 5. Restore Docker stacks
cd /opt/<service> && docker compose up -d
# 6. Verify health endpoints
```
### Emergency Restore (No S3 Access)
```bash
# Use Hetzner rescue mode / netcup recovery
# Reinstall OS, deploy SSH keys
# Rebuild from documented configs in Hudu
```