hermes-skills/skills/devops/model-failover-and-credit-tracking/references/ollama-model-ram-sizing.md

Ollama Model RAM Sizing — Core Server

Server Specs

• Model: netcup RS 2000 G12
• RAM: 15 GB total
• vCPUs: 8 (EPYC 9645)
• Swap: /dev/vzswap (compressed swap — not real RAM)

Model Testing Results (Jul 11, 2026)

Model	Params	Quant	Size on Disk	Loads?	RAM Needed	Notes
llama3.2:3b	3B	Q4_K_M	2.0 GB	✅ Yes	~3-4 GB	Default local fallback
llama3.3:70b	70B	Q4_K_M	42 GB	❌ No	~35-40 GB	Error: insufficient memory (attempted to allocate 31320 MB)

Diagnostic Commands

# Show currently LOADED models (models actually in memory)
ollama ps

# Show DOWNLOADED models (models available but may not load)
ollama list

# Check memory usage
free -h

# Check Ollama errors
journalctl -u ollama --no-pager | grep -i "insufficient memory\|failed to load"

Key Rule

ollama list shows what's downloaded. ollama ps shows what's actually loadable. A model can pass ollama list and still be unusable if RAM is insufficient. Always check ollama ps + journalctl -u ollama before declaring a model operational.

For this server (15 GB RAM), the maximum viable model is ~7B params at Q4 quantization (~4-5 GB RAM). 13B models at Q4 may fit (~8 GB) but leave little room for Ollama's overhead + system processes. 70B models are impossible without additional RAM.