Every “which model should we run?” debate ends the same way — someone says “it depends” and the meeting moves on. It depends on volume, latency tolerance, and who bears the cost. Those are three numbers, not a vibe. This page is the arithmetic.
The two roads
Hosted APIs put the hardware in someone else’s building: zero up-front cost, per-token billing, instant access to frontier models, and a dependency — pricing, rate limits, and retention policy live on the vendor’s side. Self-hosting puts the model on your machine: hardware you buy once, no per-token bill, full control of data and versions, and a ceiling — quality is bounded by what your hardware can hold, and someone owns the ops. The control spectrum from the Sovereign AI track is the policy view of this same trade.
1. Count what an API actually costs
API pricing is per token — roughly ¾ of a word. The mental math that matters:
tokens/day ≈ words × 1.33 (prompt + context + output) monthly cost ≈ tokens/day × 30 ÷ 1,000,000 × $ per 1M tokens worked example: a 2,000-word task, done 10×/day ≈ 26,000 tokens/run × 10 = 260K tokens/day at rough frontier pricing ($3 input / $15 output per 1M) → on the order of $20–60/month for that one task
Two multipliers people forget: context re-sends (long chat histories are billed again on every turn, so a long conversation can bill many times its own length) and agent amplification (an agent that thinks in steps burns 3–10× a single chat answer). Per-seat and per-request plans change the math; always price your real usage shape, not the headline.
2. Count what self-hosting actually costs
The bill is one-time hardware plus quiet ongoing costs:
up-front GPU/box (quality lives here — see the Choosing page)
ongoing power (a ~300W box running 24/7 ≈ 216 kWh/mo ≈ $30–50/mo)
your time — updates, troubleshooting, model swaps
hidden disk for model weights (full-precision 70B ≈ 140 GB)Then the real question: can this box serve your concurrency? One patient user needs far less than six users expecting chat-speed answers. A box that serves one person beautifully and a team poorly is the classic self-hosting surprise.
3. The crossover point — and the hybrid answer
Set the two cost curves side by side and there is a crossover: below some usage volume, hosted wins (no hardware idling); above it, owning flattens out and wins. For most professional use the honest answer is neither/or — it’s both, split by job:
Sensitive data (client files, personal info) → self-host, any volume High-volume bulk work (summaries, tagging) → cheapest model that passes QA Frontier-reasoning moments → hosted, pay per use Everything else → whichever is idle
Rule of thumb: spend rented tokens on problems that are big; spend owned hardware on data that is touchy. Mixing them in the wrong direction is the most expensive mistake on this page.
4. Hidden costs on both sides
Hosted: rate limits at the worst moment, price changes announced after the fact, egress costs when you migrate away, integration glue. Self-hosted: the ops learning curve, model churn (today’s best local model is not a fixed target), and the day you discover your workload needed VRAM you don’t have. Price honesty means including your own hours — at professional rates they can dominate the self-hosted column.
Worked decision, start to finish
A three-person team drafts 150 client-facing summaries a month, ~2,500 words each, with about 8K tokens of context per session:
hosted: ~150 × 14K tokens ≈ 2.1M tokens/month
at a blended ~$3.5 per 1M ≈ $8/month → API wins on cost
retention question: client files → see Sovereign AI
self-host: capable box ≈ $2–4K once + ~$40/mo power
viable IF one box serves 3 users at chat speed
wins on privacy & fixed cost; loses on peak qualityThe specific numbers are placeholders — the shape is the lesson. Run your own volumes once and the fog lifts; the “it depends” meeting becomes a five-minute calculation. When the answer touches client data, finish the decision with the sovereignty checklist. Pair this page with Choosing to translate budget into a specific model.
Cost is only half the decision. The Sovereign AI track covers the other half: who keeps your data, and what you'd have to move if a provider changed its terms.