Tokens & the Context Window
Models read text in tokens — chunks of roughly ¾ of a word (in English, ~1.3 tokens per word). "Tokenization" is why models sometimes miscount letters: they see tokens, not letters.
The context window is how many tokens the model can look at at once — your instructions, the conversation so far, and any documents, all combined. Think of it as the model's desk size:
A few pages. Older/small models; fine for short chats, cramped for document work.
Reports to small books. The modern standard — most work fits comfortably here.
Whole books or big document sets. Watch for "lost in the middle": models attend best at the start and end of huge contexts (see the What If? guide, scenario 1).
Two gotchas: bigger windows also need more memory at run time, and a big window is not the same as good attention. Feeding 200 pages to find one number is the "Noise" mistake from the Context Master page — retrieve first, then reason.
Reading a Model Card
Every published model ships with a model card (on Hugging Face, or the publisher's page). Five lines matter for a business decision:
Parameter count — how big; maps to the memory table on the Parameters page.
Context length — how much it can read at once.
Type — base / instruct / reasoning / vision / coder (see Model Types).
License — permissive, community, or research-only (see Open Weights).
Knowledge cutoff — when its training data ends. Anything newer must come from your documents (RAG), not the model's memory.
Why Benchmarks Lie a Little
Benchmarks are standardized tests (MMLU for knowledge, HumanEval for code, GSM8K for math). They're useful for coarse comparisons — and misleading if you take them at face value:
Contamination: famous test questions circulate in training data. A model can "ace" a test it has effectively memorized.
Wrong target: a 92% MMLU says little about your actual task — summarizing claims files or writing n8n code. Benchmarks measure the test, not your workload.
Leaderboard tuning: some models are polished for exactly the popular benchmarks. Great scores, ordinary real-world feel.
The only benchmark that matters: your ten real tasks, run on your real documents, scored by your own judgment.
The Decision Table
Match the task to a size class and a type — then verify with your own ten tasks:
| Task | Reach for | Type |
|---|---|---|
| Everyday chat, email drafting, summaries | 7–14B | Chat |
| Document Q&A over your own files (RAG) | 7–14B + embedding model | Chat |
| Complex analysis, planning, hard debugging | 30B+ (or MoE) | Reasoning |
| Writing / reviewing code, SQL, scripts | 14–70B | Coder |
| Scanned docs, screenshots, image data | 7–70B | Vision |
| Bulk simple classification / tagging | 1–3B | Chat / instruct |
Sensitivity test: does the work touch client identifiers, legal, medical, or financial material? Then it stays on sovereign infrastructure — no matter how good the big hosted model is.
Revisit quarterly: models improve fast. The right pick today won't be the right pick in six months — and with open weights, switching is a download, not a negotiation.