Model parameters: what they are, and why the count drives your bill

Every model is quoted by its parameter count — 7B, 70B, 400B — as if it were engine size. And in a real sense it is: parameters decide how much the model can represent, how much memory it needs, and how much each token costs you. If you deploy models, it pays to know what that number actually measures.

A parameter is a learned number

Under the hood a model is a large neural network. Each connection between neurons has a weight, and each neuron has a bias. Together, weights and biases are the parameters — the numbers the model adjusted during training so that useful text comes out. They aren't rules someone wrote; they're the distilled residue of everything the model saw.

Why the count gets quoted like horsepower

• Capacity — more parameters means more room to store patterns, so bigger models tend to be more capable (up to a point, with sharp diminishing returns).
• Memory — every parameter has to sit in memory to run. Roughly, a model needs (parameters × bytes-per-parameter) of RAM or VRAM just to load.
• Cost and latency — more parameters means more maths per token, which is why larger models are slower and pricier per call.

Precision and quantization: the same model, smaller

Parameters are stored at some numeric precision — 16-bit by default. Quantization shrinks them to 8-bit or 4-bit, roughly halving or quartering the memory and speeding inference, at a small and usually acceptable quality cost. It's how a 70B model that wants ~140 GB at full precision squeezes onto a single high-end GPU.

// bytes per parameter: fp16 = 2, int8 = 1, int4 = 0.5
const gb = (params, bytesPerParam) =>
  (params * bytesPerParam) / 1e9;

gb(7e9, 2);    // ~14 GB  — 7B at fp16
gb(70e9, 2);   // ~140 GB — 70B at fp16
gb(70e9, 0.5); // ~35 GB  — 70B quantized to 4-bit

This is why "which model fits on my hardware?" is really a question about parameters times precision — and why quantization is often the difference between a model you can self-host and one you can't.

Parameter count isn't a quality score. It's a capacity, a memory bill, and a latency budget — read it as all three.