Inference-time scaling: buying better answers with more thinking
The newest way to get a smarter answer isn't a bigger model — it's letting the model think longer at inference. Reasoning models made test-time compute a scaling axis of its own.
For years, a better answer meant a bigger model trained at greater cost. Reasoning models opened a second axis: keep the model the same, but let it spend more compute at the moment you ask. Think longer, sample more, check the work — and accuracy climbs. This is inference-time (or test-time) scaling, and it's reshaping how the best systems are built.
Spend tokens to buy accuracy
Instead of answering immediately, the model generates a chain of reasoning — exploring, self-correcting, sometimes trying several approaches — before committing. More thinking tokens buys more accuracy on hard problems, along a curve much like training scaling, except you pay per request instead of once up front.
The common techniques
- Chain-of-thought — the model reasons step by step before answering, which alone lifts performance on multi-step problems.
- Best-of-N / self-consistency — generate several answers and take the most consistent one, trading tokens for reliability.
- Reasoning models — trained to do this natively, with a tunable amount of 'thinking' you can dial per request.
The trade you're making
Inference-time scaling is pay-as-you-go intelligence. That's powerful — you can reserve deep thinking for the hard 5% of requests and answer the rest cheaply — but it's also a latency and cost knob that's easy to leave turned up. The discipline is the same as always: spend the extra compute only where the problem is worth it.
Training scaling makes the model smarter once, for everyone. Inference scaling makes this answer smarter, for a price — choose when it's worth paying.