The reasoning-effort knob: paying for thinking only when it's worth it
Reasoning models let you dial how hard they think — low, medium, or high. Used well it's a precise lever over the quality-versus-cost trade. Used carelessly it's a silent bill.
Reasoning models introduced a control the older ones didn't have: how much to think before answering. Exposed as a reasoning-effort setting (or a thinking-token budget), it lets you decide, per request, how much inference-time compute to spend. It's one of the more useful knobs in modern LLM APIs — and one of the easiest to mis-set.
What it actually controls
Higher effort lets the model generate a longer internal chain of reasoning before its final answer — exploring more, checking itself more — which lifts accuracy on genuinely hard problems. Lower effort answers faster and cheaper. It's the inference-time scaling curve, exposed as a dial you turn.
- Low — fast, cheap, and plenty for straightforward tasks; don't pay for reasoning a simple prompt doesn't need.
- Medium — a sensible default for moderately complex work.
- High — for the hard cases: intricate maths, multi-step logic, tricky code, careful planning — where the extra tokens earn their keep.
const res = await client.chat.completions.create({
model: 'o4-mini',
reasoning_effort: 'high', // 'low' | 'medium' | 'high'
messages,
});
// reserve 'high' for the requests that actually need itTreat it like a budget
Every step of extra effort costs latency and tokens — the thinking tokens are billed even though the user never sees them. The winning pattern is to route: send easy traffic at low effort and reserve high for the small slice of requests that need it, rather than turning the dial up globally and paying for reasoning on 'what's 2+2'.
The reasoning knob isn't 'make it smarter' — it's 'spend more here'. Turn it up where the problem is hard, and down everywhere else.