Eval discipline: proving your non-deterministic AI is actually correct

Traditional software is deterministic, so you test it with assertions: same input, same output. An LLM breaks that — the same input can produce different outputs. Eval discipline is how you regain confidence anyway: not by asserting a single answer, but by proving the system clears a quality bar you define (this is the observability-and-evals discipline, operationalised).

Golden datasets

The foundation is a curated set of inputs with known-good outputs or clear acceptance criteria, one per product surface. Every prompt or model change runs against it before shipping. The golden set is your regression suite for a system that has no single right answer — and it's only as good as how honestly it reflects real, hard inputs.

LLM-as-judge, with eyes open

Scoring open-ended quality by hand doesn't scale, so a strong model grades outputs against a rubric. It's powerful and it's biased — worth using, worth not trusting blindly:

• Position bias — judges favour the first (or last) option; randomise order.
• Verbosity bias — longer answers look better to a judge than they are; control for length.
• Self-preference — a model tends to prefer its own style; spot-check against human ratings.

Shadow mode and regression gates

Before a change reaches users, run it in shadow mode on real traffic and compare against the current version. Then wire evals into CI so a regression blocks the merge. The mindset shift that makes all of this work: a prompt change is a code change, and it deserves the same tests, review, and gates.

You can't prove a probabilistic system is always right. You can prove it clears a bar you defined — and refuse to ship anything that drops below it.