The silent failure problem: when your NLQ runs but returns the wrong answer
The scariest NLQ bug isn't a crash — it's a query that executes cleanly and returns a confidently wrong number. Here's how to make wrong answers detectable instead of invisible.
A syntax error in a generated query is the good kind of bug: it's loud, and it fails safe. The dangerous bug is the query that runs perfectly and returns the wrong number — a missing filter, a wrong join, the wrong aggregation. It looks like an answer, the user trusts it, and no one knows it's wrong. This silent failure is the hardest problem in NLQ.
Why it happens
- Ambiguous questions — 'top customers' by revenue? by count? this year?
- Wrong join — joining on the wrong key quietly changes the result set.
- A missing WHERE — forgetting a filter returns far too much, but still returns.
- Wrong aggregation — SUM where the user meant COUNT, or the wrong grouping.
- Timezones and units — off-by-a-day and off-by-1000 errors that produce plausible numbers.
Execution-based evals
Don't grade the generated query as a string — two very different-looking queries can be equally correct. Run it against a test database and compare the results to a known-correct answer. Only execution tells you whether the query means what the user asked, which is why execution-based evaluation is the backbone of NLQ testing.
Runtime sanity checks
- Row-count bounds — flag results that are suspiciously empty or improbably large.
- Type and shape checks — the answer to 'how many' should be a number, not a page of rows.
- Tenant-filter assertions — verify the mandatory scoping filter is present (see the query-safety post).
- Suspicious-value flags — round numbers, nulls, or zeros where you'd expect real data.
Make the interpretation visible
The best defence against a wrong interpretation is to surface it. Show the user the query, or a plain-English readback of what it did and the assumptions it made ('revenue, this calendar year, excluding refunds'). A wrong assumption the user can see is a wrong assumption they can correct — instead of a number they blindly trust.
A crash is honest; a wrong number that runs cleanly is not. The job of an NLQ system is to make its mistakes loud.