Grounding, citations, and abstention: making an LLM say 'I don't know'

A RAG system that answers every question is a liability. The failure that matters most isn't a wrong answer you can spot — it's a confident, fluent, fabricated one, complete with a citation that looks real and isn't. In high-stakes domains that's not an embarrassment, it's exposure. Grounding, attribution, and abstention are the disciplines that make an LLM's output verifiable instead of just plausible.

Grounding and span-level attribution

Grounding means every factual claim in the answer traces back to a retrieved source, not the model's memory. The strongest form is span-level attribution: each claim is linked to the specific passage that supports it, so a reader — or an automated check — can verify it. This does two things at once: it gives the user a way to check the work, and it makes hallucination detectable, because a claim with no supporting span is a red flag you can catch before it ships (the faithfulness metric from the retrieval-eval post, enforced).

Abstention: the discipline of 'I don't know'

The hardest thing to make a model do is decline. When the retrieved context doesn't support an answer, the right output is an honest abstention — 'I don't have enough to answer that' — not a confident guess. Building this in means instructing the model to answer only from the provided sources, and checking its output for support before returning it: if the claims aren't grounded, you abstain rather than emit them. A system that knows when to stay silent is worth more, in a high-stakes setting, than one that's occasionally brilliant and occasionally invents case law.

Confidence and escalation to a human

Abstention doesn't have to be a dead end — it can be a handoff. Score the confidence of an answer (retrieval quality, degree of grounding, agreement across attempts), and when it's low, escalate to human review instead of guessing or refusing outright. That turns the model into a first-pass drafter that knows its limits and routes the uncertain cases to a person, which is exactly the shape a defensible legal or medical workflow needs.

Why this dominates for high-stakes clients

For an IP, legal, or medical client, a fabricated claim citation isn't a quality issue — it's a legal liability, because someone may act on it. That single fact reorders priorities: grounding and abstention aren't nice-to-haves bolted on after the demo, they're the core of what makes the system usable at all. The model that says 'I'm not sure, here's who to ask' beats the one that fabricates with confidence every single time it matters.

In high-stakes AI, the most valuable thing a model can do isn't answer brilliantly — it's cite honestly, and admit when it can't. A confident fabrication with a fake citation is the one output that can end up in court.