Case study: Itinerary Beautifier — structured extraction under a no-hallucination constraint

Itinerary Beautifier is a system I built for travel agents: they upload a messy PDF or DOCX quote sheet, and it produces a clean, branded, print-ready HTML itinerary they can refine by chat and share with a client. The interesting engineering isn't the pretty output — it's the constraint. This is structured extraction with a no-hallucination rule: read a noisy document, pull out exactly what's there (hotels, days, prices, the customer's own contact details), invent nothing, and render it deterministically. Here's how it's built and, more usefully, why.

The stack is Next.js 16 on Vercel serverless, MongoDB Atlas, and the Anthropic Messages API for extraction and chat edits. Everything below is shaped by two forces: the no-hallucination constraint, and the fact that every function runs serverless.

Two rules that shaped everything

• The file never leaves the browser. Text extraction (pdfjs for PDF, mammoth for DOCX) runs client-side; only the extracted plain text is POSTed. That's a privacy property — uploaded quotes contain client PII — and it also offloads CPU from the serverless functions.
• The model produces data, never markup. The LLM's only job is to emit a strict JSON object. All HTML is rendered by a deterministic, hand-written template with everything escaped. The two rules are independent, and both earn their keep below.

Structured outputs, not prompt-and-parse

Extraction uses the Messages API's structured outputs: the model is constrained to emit JSON matching a schema, so there are no markdown fences, no 'Here is your JSON:' preambles, no prose to strip. But structured outputs guarantee shape, not semantics — so I still Zod-validate every response and re-prompt once on a validation failure. Two layers: the API's braces, and Zod as the belt (this is the structured-outputs discipline in practice).

There are deliberately two representations of the same schema, kept in sync by hand: a Zod schema for runtime validation and inferred types, and a JSON Schema fed to the API to constrain output. Why not generate one from the other? Structured outputs require every object to declare additionalProperties:false and list its required keys, and nullable fields must be typed as ['string','null']. Hand-writing the JSON Schema keeps full control over those constraints; a converter would fight them. The cost is that adding a field means editing both — a documented, accepted trade-off.

When extraction misses something, check the schema before the prompt

The most instructive bug in the project: the customer's phone and email were being dropped. The instinct is to blame the prompt or the model. The real cause was the schema — it simply had nowhere to put them. The model could read the details fine; it had no field to write them into. Adding a traveler_contact object and a free-form trip_details array (for the booking facts every agency lists differently) fixed it outright. Structured outputs are a strong lever on extraction quality precisely because the model can only populate fields that exist. When extraction is missing something, check the schema before you touch the prompt.

First, is it even an itinerary?

Structured outputs have a dark side: the model will happily dress up anything as a trip. Feed the itinerary schema an invoice, a bill, or a CV and it'll invent a plausible-looking itinerary out of nothing. So before spending a structuring pass, a cheap classification gate asks Haiku one question — is this actually a travel itinerary, quote, or package? — and anything that isn't never reaches the extractor. The whole design is about keeping that correctness check cheap and non-blocking:

• Runs on a sample, not the whole document — an invoice never starts reading like an itinerary, so the gate classifies the first ~8k characters rather than re-billing a 200k-character file.
• A wrong upload doesn't burn a daily slot — on rejection the route refunds the quota and returns a 422, and the UI shows a friendly 'that doesn't look like an itinerary' dialog rather than a terse error, leaving the agent on the upload page to try again.
• Fail-open — if the classifier itself errors, the request falls through to structuring (the real work), so a transient hiccup never blocks a genuine itinerary.
• Its tokens are counted too — when an itinerary is built, the classification call is logged alongside it, so the cost readout stays honest.

Like the escalation heuristic below, the verdict is deliberately biased toward yes: a messy WhatsApp trip note should pass, and only documents that plainly aren't itineraries get turned away — because the expensive failure is a rejected customer, not a few paise of Haiku.

Haiku-first, Sonnet-fallback: ~90% cheaper on the common case

This is the single biggest cost lever. Output tokens are priced roughly 5× input, and adaptive thinking spends tokens on the expensive output side — so the extraction is tiered. A cheap pass runs first, and an expensive model is the safety net, not the default.

• Cheap pass — Claude Haiku 4.5, no thinking. It handles straightforward day-by-day extraction well at a fraction of Sonnet's cost.
• Escalate to Claude Sonnet 5 (adaptive thinking, medium effort) only when the cheap pass returns invalid JSON or looks to have missed the customer's booking/contact block.
• The escalation trigger is deliberately biased toward escalating — a needless Sonnet call costs a few paise, but shipping an itinerary missing the customer's details is the failure mode I care about most.

let result = await extractWithHaiku(text);        // cheap first

if (!isValid(result) || likelyMissedContact(text, result)) {
  result = await extractWithSonnet(text, {         // escalate only when needed
    thinking: { type: 'adaptive' },
    effort: 'medium',
  });
}
return result;

Measured: a clean third-party quote runs Haiku-only at ~₹0.76; an ambiguous one that escalates costs ~₹4.77 (Haiku + Sonnet); the old Sonnet-only path was ~₹7. That's roughly 90% cheaper on the common case. The trade-off is honest: a document that always escalates costs slightly more than Sonnet alone, so net savings depend on the escalation rate — which, crucially, the admin dashboard now measures in production rather than leaving it an estimate (see the running-LLMs-in-production post on cost per successful task).

Thinking and effort, tuned per call

• Sonnet fallback — adaptive thinking, medium effort. This was empirically necessary: on a real quote, a thinking-disabled pass silently dropped the customer's phone and email; medium-effort thinking extracted them, because the booking block was buried in marketing copy and FAQs.
• Haiku pass — no thinking. The whole point of the cheap pass is to be cheap; the Sonnet fallback is the net for the cases thinking would help.
• Chat revisions — thinking disabled. Edits ('make the header blue', 'fix Day 3's hotel') are shallow transforms of an already-structured object; disabling thinking keeps them fast and cheap.

One honest caveat: this is not deterministic. Even Sonnet with adaptive thinking is a coin-flip on genuinely ambiguous documents — for instance a self-generated quote where the same contact appears both in a booking field and in a 'Contact us' footer, so the prompt's agency-exclusion rule legitimately applies. That ambiguity is inherent to the task, not a bug in a given model.

No-hallucination prompting

The system prompts are the policy layer, and they're strict about the one thing that matters:

• Extract only what's present — use null or omit rather than invent a hotel, price, or date.
• Disambiguate the hard cases precisely — a contact among the booking fields is the customer's, even if its domain happens to match the agency's, while a support line in a header or footer is the agency's and is dropped.
• The only original prose the model may write is a 2–3 sentence summary — and even that may not state a fact (a hotel, a price) that isn't in the source.
• Normalise country to a full English name, because it's used to disambiguate place names for image lookup, so consistency matters downstream.

Deterministic rendering: the model never writes HTML

The model's job ends at data. A deterministic, hand-written template turns the validated JSON into HTML — the model never produces markup. That's an AI-safety decision as much as a rendering one: every value that comes from the model is escaped before it's interpolated, and URLs are restricted to safe schemes, so a prompt-injected document can't smuggle a script tag into the output — the 'retrieved content is data, not instructions' principle from the prompt-injection post, enforced structurally. Keeping generation constrained to data rather than freeform HTML is what makes the no-hallucination guarantee hold at the output boundary too.

Cost tracking that closes the loop

Because there are now two models at different prices, cost tracking is per-model. A small pricing table computes USD cost from token usage at the model's rate in effect when it was captured, and each stored record freezes the price it was actually billed at. Extraction returns every model call it made — so on an escalation, both the Haiku and Sonnet spend are logged, not just the winning call. The admin dashboard aggregates all of it (tokens and cost, USD and INR, per agent and per itinerary). That's what turns the escalation-rate assumption behind the whole Haiku-first bet from a guess into a measured number.

Trade-offs I'd revisit

• Haiku-first is a bet on the escalation rate — a clear win when Haiku succeeds, a small loss when a document always escalates. Now that the rate is measured, the next move (if self-generated, contact-in-footer quotes dominate) is to tune the escalation heuristic or try a small Haiku thinking budget before falling back.
• The itinerary gate adds a Haiku call to every generation — cheap, but spent even on the common case where the document really is an itinerary. Worth it to stop dressing up invoices as trips; if the classifier ever proves reliable enough, it could fold into the first structuring pass to reclaim the call.
• Two schemas by hand — accepted for control over the structured-output constraints; a codegen step is the obvious cleanup if the schema starts to churn.
• Non-determinism on ambiguous inputs — no prompt tuning removes it; the honest fixes are better source signals and the completeness heuristic, not a bigger model.

The hard part of this project was never making it look good — it was making a probabilistic model behave: extract only what's there, escalate only when it must, and render nothing it didn't earn.