Observability beyond LangSmith: Langfuse and structured logging

The LangSmith post covered tracing an LLM app, but it left two gaps a serious client will probe. First: what if they can't send their data to a hosted SaaS? Second: LLM traces show what the model did, but not how an HTTP request became a graph run became a database write. Here are both answers.

Langfuse: the open-source, self-hostable option

Langfuse is an open-source LLM observability platform — tracing, evals, prompt management, and cost/latency dashboards — in the same space as LangSmith, with one decisive difference: you can self-host it. For many enterprises that isn't a preference, it's a requirement: data residency and confidentiality rules mean model inputs and outputs (which may contain their customers' or their own sensitive data) cannot leave their infrastructure. A self-hosted Langfuse keeps the entire trace inside their boundary. When a client asks 'what are the open-source options?', this is the answer to have ready — the ability to run observability on their own servers is often what unblocks the deal.

LLM traces aren't the whole picture

An LLM trace tells you what happened inside the model call. It doesn't, on its own, tell you that request abc123 from tenant 42 hit this endpoint, triggered this graph run, and wrote this row — the end-to-end story you need when something breaks in production. For that you need structured application logs that carry correlation identifiers, alongside the LLM traces.

Structured logging with request and tenant ids

Log as JSON, not free text, so logs are queryable, and stamp every line with a request_id and a tenant_id. The clean way to propagate those through an async app (FastAPI) without threading them through every function is contextvars: set them once per request in middleware, and every log call within that request — however deep in the graph — reads them from context. Now you can filter all logs for one request across HTTP, the graph run, and the DB write, and reconstruct exactly what happened for one tenant's one call.

import contextvars, logging, json

request_id = contextvars.ContextVar('request_id')
tenant_id = contextvars.ContextVar('tenant_id')

# FastAPI middleware sets these once per request; deep code just reads them:
logging.info(json.dumps({
    'event': 'graph_node',
    'node': 'retrieve',
    'request_id': request_id.get(),
    'tenant_id': tenant_id.get(),
}))
# now every log line for one request correlates: HTTP -> graph run -> DB write.

LangSmith isn't the only observability answer — Langfuse is the one you can self-host when data can't leave the building. And traces show the model's story; structured logs with a request_id show the whole system's.