Langfuse, LangSmith and Logfire

LLM observability covers what to capture. This is where it goes — the three platforms named most often in ads, what each is actually optimised for, and the one question that decides between them.

Verified 2026-08

Verified 2026-08-18. This category consolidated hard in 2026; re-check before quoting ownership or pricing.

The one question: who owns the instrumentation

All three now ingest OpenTelemetry. That changes the decision, because the expensive part of observability is instrumenting the code, and OTel makes that part portable:

# Instrument once, against the standard.
from opentelemetry import trace
tracer = trace.get_tracer(__name__)

with tracer.start_as_current_span("rag.answer") as span:
    span.set_attribute("gen_ai.request.model", model)
    span.set_attribute("gen_ai.usage.input_tokens", n_in)

Point the exporter at whichever backend you are on. A team that instruments with a vendor’s proprietary SDK has bought a migration; a team that instruments with OTel has bought a config change. Say that first — it reframes the question from “which tool” to “which dependency”.

Langfuse

Open source (MIT), self-hostable, and acquired by ClickHouse in January 2026 — with the MIT licence, self-hosting and the roadmap publicly committed to. Naming the acquisition is a cheap currency signal; assuming it made Langfuse proprietary is wrong.

from langfuse import observe, get_client

@observe()
def answer(question: str) -> str:
    # nested span, automatically
    docs = retrieve(question)
    out = llm(prompt(question, docs))
    get_client().update_current_trace(
        input=question, output=out,
        metadata={"chunks": len(docs)},
    )
    return out

The @observe decorator builds the trace tree from the call stack, so nesting is free. Its SDK is built on OTel, which is why it doubles as a generic backend.

Pick it when the data must stay inside your boundary, or when volume makes per-trace pricing painful — self-hosting is a Compose file with ClickHouse and Postgres behind it, and the licence cost is zero. Prompts, completions and evaluation datasets are exactly the PII-bearing material that makes PII, privacy and the EU AI Act relevant.

LangSmith

LangChain’s first-party platform, and the trade is explicit: zero setup if you are on LangChain or LangGraph, a paid dependency if you are not.

# Nothing to import. The framework already emits.
import os
os.environ["LANGSMITH_TRACING"] = "true"
os.environ["LANGSMITH_API_KEY"] = key

# full trace appears
graph.invoke({"messages": [...]})

Every node, tool call, retry and token count shows up without a line of instrumentation, which is hard to beat while you are debugging a graph. It also ships the Prompt Hub, annotation queues and an evaluation runner.

The constraints worth stating: self-hosting is Enterprise-only, and pricing is per trace, so cost scales with traffic rather than with team size. At high volume the gap against self-hosted Langfuse decides the question on its own.

Logfire

Pydantic’s platform, and the one that fits the stack in this repo. It is an OTel platform first and an LLM tool second — which is the point:

import logfire

logfire.configure()
logfire.instrument_fastapi(app)
logfire.instrument_sqlalchemy()
logfire.instrument_httpx()
logfire.instrument_pydantic_ai()

with logfire.span("checkout {order_id}", order_id=order.id):
    ...

Four lines and the HTTP request, the SQL it ran, the outbound calls and the LLM spans are one trace. That is the argument: an LLM call is one span in a request that also hit Postgres and two services, and correlating those across two tools by timestamp is the thing everyone does and nobody enjoys.

Two more things it does that the others do not. Spans are queryable in SQL, not a bespoke filter language:

SELECT span_name, avg(duration)
FROM records
WHERE attributes ->> 'gen_ai.request.model' = 'claude-opus-5'
GROUP BY 1
ORDER BY 2 DESC;

And it understands Pydantic validation, so a failure shows the model, the field and the input that broke it — the boundary failure from Pydantic in practice becomes a span rather than a log line.

Gotcha: Logfire is a general APM that handles LLM spans, not an evaluation platform. No dataset runner, no annotation queue, no prompt registry. If the requirement is “score 200 cases on every PR”, that is The eval and observability tooling, not this.

Choosing

Running two is normal and not a failure: an APM for the request and a prompt-aware tool for the LLM half. What is a failure is instrumenting twice.

The position to hold: the platform is the least portable-relevant decision, and the evaluation suite is the asset. Traces are how you debug one request; the eval set is how you know a change was an improvement, and it outlives whichever backend you picked.

Related

• LLM observability — what to capture, and the OTel GenAI conventions
• Cost attribution — the token spend these traces feed
• The eval and observability tooling — the scoring half
• OpenTelemetry — OTel outside the LLM context
• Pydantic AI — what Logfire instruments natively