Your agent worked fine in the demo. Then it shipped, a user asked something slightly weird, and it called the wrong tool three times before hallucinating a refund policy that doesn’t exist. Now you’re staring at production logs that show you the final answer and nothing about how it got there.
That’s the moment people go shopping for an LLM observability tool. The problem is that the category has quietly split into three different jobs, and most comparison posts — usually written by one of the vendors — pretend their tool does all three equally well. It doesn’t. Pick wrong and you’ll either overpay for evals you never run or bolt on a second tool six weeks later.
Here’s how I’d actually think about Langfuse, LangSmith, Braintrust, Arize Phoenix, and Helicone in 2026, based on what each one is genuinely good at.
The three jobs, and why one tool rarely nails all three
Strip away the marketing and these tools do some mix of three things:
Tracing and debugging. Recording every step of an agent run — the prompts, the tool calls, the retrieved chunks, the latency at each hop — so when something breaks you can replay it instead of guessing. This is the job everyone needs first.
Evals. Scoring outputs systematically, either offline against a test dataset before you deploy, or online against live traffic. This is where you catch regressions (“did my prompt change make the summarizer worse?”) and quality drift. It’s the hardest job to do well and the one most teams underuse.
Cost and latency visibility. Knowing which feature, user, or agent run is burning tokens, and where your p95 latency lives. Sounds trivial until your bill triples and nobody can say why.
The reason no tool aces all three is that they pull architectures in different directions. Deep tracing wants SDK instrumentation in your code. Frictionless cost tracking wants a proxy you can drop in without touching anything. Rigorous evals want a data platform built around datasets and scores. Every tool here started from one of those bets and grew outward, and you can still feel the seams.
Langfuse: the open-source default
If I had to recommend one tool to a team that doesn’t know what it needs yet, it’d be Langfuse. It’s MIT-licensed, genuinely full-stack (tracing, evals, prompt management, datasets), and you can self-host the whole thing for free with no feature gating. That combination is rare.
The framework-agnostic part matters more than it sounds. Langfuse doesn’t care whether you’re on LangChain, LlamaIndex, the raw OpenAI SDK, or something you wrote yourself. You instrument with a decorator or an OpenTelemetry integration and you’re done.
Pricing on the cloud version: the Hobby tier is free with 50,000 units a month, 30-day retention, and 2 users. Core is $29/month, Pro is $199/month, and Enterprise jumps to $2,499. All paid tiers include unlimited users, and overage runs $8 per 100,000 units. That per-unit model is friendlier than per-seat once your team grows past a handful of people.
One thing worth knowing: ClickHouse acquired Langfuse in January 2026. So far nothing changed for pricing, licensing, or self-hosting, which is the outcome you’d hope for. But it’s a signal that the tool is now aimed squarely at the “store and query huge volumes of trace data” problem, which is exactly what ClickHouse is built for.
The catch with self-hosting is the one people always underestimate. “Free” Langfuse in production needs Postgres, a ClickHouse cluster, an object store, Redis, and app servers. Run all that at real volume with someone on-call for it, and the honest total lands somewhere around $3,000–4,000/month in infra plus ops time — often more than just paying for cloud Pro. Self-host because you need data sovereignty, not because you think it’s cheaper. Usually it isn’t.
LangSmith: the right call if you live in LangChain
LangSmith is made by the LangChain team, and that’s both the pitch and the caveat. If your stack is LangChain or LangGraph, LangSmith is the path of least resistance — the tracing lights up with basically zero extra work, and the newer agent-building and evaluation features are designed hand-in-glove with the framework.
Pricing is per-seat, which is the thing to watch. The free tier exists for solo developers. Plus is $39 per seat per month and includes 10,000 traces, with overage at $2.50 per 1,000 base traces (14-day retention) or $5.00 per 1,000 if you want the 400-day extended retention. Plus also unlocks up to 3 workspaces, which is what lets you separate dev, staging, and prod cleanly.
Do the math on a ten-person team and you’re at $390/month in seats before a single trace overage. Compare that to Langfuse Pro at a flat $199 with unlimited users and you can see the philosophical difference: LangSmith bills the people, Langfuse bills the data.
My honest read: LangSmith is the correct choice when you’re already committed to LangChain and value the tight integration more than portability. If you’re framework-agnostic or actively trying to avoid lock-in, the gravitational pull toward the rest of the LangChain ecosystem is a reason to look elsewhere. You’re not just picking an observability tool, you’re deepening a bet on one framework.
Braintrust: when evals are the whole point
Braintrust comes at this from the eval-first direction, and it shows. If your problem isn’t “help me debug one broken trace” but “help me systematically prove that version B of this prompt is better than version A across 500 cases,” this is the tool built for that workflow. Datasets, scorers, experiments, and side-by-side comparison are the core, not an afterthought bolted onto a tracing product.
Pricing tells you who it’s for. Starter is free with 1 GB of processed data and 10,000 scores a month. Pro is a flat $249/month — no per-seat charge, unlimited users — including 5 GB of data, 50,000 scores, and 30-day retention. Overage is $3/GB and $1.50 per 1,000 scores. Enterprise is custom.
The billing detail that trips people up: scores are metered separately from data, and every score counts. An LLM-as-a-judge evaluation, a heuristic check, a human review click — each one burns quota. If you run heavy eval workloads with LLM judges (and the whole point of Braintrust is that you will), model your score volume carefully, because that’s the number that scales with your ambition, not your traffic.
Braintrust is the priciest entry point here, and that’s the right filter. If you can’t yet articulate what you’d put in an eval dataset, you’re not ready for it and Langfuse’s evals will cover you fine. If you already have a growing test set and eval is becoming a daily ritual, the $249 buys a workflow the general-purpose tools don’t match.
Arize Phoenix: research-grade metrics, notebook-first
Phoenix is the one that comes out of the traditional ML observability world, and it feels like it. It ships with a deep library of research-backed evaluation metrics — the kind of thing you’d want if you’re doing serious RAG relevance analysis or measuring hallucination rates with something more rigorous than “ask GPT if this looks right.” It’s notebook-first, which makes it a natural fit for data scientists and ML engineers who already live in Jupyter.
Phoenix is open source and free to self-host, with one important asterisk. It’s under the Elastic License 2.0, not a fully OSI-approved license. ELv2 lets you use, modify, and run it internally for commercial purposes, but it prohibits offering Phoenix as a hosted or managed service to third parties. For 99% of teams instrumenting their own app, that restriction never bites. If you’re a platform company planning to resell observability, read the license first. It’s a meaningfully different deal from Langfuse’s MIT or Helicone’s Apache 2.0.
Phoenix connects upward to Arize’s enterprise platform (AX) when you outgrow the open-source piece and need production ML telemetry at scale. I’d reach for Phoenix when eval depth and metric rigor matter more than the polish of a hosted SaaS dashboard — and when the people using it are comfortable in a notebook, not asking for a point-and-click UI.
Helicone: the two-minute cost tracker
Helicone made the opposite architectural bet from everyone else, and for a specific need it’s the smartest choice on this list. It’s a proxy. You change your base URL to point at Helicone, and now every request flows through it and gets logged — cost, latency, tokens, all of it. Integration is one line. No SDK, no decorators, no instrumenting each step.
That proxy model is the whole story. The upside: it’s the fastest possible way to get cost and latency visibility across multiple providers, and it throws in caching, retries, and rate-limit handling for free because it sits in the request path. The downside: a proxy sees requests and responses, not the internal structure of a multi-step agent. You won’t get the rich, nested trace of “planner called tool A, which returned X, which the model then reasoned about” the way SDK-instrumented tools give you.
Helicone is open source (Apache 2.0) and self-hostable. The free cloud tier covers 100,000 requests a month, and paid plans start around $79/month for advanced features like custom alerts and team management.
Use Helicone when you need cost and latency dashboards today and you don’t need deep agent tracing or evals yet. It’s also a genuinely good option when you can’t modify the inference code to add SDK instrumentation — route the traffic through a self-hosted Helicone instance and you get centralized logging without touching the model server. Plenty of teams run Helicone for cost alongside a second tool for tracing, and that’s a reasonable stack, not a failure.
Picking by what you actually need
Cut through it like this:
- You don’t know yet, or you want portability and self-host optionality → Langfuse. It’s the safe default that covers all three jobs decently and doesn’t lock you in.
- You’re all-in on LangChain/LangGraph → LangSmith. The integration tax is worth it; just watch the per-seat bill as you grow.
- Evals are your central problem → Braintrust. Nothing else here treats systematic evaluation as the main event.
- You need research-grade metrics and your team lives in notebooks → Arize Phoenix, license caveat noted.
- You need cost and latency visibility this afternoon → Helicone. One line, done.
The pricing trap nobody warns you about
The sticker price is the least interesting number. What actually determines your bill is the billing unit, and each tool picked a different one: LangSmith bills per seat, Langfuse bills per unit of trace data, Braintrust bills data volume and scores separately, Helicone bills request count.
That means the cheapest tool depends entirely on your shape. A twelve-person team running low traffic gets punished by LangSmith’s per-seat model and rewarded by Langfuse’s flat unlimited-user tiers. A tiny team running massive eval sweeps will feel Braintrust’s score metering long before headcount matters. A high-volume, low-complexity app gets cost visibility cheapest through Helicone’s request-based free tier.
So don’t compare the $199 to the $249 to the $39. Estimate your traces per month, your team size, and your eval score volume for the next year, then run each tool’s pricing against those numbers. The winner usually isn’t the one with the lowest headline price.
If you’re just starting, spin up self-hosted Langfuse or the Helicone free tier this week and get one real agent instrumented. Seeing your own traces — the actual tool calls, the actual token counts — tells you more about which of the three jobs you really need than any comparison table will, including this one.