Observability and evals
These tools measure agents; nobody measures them. Compare them on what you can verify: open-source license and self-hosting, OpenTelemetry support, and unit prices.
Tools
Alphabetical. Links go to each vendor's own site.
| Tool | What it does | Evidence | Price or terms |
|---|---|---|---|
| AgentOps | session replays and cost tracking; Python SDK open source | none yet | free tier + paid |
| Arize Phoenix | OpenTelemetry-native tracing + evals (Apache-2.0) | none yet | open source free; cloud from $50/mo |
| Braintrust | eval-first platform + AI gateway | none yet | free $10 credits; Pro $249/mo |
| Langfuse | tracing, prompts, evals; MIT core, self-hostable | none yet | Hobby free (50k units/mo); Core $29/mo |
| LangSmith | LangChain's agent engineering platform | none yet | free 5k traces/mo; Plus $39/seat/mo |
Caveats
- Self-hosting Langfuse v3 needs Postgres, ClickHouse, Redis and S3.