Bring your AI app
to production.

Observability, analytics and tests for AI agents and chatbots.

Get Started →

free up to 1k events / day

Stay on top of costs

Monitor requests and costs segmented by user and model. Optimize your prompts and save money.

Debug complex agents

Replay agent executions with traces and find out what went wrong and where.

Lift the veil on users

Track user activity and costs. Find out who are the power users.

Be alerted when things go wrong

Write assertions and English or code and run them against your agents to make sure they work as expected.

Replay user conversations

Replay user conversations and identify gaps in your chatbot's knowledge.

Capture user feedback

Collect feedback from your users and improve your agents.

Create training datasets

Label your outputs based on tags and feedback then easily export datasets to fine-tune your models, improving your app's quality and reducing costs.

We worked hard to make it easy.

Integrate one of our SDK in minutes, not hours.

import openai
from llmonitor import monitor

monitor(openai)

openai.api_key = "sk-..."

chat_completion = openai.ChatCompletion.create(
  model="gpt-3.5-turbo", 
  messages=[{"role": "user", "content": "Hello world"}]
)

Explore the docs →

Integrated in the ecosystem.

We recorded a 1 minute demo video.

(expect better as we ship a lot)

Clone, contribute and remix.

We are committed to building in the open and sharing everything we build.

llmonitor: 192

Get started in minutes.

Run it on your own or get started in minutes with our hosted version.

Get Started (it's free)

Browse the docs

Open Source

Self Hostable

Agent Tracing

Analytics

Cost Analysis

User Tracking

LangChain Support