LangSmith alternatives to capture logs for LangChain

LangSmith, developed by the LangChain team, is a great tool for observing and managing the lifecycle of LangChan apps.

It has a few limitations though: it;s not open-source, has limited compatibility with frameworks outside of LangChain, and due to its popularity, personalized support from their team comes at a significant extra cost.

Below, we explore 5 viable alternatives to LangSmith, each offering comparable functionalities and more.

Lunary

Lunary is a platform for AI devs that features almost all the cool stuff from LangSmith, plus:

• 100% open-source and self-hostable
• Fully compatible with LangChain and other libraries
• Clean and simple UI
• Manage prompt templates, run evaluations & more
• New features every months
• Experiment with models such as Mistral, Claude v3, Gemini Pro
• Bootstrapped company that cares more about customers than investors

Lunary is free to use until you hit 1000 logs collected per day, which should be enough until you go into production.

The paid plans start at $29 per month for 4 users, a fraction of the cost of LangSmith.

PromptLayer

PromptLayer is another great tool more focused on managing prompts. It also offers prompt management as well as analytics.

However, PromptLayer pricing structure is not clear from looking at their website, and is not open-source which may be a deal-breaker for some projects.

Arize

Another big player in the ML observability game, Arize boasts a powerful set of tools around all steps of building a machine-learning application. Arize is not generally focused on LLMs but has recently taking a big turn in building LLM features.

The drawback with Arize is that it's more complicated to learn to use. It's also not open-source and offers only 1K logs per month in the free plan (which is 30x less than Lunary.)

Helicone

If your observability needs are simpler than what LangSmith offers, Helicone can be a great choice.

It's a simple, open-source tool that offers a clean UI and is easy to use.

Helicone originally works as a proxy, giving you a replacement URL for your OpenAI calls that automatically logs the requests and responses. It's a great way to get started with observability without having to add an extra library to your code.

Weights & Biases

Weights & Biases (known as Wandb) is a popular tool for MLOps that's been around for a while. It offers a wide range of features for LLMs, including tracing, logging, fine-tuning, evaluations, visualization, and collaboration tools.

The platform and integration might be a bit overwhelming to get accustomed to, but it's a great choice for ML-heavy projects that need absolutely everything in one place.

Note that it's not open-source, but it's free to use for small projects. The pricing is then done per hour which might be a bit confusing to relate to your usage.

For larger enterprises with big ML teams, it's a great choice. WandDB even used by the teams at OpenAI.