LangSmith: Debug and Play with your Agentic AI Project

When working in a multi-agent system, the process often becomes complex with multiple chains and communication between various nodes. You want to understand how the agents are interacting, how the state is updated by each agent, what input goes to the agent and what its output is, and how much time it takes to respond. You would also want to understand whether LLM calls are performed by each agent and how long it takes to generate the output.

If you are improving your prompts iteratively, you would also want to version them so that you can keep track of prompts and the success or failure of each prompt style.

This is where LangSmith comes into the picture. LangSmith is developed by the team behind LangChain and is built to trace and observe your Agentic AI project. It traces how your project or your entire agentic flow works, to put it in a very simple manner.

It is very intuitive, easy to read and maintain, and helps track different runs, making it easier to build and improve your Agentic AI code. Who doesn't want to improve, right? Be it for your personal or professional projects, improving products involves having a good observability tool for both your mistakes and successes.

What features does LangSmith provide?

LangSmith provides an interface for your Agentic AI project. When your application processes a request, the entire execution is captured as a trace. Inside that trace, each step or component execution is recorded as a run.

Each run is essentially a visual representation of inputs, processes, and outputs.

It shows the list of nodes defined in the workflow. If you are using LangGraph, a node is simply a function that performs some action. In many cases, a node may represent an agent that takes action on the input it is provided with, but nodes can also represent other steps such as tool calls or processing functions.

Each node may also interact with the state of the system. State is something that flows throughout the nodes. It is like the current status of your input as it moves through the workflow.

As the input passes through each node, the node performs some action and may write its output to the state. Until the final node is called, the state keeps getting updated, and this eventually becomes the final output of your multi-agent framework.

This state management and the input/output of each node can be seen in LangSmith runs. This makes your project flow visible and easier to track and build. Whether the state is getting updated correctly by each node or not, or if there are issues in tool access, everything becomes clearer when you track the state in LangSmith.

Performance visibility

LangSmith also enables you to see the time each node takes to process. This helps identify bottlenecks in the workflow and shows which node is taking the maximum time.

Based on this, you can make improvements to a particular node. You can check whether the time is spent on tool access, prompt size, embedding model generation, or an LLM API call. Accordingly, you can optimize the respective node.

Cost awareness

LangSmith also provides an estimate of the cost incurred for a run based on the token usage and model pricing. This helps you stay cost-conscious during development.

You can compare different runs with respect to cost and decide whether it aligns with the budget and efficiency of the project.

For example, if a particular node can perform a task using a simple function, then calling an LLM may not be required for that decision.

For instance, searching for a specific word in the input can easily be done using regex and would not require an LLM call. These small practices can help reduce cost significantly.

Integrating LangSmith with a multi-agent project

Having discussed the features and benefits of LangSmith, let's now look at its integration with a multi-agent project. It is relatively simple and easy.

You need to log in to langsmith.com, generate an API key, and store it in your .env file. You can also define the project name in the .env file. This links your project codebase with the LangSmith portal.

Once the LangSmith configuration and environment variables are loaded in your project, frameworks like LangChain or LangGraph automatically send tracing information to LangSmith when the application runs.

When you run your workflow, each node execution gets reported to LangSmith and is displayed in the LangSmith UI.

When you run the project, the LangSmith trace becomes visible with your project name. Every time you run the project, you will see different traces and runs with the overall results and intermediate steps.

Final thoughts

LangSmith provides an intuitive and comprehensive way to observe multi-agent AI projects.

In the world of agents, it is important not only to build a good system but also to understand how it behaves internally. Observability helps you debug issues, optimize performance, and control costs.

Tools like LangSmith make it much easier to understand what is happening inside your agentic workflow and help you build systems that are reliable and long-lasting.