Skip to main content
This page defines the tracing standards required to unlock important capabilities in LangSmith. Structuring your traces correctly is required to make the most of LangSmith: 1. Rich debugging experience: LangSmith provides structured rendering of message lists, making it easier to visualize and understand the interaction history. 2. LangSmith features: Features like Polly, LangSmith Fetch and multi-turn evals require properly structured traces to work correctly.

​
Understanding Threads, Traces and Runs

LangSmith has three levels that work together to capture your agent’s behavior:

​
Threads

A Thread groups multiple interactions together so you can see the history over time. In the LangSmith UI, threads can be viewed in the “Threads” tab within a Tracing Project.

​
Traces

A Trace represents a single request/response cycle, also known as a “turn”. It contains everything that happened during one execution of an agent. Multiple traces, grouped together, form a thread. In the LangSmith UI, a trace is the “parent” node in the execution tree.

​
Runs

A Run is an individual operation within a trace: an LLM call, a tool execution, middleware, or any other step in your agent’s process. One or more runs make up a trace. In the LangSmith UI, runs are “children” and “grandchildren” (and onward as needed) nodes in the tree. If a trace fails, you would examine individual runs to identify which step had an error.
LangSmith UI showing the threads table.

​
Trace Structure

Structure your traces so each one is self-contained and replayable:
  • Input: The new message or instruction. For example: What’s the status of the Tokyo shipment?
  • Output: Output is the state of the interaction after the turn. Represents the current memory (interaction history + new messages) of the agent after the turn is complete. For example: What’s the status of the Tokyo shipment? -> Checked the shipping logs -> Found the Tokyo tracking number -> Confirmed it arrived at port -> Told the user.
The benefit of this approach is any trace can be understood independently without needing to search for prior context. Both inputs and outputs of a trace must have messages as a top-level key that represents the interaction. Input: 
{
  "messages": [
    // User's current request
  ],
  "additional_fields":{
    // Optional additional fields
  }
}
Output: 
{
  "messages": [
    // History from prior turns + user request + LLM response
  ],
  "additional_fields":{
    // Optional additional fields
  }
}
You can think of the output as the intraction state after the turn, which is why outputs should include the full message history—not just the latest response. The output is a complete “receipt” of what happened in that turn. View Threads for more information on how to configure threads, and the expected trace structure within a thread.

​
Message Format Standards

In the above section, we covered the expected structure of traces. This section covers the expected structure of messages within a trace.
If you don’t log your LLM traces in the suggested formats, you will still be able to log the data to LangSmith, but it may not be processed or rendered in expected ways, and you will not be able to use some features, like Polly, LangSmith Fetch and multi-turn evals.

​
Required Structure

The messages key should be follow LangChain, OpenAI Chat Completions or Anthropic messages formats. If you’re using other models, or tracing a custom model, you’ll need to modify the structure of the messages array to follow one of the supported schemas for best results.
If you’re using LangChain OSS to call language models or LangSmith wrappers (OpenAI, Anthropic), this formatting is handled automatically.

​
Schema

​
Examples

inputs = {
  "messages": [
    {
      "role": "user",
      "content": [
        {
          "type": "text",
          "text": "What's the capital of France?"
        }
      ]
    }
  ]
}

outputs = {
  "messages": [
    {
      "role": "assistant",
      "content": [
        {
          "type": "text",
          "text": "The capital of France is Paris."
        },
        {
          "type": "reasoning",
          "text": "This is a straightforward geography question..."
        }
      ]
    }
  ]
}

​
FAQ

If you’re using a custom input or output format, you can convert it to a LangSmith compatible format using process_inputs and process_outputs with the @traceable decorator to transform your custom format into LangSmith-compatible structure:For details, refer to the Log LLM calls page.
Log the final accumulated result after streaming completes:
@traceable(run_type="llm")
def call_llm_streaming(messages):
    chunks = []

    # Stream response
    for chunk in client.chat.completions.create(
        model="gpt-4",
        messages=messages,
        stream=True
    ):
        chunks.append(chunk.choices[0].delta.content or "")

    # Log complete response
    full_text = "".join(chunks)
    return {
        "role": "assistant",
        "content": [{"type": "text", "text": full_text}]
    }
Don’t log individual chunks—log the complete message once streaming finishes.
Include all tool calls and their results in the message sequence:
outputs = {
  "messages": [
    {
      "role": "assistant",
      "content": [
        {"type": "tool_call", "name": "search", "args": {...}, "id": "call_1"},
        {"type": "tool_call", "name": "calculate", "args": {...}, "id": "call_2"}
      ]
    },
    {
      "role": "tool",
      "tool_call_id": "call_1",
      "content": [{"type": "text", "text": "Search results..."}]
    },
    {
      "role": "tool",
      "tool_call_id": "call_2",
      "content": [{"type": "text", "text": "Calculation result..."}]
    },
    {
      "role": "assistant",
      "content": [{"type": "text", "text": "Based on the search and calculation..."}]
    }
  ]
}
The tool_call_id links each result to its corresponding call.
Traces are stored according to retention policies, which can only be modified if you are on self-hosted LangSmith.
  1. Use references instead of raw data: Store files externally and use the id field
  2. Filter before logging: LangSmith provides multiple approaches to protect your data before it’s sent to the backend here.
  3. Configure Trace Deletion: Set up trace deletion rules.
Edit this page on GitHub or file an issue.
Connect these docs to Claude, VSCode, and more via MCP for real-time answers.