Tracing quickstart

Observability is a critical requirement for applications built with Large Language Models (LLMs). LLMs are non-deterministic, which means that the same prompt can produce different responses. This behavior makes debugging and monitoring more challenging than with traditional software. LangSmith addresses this by providing end-to-end visibility into each request. Every request generates a trace, which is a complete record of what happened. Within a trace are individual runs, the specific operations your application performed, such as an LLM or tool call. In this quickstart, you will set up a simple agent with function calling (also known as tool calling) that’s fully traced in LangSmith.

Prerequisites

Before you begin, make sure you have:

A LangSmith account: Sign up or log in at smith.langchain.com.
A LangSmith API key: Go to Settings > API Keys and create a new API key.
An OpenAI API key: Generate this from the OpenAI dashboard.

This quickstart will use OpenAI Responses API. You can adapt this to your LLM provider of choice. If you’re using Anthropic, use the Anthropic wrapper to trace your calls. For other providers, use the traceable wrapper.

If you’re building an application with LangChain or LangGraph, you can enable LangSmith to automatically trace your application by setting environment variables. Get started by reading the guides for tracing with LangChain or tracing with LangGraph.

1. Create a directory and install dependencies

In your terminal, create a directory for your project and install the dependencies in your environment:

mkdir ls-observability-quickstart && cd ls-observability-quickstart
python -m venv .venv && source .venv/bin/activate
python -m pip install --upgrade pip
pip install -U langsmith openai

2. Set up environment variables

Set the following environment variables:

LANGSMITH_TRACING
LANGSMITH_API_KEY
OPENAI_API_KEY (or your LLM provider’s API key)
(optional) LANGSMITH_WORKSPACE_ID: If your LangSmith API key is linked to multiple workspaces, set this variable to specify which workspace to use.

export LANGSMITH_TRACING=true
export LANGSMITH_API_KEY="<your-langsmith-api-key>"
export OPENAI_API_KEY="<your-openai-api-key>"
export LANGSMITH_WORKSPACE_ID="<your-workspace-id>"

If you’re using Anthropic, use the Anthropic wrapper to trace your calls. For other providers, use the traceable wrapper.

To send traces to a specific project, use the LANGSMITH_PROJECT environment variable. If this is not set, LangSmith will create a default tracing project automatically on trace ingestion.

3. Build your application

In this step, you’ll build a simple agent that uses OpenAI’s Responses API with function calling. This example shows a weather assistant that can look up weather information. For simplicity, you’ll return a static forecast.

# main.py
import json
from openai import OpenAI

client = OpenAI()

# Define callable tools
tools = [
    {
        "type": "function",
        "name": "get_weather",
        "description": "Get the weather for a city.",
        "parameters": {
            "type": "object",
            "properties": {
                "city": {"type": "string", "description": "City name, e.g. Tokyo"}
            },
            "required": ["city"],
        },
    }
]

def get_weather(city: str):
    return {"city": city, "forecast": "72°F and sunny"}

def run_agent(messages: list):
    """
    messages: [{"role": "...", "content": "..."}]
    returns: {"messages": [...]} (full message history)
    """
    # Create a running input list we will add to over time
    input_list = list(messages)

    # Prompt the model with tools defined
    r1 = client.responses.create(
        model="gpt-5-mini",
        tools=tools,
        input=input_list,
        instructions="You are a helpful assistant. Use tools when needed.",
    )

    # Save model output items for subsequent requests
    input_list += r1.output

    # Execute tool call (minimal, single-tool example)
    for item in r1.output:
        if item.type == "function_call" and item.name == "get_weather":
            args = json.loads(item.arguments)
            result = get_weather(**args)

            # Provide function call results to the model
            input_list.append(
                {
                    "type": "function_call_output",
                    "call_id": item.call_id,
                    "output": json.dumps(result),
                }
            )
            break

    # Second model call (model uses tool result to answer)
    r2 = client.responses.create(
        model="gpt-5-mini",
        tools=tools,
        input=input_list,
        instructions="Answer the user using the tool result.",
    )

    input_list += r2.output

    return {"messages": input_list}

if __name__ == "__main__":
    result = run_agent(
        [{"role": "user", "content": "What's the weather in San Francisco?"}]
    )
    print(result["messages"])

4. Add Tracing

Now that we’ve defined our application, let’s add tracing to it with just three changes:

Import LangSmith utilities - wrap_openai and traceable.
Wrap the OpenAI client - Automatically traces all OpenAI calls.
Wrap tool functions - Creates runs for tool execution, can also be used to trace other functions.

Follow LangSmith Tracing Standards to ensure proper message rendering and to use features like Polly, LangSmith Fetch, and multi-turn evals:

Include full conversation history in each trace (not just the current message)
Use {"messages": [...]} as the top-level structure for inputs and outputs

Learn more: Tracing Standards

# main.py
import json
from openai import OpenAI
from langsmith import traceable
from langsmith.wrappers import wrap_openai

# Traced OpenAI client
client = wrap_openai(OpenAI())

# Define callable tools
tools = [
    {
        "type": "function",
        "name": "get_weather",
        "description": "Get the weather for a city.",
        "parameters": {
            "type": "object",
            "properties": {
                "city": {"type": "string", "description": "City name, e.g. Tokyo"}
            },
            "required": ["city"],
        },
    }
]

@traceable(name="get_weather", run_type="tool")
def get_weather(city: str):
    return {"city": city, "forecast": "72°F and sunny"}


@traceable(name="weather_agent")
def run_agent(messages: list):
    """
    messages: [{"role": "...", "content": "..."}]
    returns: {"messages": [...]} (full message history)
    """
    # Create a running input list we will add to over time
    input_list = list(messages)

    # Prompt the model with tools defined
    r1 = client.responses.create(
        model="gpt-5-mini",
        tools=tools,
        input=input_list,
        instructions="You are a helpful assistant. Use tools when needed.",
    )

    # Save model output items for subsequent requests
    input_list += r1.output

    # Execute tool call (minimal, single-tool example)
    for item in r1.output:
        if item.type == "function_call" and item.name == "get_weather":
            args = json.loads(item.arguments)
            result = get_weather(**args)

            # Provide function call results to the model
            input_list.append(
                {
                    "type": "function_call_output",
                    "call_id": item.call_id,
                    "output": json.dumps(result),
                }
            )
            break

    # Second model call (model uses tool result to answer)
    r2 = client.responses.create(
        model="gpt-5-mini",
        tools=tools,
        input=input_list,
        instructions="Answer the user using the tool result.",
    )

    input_list += r2.output

    return {"messages": input_list}

if __name__ == "__main__":
    result = run_agent(
        [{"role": "user", "content": "What's the weather in San Francisco?"}]
    )
    print(result["messages"])

5. Run the code

Execute the script:

python main.py

6. View the Trace in LangSmith

In the LangSmith UI, navigate to the default Tracing Project for the workspace you specified in Step 2). You’ll see the agent’s trace. It should look like this: https://smith.langchain.com/public/c1ac97f8-a022-4e0f-8380-59cab47f28e5/r.

Next steps

Here are some topics you might want to explore next:

Tracing integrations provide support for various LLM providers and agent frameworks.
Filtering traces can help you effectively navigate and analyze data in tracing projects that contain a significant amount of data.
Trace a RAG application is a full tutorial, which adds observability to an application from development through to production.
Sending traces to a specific project changes the destination project of your traces.

After logging traces, use Polly to analyze them and get AI-powered insights into your application’s performance.

Edit this page on GitHub or file an issue.

Connect these docs to Claude, VSCode, and more via MCP for real-time answers.

Tracing setup

Configuration & troubleshooting

Viewing & managing traces

Automations

Feedback & evaluation

Monitoring & alerting

Data type reference

Prerequisites

1. Create a directory and install dependencies

2. Set up environment variables

3. Build your application

4. Add Tracing

5. Run the code

6. View the Trace in LangSmith

Next steps

Tracing setup

Configuration & troubleshooting

Viewing & managing traces

Automations

Feedback & evaluation

Monitoring & alerting

Data type reference

​Prerequisites

​1. Create a directory and install dependencies

​2. Set up environment variables

​3. Build your application

​4. Add Tracing

​5. Run the code

​6. View the Trace in LangSmith

​Next steps

Prerequisites

1. Create a directory and install dependencies

2. Set up environment variables

3. Build your application

4. Add Tracing

5. Run the code

6. View the Trace in LangSmith

Next steps