How to Trace and Debug LLM Apps With Langfuse

When an LLM feature misbehaves in production, your application logs rarely tell you why. The exact prompt, the retrieved context, the token counts, the latency, and the model's raw response all live in different places, and by the time you stitch them together the user has churned. Langfuse pulls all of that into a single trace so you can see precisely what the model saw and what it cost. The short version: run Langfuse with Docker Compose, pip install langfuse, wrap your code with the @observe() decorator or swap in the drop-in OpenAI client, and every call lands in the UI as a nested, inspectable trace.

What Do You Need Before You Start?

The self-hosted stack is heavier than it looks, because ClickHouse, Redis, and MinIO all run alongside the app. Plan for a real machine, not a 1 GB VPS.

• Docker and the Docker Compose plugin. The docs recommend at least 4 cores, 16 GiB of memory, and 100 GB of storage for a production VM; a laptop with 8 GB will start it but strain under load.
• Python 3.9 or newer for the SDK examples below.
• An LLM provider key (this guide uses OpenAI) or any OpenAI-compatible endpoint, including a local vLLM server.
• A few minutes of patience on first boot while ClickHouse initializes.

How Do You Stand Up Langfuse and Capture Your First Trace?

Follow these six steps in order. When you finish, you will have a running instance and one real trace to inspect. You can complete the whole task from this section alone.

1. Clone the repo and rotate the secrets. Every value marked # CHANGEME in docker-compose.yml must be replaced with a long random string. Generate them with openssl. The ENCRYPTION_KEY in particular has to be exactly 64 hex characters (a 256-bit key), or the worker refuses to start.

   git clone https://github.com/langfuse/langfuse.git
   cd langfuse
   
   # generate values for the CHANGEME lines in docker-compose.yml
   openssl rand -hex 32   # use for NEXTAUTH_SECRET
   openssl rand -hex 32   # use for SALT
   openssl rand -hex 32   # use for ENCRYPTION_KEY (must be 64 hex chars)

2. Start the stack. This pulls and launches all six services: the web app, the worker, ClickHouse, Postgres, Redis, and MinIO. First boot takes two to three minutes.

   docker compose up -d
   docker compose ps        # confirm every service is healthy

3. Create your account and project. Open http://localhost:3000. The first user you register becomes the instance owner. Create an organization, then a project inside it. Open Project Settings, API Keys and copy the public key (pk-lf-...) and secret key (sk-lf-...).
4. Install the SDK and export the keys. Set LANGFUSE_BASE_URL to your own instance so traces stay local.

   pip install langfuse openai
   
   export LANGFUSE_PUBLIC_KEY="pk-lf-..."
   export LANGFUSE_SECRET_KEY="sk-lf-..."
   export LANGFUSE_BASE_URL="http://localhost:3000"
   export OPENAI_API_KEY="sk-proj-..."

5. Instrument a function. The drop-in OpenAI client traces every call with no other change. Wrapping the outer function in @observe() groups the calls it makes into one parent trace.

   from langfuse import observe, get_client
   from langfuse.openai import openai   # drop-in replacement
   
   @observe()
   def answer(question: str) -> str:
       resp = openai.chat.completions.create(
           model="gpt-4o-mini",
           messages=[{"role": "user", "content": question}],
       )
       return resp.choices[0].message.content
   
   if __name__ == "__main__":
       print(answer("Explain a KV cache in one sentence."))
       get_client().flush()   # required in short-lived scripts

6. Open the trace. Run the script, then refresh the Tracing tab in the UI. You will see one trace named answer with the OpenAI generation nested inside it, complete with the prompt, the completion, latency, token counts, and estimated cost.

If you plan to run this inside a web server or notebook instead of a script, you can skip the explicit flush(). The background thread sends events on its own schedule; you only need flush() when the process is about to exit.

How Should You Instrument a Real Application?

The quickstart traces a single call. Production apps chain retrieval, tool calls, and multiple model invocations, and the value of Langfuse is seeing them as one tree.

How Do You Trace a Bare OpenAI Call?

You already did, in step five. The only change from the standard SDK is the import: from langfuse.openai import openai. Everything downstream, including streaming responses and async calls, is captured automatically. Pass name="..." to a call to label it in the UI.

How Do You Trace a LangChain or LangGraph Chain?

Use the callback handler instead of the decorator. Attach it through the config argument on any invoke, and Langfuse records each LLM, tool, and retriever step in the chain. This is the cleanest way to see why a LangChain workflow or an agent took the path it did.

from langfuse import get_client
from langfuse.langchain import CallbackHandler

langfuse_handler = CallbackHandler()

result = chain.invoke(
    {"input": "your input"},
    config={"callbacks": [langfuse_handler]},
)

get_client().shutdown()   # flush before a short-lived process exits

How Do You Add Scores and User Feedback?

A trace tells you what happened; a score tells you whether it was any good. Attach numeric or categorical scores to a trace, either from a thumbs-up in your UI or from an automated check. Those scores become filterable columns, so you can pull up every low-rated response and read exactly what the model saw. This is the bridge between passive tracing and the kind of offline testing covered in running evals with Promptfoo.

How Does Langfuse Compare to Other Tracing Tools?

Several tools solve LLM observability. The practical dividing lines are whether you can self-host without an enterprise contract, the license, and where trace data actually lives.

Langfuse and Helicone both back their traces with ClickHouse, which is why aggregate queries over hundreds of thousands of traces stay fast. Helicone leans on a proxy in front of your provider; Langfuse instruments in your own code, which gives you finer control over what a trace contains.

Which Signals Actually Matter When You Debug?

Once traces flow, resist the urge to stare at all of them. Three views catch most real problems.

• Latency breakdown. A slow response is usually one slow step. The trace tree shows whether the retriever, a tool call, or the model itself ate the time.
• Cost per trace. Langfuse maps token usage to a price per model. Sort by cost and you will find the runaway prompt that quietly doubled your bill. Pair this with prompt caching to bring it down.
• The retrieved context. For RAG, the model is only as good as what you fed it. Read the actual chunks in the trace before you blame the model, then tune retrieval or add a reranker.

What Breaks First, and How Do You Fix It?

Almost every first-run problem falls into one of these.

• Traces never appear in the UI. Two usual causes. In a short script you forgot flush() or shutdown(), so the process exited before sending. Or your SDK is pointed at the cloud: older code used LANGFUSE_HOST, current v3 uses LANGFUSE_BASE_URL. Set the one your SDK version reads, to http://localhost:3000.
• The worker or ClickHouse container will not start. Check docker compose logs. The most common cause is an ENCRYPTION_KEY that is not exactly 64 hex characters. The second is too little memory; give the host more RAM.
• 401 or 403 on ingestion. You swapped the public and secret keys, or the keys belong to a different project than the one you are viewing. Copy both again from Project Settings.
• Cost shows as zero. Langfuse only prices models it recognizes. If you serve a custom or self-hosted model, define a model price in the project settings so token usage converts to a cost.
• Every call is its own trace. You called the model outside any @observe() function or without the callback handler, so nothing groups them. Wrap the entry point.

What Should You Do Next?

You have traces. Turn them into a feedback loop.

• Wire user feedback into scores, then filter for the worst-rated traces and read them as a triage queue. Start with the scores documentation.
• Capture a set of real production traces as a dataset and replay them against a new prompt or model before you ship. See the Python instrumentation guide for advanced nesting and metadata.
• Add tracing to your framework code with the LangChain and LangGraph integration, or trace any OpenAI-compatible endpoint with the OpenAI drop-in.
• Harden the deployment before it faces real traffic. The single-machine Compose setup lacks high availability and backups; review the Docker Compose self-hosting notes and move to Kubernetes if you need scale.

Sources

• Langfuse: Self-hosting with Docker Compose
• Langfuse: Python SDK instrumentation
• Langfuse: OpenAI Python integration
• Langfuse: LangChain and LangGraph integration
• Langfuse source repository (GitHub)