Core Concepts
Runnables and Chains
Runnables represent the fundamental unit of execution in LangChain, providing a unified interface for defining tasks that can be invoked, batched, or streamed. Chains are constructed by composing these units together, allowing developers to create complex sequences that manage data flow automatically. You reach for these abstractions whenever you need to build robust, modular LLM applications where reliability, traceability, and maintainability are critical production requirements.
The Runnable Protocol
The Runnable protocol is the cornerstone of modern LangChain, designed to normalize how different components interact. By standardizing methods like 'invoke', 'stream', and 'batch', the framework ensures that every component behaves predictably regardless of its internal complexity. When you define a component as a Runnable, you gain access to built-in asynchronous support and lifecycle hooks without writing extra infrastructure code. This abstraction works because it treats every piece of a pipeline—from simple prompt templates to complex model calls—as a function that accepts input and returns output. The magic lies in the 'pipe' operator, which allows you to stitch these units together into a continuous stream of data. Understanding this protocol is essential because it is the interface that enables seamless integration across the entire ecosystem, ensuring that your logic remains decoupled from the underlying model implementation details.
from langchain_core.runnables import RunnableLambda
# A simple runnable that transforms input into uppercase
# The Runnable protocol requires an 'invoke' method or a callable structure
uppercase_runnable = RunnableLambda(lambda x: x.upper())
# We invoke it directly to see the execution flow
result = uppercase_runnable.invoke("hello langchain")
print(result) # Outputs: HELLO LANGCHAINComposing Chains with Pipe
Chains are created by composing multiple Runnables using the pipe operator ('|'). When you connect these components, you are essentially defining a directed graph where the output of one step becomes the input to the next. This mechanism works by creating a new Runnable object that sequentially executes the components in the chain. The reason this is powerful is that the chain itself maintains the Runnable interface; you can treat a complex chain just like a single, simple component. This recursive composition allows you to build sophisticated workflows that are easy to debug because each link in the chain is isolated and observable. By composing these units, you ensure that the entire pipeline adheres to the same error handling and input validation standards, significantly reducing the surface area for bugs when data moves between different processing states in your LLM application architecture.
from langchain_core.prompts import ChatPromptTemplate
from langchain_openai import ChatOpenAI
# Create a simple chain using the pipe operator
prompt = ChatPromptTemplate.from_template("Translate this to French: {text}")
model = ChatOpenAI(model="gpt-4")
# The pipe operator connects prompt output to model input
chain = prompt | model
# Invoking the chain passes the input through the pipeline
response = chain.invoke({"text": "Hello world"})
print(response.content)Data Persistence with RunnablePassthrough
A common challenge in building chains is needing to carry forward parts of the original input while processing other sections through a chain. This is where RunnablePassthrough becomes indispensable. It allows data to pass through a specific point in a pipeline without being modified, which is crucial for maintaining context. When you use a dictionary or a specific key in your input, you often need to combine variables to satisfy the requirements of a downstream prompt template. RunnablePassthrough works by allowing you to inject additional data into the execution flow, effectively 'merging' user input with retrieved context or system state. By using this, you maintain a clean data flow where you can explicitly define which variables are processed by the LLM and which remain static, ensuring that the state remains coherent throughout the duration of the request life-cycle in your chain execution.
from langchain_core.runnables import RunnablePassthrough
# Setup a chain that passes through extra context
# We use RunnablePassthrough to keep 'user_input' intact
chain = (
{"context": lambda x: "Knowledge base info", "question": RunnablePassthrough()}
| ChatPromptTemplate.from_template("Context: {context}\nQuestion: {question}")
| ChatOpenAI()
)
# The input string is passed into the 'question' key while context is injected
print(chain.invoke("What is LangChain?").content)Binding Parameters to Runnables
Sometimes a Runnable needs specific configuration parameters, like a different temperature for a model or specific stop sequences, that are not part of the input data itself. The '.bind()' method allows you to attach these configurations directly to a component, creating a new version of that component that carries its settings with it. This works because the bind method creates a closure around the original object, ensuring that whenever the component is executed, the bound parameters are automatically included in the invocation. This is superior to manually passing settings because it keeps your business logic separate from your configuration logic. By binding parameters, you ensure that specific chain components are always executed under the correct settings, which is essential for ensuring consistent behavior in production environments where different parts of your application might require distinct model configurations or constraints.
# Bind specific model settings to the model object
model = ChatOpenAI(model="gpt-4")
# Create a specific version of the model with a custom temperature
creative_model = model.bind(temperature=0.9)
# This model will now consistently use a high temperature
response = creative_model.invoke("Write a poem about code.")
print(response.content)Managing Fallbacks and Error Handling
Production applications must be resilient to errors, such as API rate limits or model outages. LangChain simplifies this with the 'with_fallbacks' method, which allows you to define a secondary sequence to execute if the primary one fails. This works by wrapping the primary Runnable in a fault-tolerant logic layer that monitors the execution output. If an exception occurs, the chain automatically pivots to the fallback sequence, ensuring that the user experience is not disrupted by backend service instability. This is a fundamental concept in building robust LLMs because it decouples your application's reliability from the availability of any single vendor or model endpoint. By defining clear fallback paths, you ensure that your chain is capable of gracefully degrading or attempting alternative solutions, which is a hallmark of professional-grade software engineering within the context of generative artificial intelligence systems.
# Define a model that fails
primary_model = ChatOpenAI(model="invalid-model")
secondary_model = ChatOpenAI(model="gpt-4")
# Define a chain with a fallback
chain = primary_model.with_fallbacks([secondary_model])
# If primary fails, the chain automatically attempts the fallback
result = chain.invoke("Hello, are you there?")
print(result.content)Key points
- The Runnable protocol standardizes component interaction through methods like invoke, stream, and batch.
- The pipe operator is the primary mechanism for composing complex sequences from individual components.
- RunnablePassthrough enables the injection of static data into pipelines while preserving original input.
- The .bind method allows for the encapsulation of configuration parameters within a specific runnable instance.
- Chains maintain the Runnable interface, allowing for recursive composition of complex LLM workflows.
- Fallbacks provide a built-in mechanism for handling errors and ensuring reliability in production.
- Isolation of components within a chain allows for easier testing, debugging, and observation of data flow.
- Normalization of interfaces ensures that all parts of the framework interact predictably without model-specific code.
Common mistakes
- Mistake: Manually chaining components without using the LCEL pipe operator (|). Why it's wrong: It breaks the abstraction layers and makes debugging harder. Fix: Use the pipe operator which handles input/output schema validation automatically.
- Mistake: Assuming every object in a chain can handle a raw string as input. Why it's wrong: Chains often expect dictionaries or specific LangChain objects. Fix: Use `RunnablePassthrough` or prompt templates to structure input data correctly.
- Mistake: Over-reliance on the deprecated `LLMChain` legacy class. Why it's wrong: It creates rigid structures that are hard to compose. Fix: Compose discrete `Runnable` objects using LCEL for better reusability.
- Mistake: Forgetting to define output parsers in a sequence. Why it's wrong: The LLM returns a string, but your next component might expect a structured object. Fix: Append `StrOutputParser()` or custom JSON parsers to the end of your LLM call.
- Mistake: Passing state through chains using global variables instead of `RunnablePassthrough`. Why it's wrong: It creates race conditions and prevents parallel execution. Fix: Pass context through the chain using `assign` or `RunnablePassthrough.assign()` to maintain state immutability.
Interview questions
What is a Runnable in LangChain, and why is it considered the fundamental building block?
A Runnable is a standard interface implemented by most LangChain components, including prompts, LLMs, and retrievers, which allows them to be seamlessly invoked. It is the fundamental building block because it provides a uniform protocol—specifically the 'invoke', 'stream', and 'batch' methods—that enables components to be composed into complex sequences. By standardizing this interface, LangChain ensures that you can swap out any part of your pipeline, such as replacing one model with another, without having to rewrite the surrounding logic, because every Runnable adheres to the same predictable execution contract.
How do you create a basic chain using the LangChain Expression Language (LCEL)?
To create a basic chain using LCEL, you use the pipe operator (|) to connect multiple Runnables. For example, if you have a prompt template and an LLM, you would define them and then chain them: 'chain = prompt | model'. When you call 'chain.invoke(input)', the prompt receives the input, formats it, and passes the resulting string or message sequence automatically to the LLM. This is powerful because it abstracts away the boilerplate of manual data passing and state management between components.
What is the purpose of the RunnablePassthrough component within a chain?
RunnablePassthrough is a utility component that allows you to pass data through to a subsequent step without modifying it, or to add extra data to the input mapping. It is essential when you have a chain that requires multiple inputs. For instance, if a prompt needs both a user's question and context from a retriever, you can use a dictionary with a RunnablePassthrough to 'passthrough' the user's input while simultaneously invoking the retriever to populate the context field. This keeps the data flow clean and explicit.
Compare using LCEL versus the legacy 'Chain' classes like LLMChain for building pipelines. Which is preferred and why?
While legacy classes like LLMChain were convenient for simple tasks, they often acted as 'black boxes' that were difficult to customize or inspect. LCEL is significantly preferred today because it is declarative, composable, and transparent. LCEL allows you to see the entire pipeline as a clear flow of components connected by pipes, making it trivial to add streaming, parallel execution, or logging at any stage. Furthermore, LCEL provides better native support for asynchronous programming and batching, which are critical for performance in production-grade applications that legacy classes struggled to handle cleanly.
How does the 'batch' method on a Runnable improve performance when processing multiple inputs?
The 'batch' method allows a chain to process a list of inputs in parallel rather than iterating through them sequentially. When you call 'chain.batch([input1, input2, ...])', LangChain optimizes the execution to send these inputs to the underlying components concurrently, provided the component supports parallelism. This is vital for reducing latency when handling high volumes of requests, as it avoids the 'wait time' associated with individual network calls. It effectively maximizes throughput by ensuring your LLM or other APIs are utilized as efficiently as possible within a single execution block.
Explain how to integrate a custom function into a chain using RunnableLambda, and why this is often necessary.
RunnableLambda is used to wrap a standard Python function into a Runnable, allowing you to insert custom logic directly into an LCEL pipeline. This is necessary because while most components like prompts and LLMs are built-in, you often need to perform data manipulation, such as formatting a dictionary, filtering a list, or parsing a complex JSON string between stages of your chain. By using 'RunnableLambda(my_function)', you ensure that your custom logic respects the standard invocation protocol, allowing it to work seamlessly with streaming, batching, and async operations alongside the rest of your chain components.
Check yourself
1. What is the primary function of the LCEL pipe operator (|) in a sequence of Runnables?
- A.It concatenates two string objects together
- B.It defines a new dependency graph for external APIs
- C.It passes the output of the left component as the input to the right component
- D.It serializes the entire chain into a single JSON object
Show answer
C. It passes the output of the left component as the input to the right component
The pipe operator is the core of LCEL, designed to create a pipeline where the result of one step automatically feeds into the next. It is not for string concatenation, external graph definitions, or simple JSON serialization.
2. When building a chain, why would you use a RunnablePassthrough?
- A.To terminate a chain prematurely if the input is empty
- B.To allow existing input data to be passed through to later steps in a chain alongside new computed data
- C.To convert a raw string input into a formatted PromptValue
- D.To force the LLM to skip the internal prompt template step
Show answer
B. To allow existing input data to be passed through to later steps in a chain alongside new computed data
RunnablePassthrough allows you to preserve the original input context while adding new keys to the chain, which is essential for prompts needing both user input and context. It does not terminate chains, perform string formatting, or bypass prompts.
3. If you have a chain of 'PromptTemplate | LLM | StrOutputParser', what is the data type returned by the entire chain?
- A.A dictionary containing metadata about the LLM token usage
- B.A PromptValue object ready for the next component
- C.A raw string representing the final generated content
- D.A list of Message objects
Show answer
C. A raw string representing the final generated content
The StrOutputParser specifically takes the output of the LLM and extracts the string content, so the final result of this sequence is a string. The other options are intermediate states or metadata not produced by the parser.
4. How do you add conditional logic or branching within an LCEL chain?
- A.By using the 'if' statement inside a RunnableLambda
- B.By using a RunnableBranch which maps inputs to different runnables based on predicates
- C.By utilizing the 'OR' operator between two chain components
- D.By defining a separate LangChain project for every possible outcome
Show answer
B. By using a RunnableBranch which maps inputs to different runnables based on predicates
RunnableBranch is the idiomatic way to handle branching logic based on the input data. Using raw 'if' statements outside a lambda is not chain-compatible, the OR operator does not exist for this purpose, and creating multiple projects is inefficient.
5. Why is it beneficial to bind parameters like 'temperature' to an LLM using the .bind() method within a chain?
- A.It ensures that the temperature is updated dynamically at runtime based on the user's history
- B.It allows you to specify constant parameters for that specific step without altering the global LLM configuration
- C.It forces the LLM to restart every time the chain is invoked
- D.It converts the LLM into an asynchronous function
Show answer
B. It allows you to specify constant parameters for that specific step without altering the global LLM configuration
The .bind() method provides a clean way to pass constant arguments to an LLM component during the chain construction. It does not control dynamic updates at runtime, trigger restarts, or change the synchronous/asynchronous nature of the call.