14 — Agent Frameworks: LangChain, LlamaIndex, and the Build-It-Yourself Path

"Just use LangChain" was my reflex for any agent or RAG project, and it worked until I couldn't tell what the framework was actually doing for me. The frame that fixed that: frameworks are an opinionated assembly of the same primitives — chunking, embeddings, retrieval, generation, the agent loop — and the alternative, wiring SDKs directly, is more reachable than it sounds. [1][2] The framework isn't doing something I couldn't do myself; it's doing many small things I'd rather not rewrite, with opinions baked in about how they fit together. Knowing that changes the decision from "which framework" to "how much of the assembly do I want to hand off."

The framing that finally landed is a build-versus-assemble spectrum. At one end, a framework like LangChain or LlamaIndex gives me a complete, opinionated pipeline out of the box. At the other end, I wire the individual SDKs together myself — a text splitter here, an embedding API there, a vector database SDK, a generator call. The middle ground is a lighter framework or a manual implementation of just the pieces I need. Where I land depends on how standard my problem is and how much control I want.

LangChain: the general-purpose orchestrator

LangChain is the broadest of the frameworks — a development toolkit for building applications powered by language models, focused on creating _chains_, sequences of operations where the model interacts with databases, APIs, and other models to perform complex tasks [1]. A chain might be "retrieve documents, stuff them into a prompt, call the model, parse the structured output," and LangChain provides the abstractions to compose those steps. It also extends naturally into agents: the same composition primitives become the agent loop when the model is allowed to decide which step runs next.

The value is the breadth — there's an integration for almost everything — and the cost is the abstraction weight. LangChain's chains hide a lot of machinery, which speeds up the happy path and slows down debugging when something goes wrong inside a chain. For standard RAG and agent patterns it's excellent; for unusual requirements the abstractions can fight back.

LlamaIndex: data-first RAG

LlamaIndex is the data-framework counterpart, built specifically to connect LLMs to structured and unstructured data sources [2]. Where LangChain started from the orchestration side, LlamaIndex started from the _indexing_ side — its strength is ingesting documents, databases, and APIs, indexing them for retrieval, and exposing them to an LLM through clean query interfaces. For a feature whose center of gravity is "I have a lot of data and I need an LLM to query it," LlamaIndex's data connectors and index abstractions are often a better fit than LangChain's generic chains.

In practice the two overlap heavily and both can do RAG and agents. The choice often comes down to which abstraction feels closer to the problem: chains and tools (LangChain) or documents and indices (LlamaIndex).

Haystack, RAGFlow, and the focused alternatives

Not every problem needs a general framework. Haystack is an open-source Python framework focused on search and question-answering pipelines — connect data sources, pick a model, set up pipelines that retrieve passages, run the model, and rank results, with backends like Elasticsearch, OpenSearch, FAISS, and Pinecone [3]. RAGFlow is a newer framework dedicated specifically to streamlining RAG pipeline creation, evaluation, and deployment, with modular components for data loaders, retrievers, and generators [4]. These focused tools are often a better fit than a general framework when the problem is squarely within their scope, because they encode fewer opinions about things outside it.

Using SDKs directly: the build-it-yourself path

Here's the part I underappreciated: I don't have to use a framework at all. The roadmap makes a point of this, and it's worth taking seriously [5]. If I understand the RAG stages — chunk, embed, retrieve, generate — I can assemble them myself from the underlying SDKs:

• Chunk with a text-splitter package (e.g. @langchain/textsplitters) — usable independently of the full LangChain framework.
• Embed through any embedding API via its SDK (OpenAI, Cohere, a local model).
• Store and retrieve via the vector database's own SDK (Supabase, Pinecone, Qdrant).
• Generate via the model API.

The payoff of building it yourself is transparency — every step is code I wrote and can debug — and the freedom to pick the best tool for each stage instead of what the framework bundles. The cost is that I'm responsible for the wiring, the error handling, and the retries that a framework would have handled. For a simple RAG pipeline this is genuinely tractable; for a complex multi-agent system it's a lot of glue code.

Agent SDKs: the model vendors' offerings

A separate category is the agent SDKs from the model vendors themselves. OpenAI's AgentKit / Agents SDK provides tools and abstractions for managing agent state, behavior, and tool connections [6]. Anthropic's Claude Agent SDK offers pre-built components for planning, tool usage, memory management, and human interaction built around Claude [7]. Google's Agent Development Kit (ADK) streamlines agent development with orchestration, tool integration, and evaluation [8]. And Vertex AI is Google Cloud's broader platform for building, training, and deploying ML models — including agent-building tools on managed infrastructure [9].

These vendor SDKs differ from the framework-agnostic ones (LangChain, LlamaIndex) in that they're shaped around a specific model family's strengths. The tradeoff is tighter integration with that vendor's features (prompt caching, extended thinking, specific tool formats) against portability across vendors. I reach for them when I've committed to a vendor and want the best-supported path; I reach for the agnostic frameworks when portability matters more.

Manual implementation: maximum control

The far end of the spectrum is manual implementation — building the agent logic from scratch, without relying on a framework's abstractions [10]. This is the most work but gives maximum control and customization. OpenAI's "practical guide to building agents" and Anthropic's docs on custom subagents both acknowledge this path exists and is legitimate for sophisticated use cases. The honest assessment: manual implementation earns its place when the framework's abstractions are actively getting in the way — when I need a control flow or a tool-calling pattern the framework doesn't support cleanly. For most features, it's more work than justified.

How I use this

My rule is to match the framework choice to the problem's shape and my need for control. For a standard RAG feature, I'll use LlamaIndex or LangChain to move fast, because the patterns are well-trodden and the abstractions fit. For a feature with unusual retrieval or agent requirements, I'll wire the SDKs directly so I can debug every stage and pick the best tool per step — the transparency is worth the glue code. For vendor-specific features where I've committed to a provider, I'll consider that vendor's agent SDK for the integration quality. And I treat the choice as reversible: starting with a framework to learn the shape of the problem, then dropping to direct SDKs once I understand which abstractions were helping and which were hurting, is a perfectly reasonable path. The framework is a productivity tool, not a commitment — and knowing the primitives underneath is what lets me move between them.

References

[1] LangChain, 2024. [Online]. Available: https://www.langchain.com/

[2] LlamaIndex, "Documentation," 2024. [Online]. Available: https://docs.llamaindex.ai/en/stable/

[3] deepset, "Haystack," 2024. [Online]. Available: https://haystack.deepset.ai/

[4] RAGFlow, 2024. [Online]. Available: https://ragflow.io/

[5] npm, "@langchain/textsplitters," 2024. [Online]. Available: https://www.npmjs.com/package/@langchain/textsplitters

[6] OpenAI, "Introducing AgentKit," 2024. [Online]. Available: https://openai.com/index/introducing-agentkit/

[7] Anthropic, "Agent SDK overview," 2024. [Online]. Available: https://platform.claude.com/docs/en/agent-sdk/overview

[8] Google, "Agent Development Kit," 2025. [Online]. Available: https://google.github.io/adk-docs/

[9] Google Cloud, "Vertex AI," 2024. [Online]. Available: https://cloud.google.com/generative-ai-studio

[10] OpenAI, "A practical guide to building agents," 2024. [Online]. Available: https://cdn.openai.com/business-guides-and-resources/a-practical-guide-to-building-agents.pdf