RAG as an agent: retrieval as a worker node in a multi-agent system

Here's a design question that separates people who've drawn the diagram from people who've built it: you have a multi-agent system, and you need RAG somewhere in it. Where does it go? The instinct is to bolt retrieval onto the orchestrator. The better answer is to make RAG its own worker.

Sequential RAG vs an agentic flow

A plain RAG pipeline is a straight line — retrieve, rerank, generate. An agentic system is a graph with branching and feedback (the workflows-vs-agents post draws the distinction). Take a travel assistant: the orchestrator decomposes the user's intent and delegates to specialised workers — a flight agent, a hotel agent, a knowledge agent for everything unstructured — then synthesises their results. RAG belongs inside one of those workers, not smeared across the orchestrator's routing logic.

Why a dedicated knowledge agent

• Decoupling — the retrieval mechanics (chunking, hybrid search, reranking, compression) stay inside the knowledge agent, so the orchestrator only routes and never needs to know a vector index exists.
• Mixed data — the API workers hit structured, live APIs (flights, hotels); the knowledge agent handles unstructured documents. One system, two kinds of source, cleanly separated by worker.
• Parallelism — the knowledge agent's retrieval runs concurrently with the API calls, so RAG latency overlaps the flight and hotel lookups instead of adding to them (the parallelization pattern, in service of the p95 budget from the distributed-systems post).

The orchestrator owns state; the workers don't

The orchestrator holds the shared state object as the single source of truth; each worker, the knowledge agent included, receives only the slice it needs and returns structured output that a reducer merges back without races (the LangGraph-internals post covers state and reducers). One nuance worth stating: the knowledge agent is the only worker that really needs a memory layer — flight and hotel data is fetched live and deliberately not persisted, so retrieval is the one place state accumulates.

The boundary that keeps it clean

Putting RAG behind a worker node is the least-agency principle applied to architecture: give each part exactly one job. The orchestrator routes, the API workers act, the knowledge agent retrieves. And because retrieval is encapsulated, you can upgrade the RAG inside it — add reranking, contextual compression, corrective checks — without touching the orchestration at all. That's the whole payoff of the boundary.

Don't teach your orchestrator to do RAG. Give it a knowledge agent to delegate to — then retrieval improves behind a clean interface while the orchestrator keeps doing the one thing it's good at: deciding who acts next.