GraphRAG: retrieval over a knowledge graph, not just a vector index
Vector RAG finds similar chunks. GraphRAG builds a knowledge graph of entities and relations from your documents and traverses it — which is what you need for multi-hop questions and provenance.
Vector RAG answers 'find me the passage that's similar to this question'. Some questions aren't about similarity at all — they're about relationships: how facts across different documents connect. That's what GraphRAG is for, and in relationship-heavy domains it answers questions plain vector search structurally can't.
From documents to a graph
GraphRAG runs an extraction pass first: it pulls entities (people, organisations, technical terms) and the relations between them out of the documents, and assembles them into a knowledge graph — nodes are entities, edges are relationships. Optionally, community detection groups densely-connected nodes and summarises each community, giving a hierarchy from fine detail up to a corpus-wide overview.
Querying by traversal
Instead of only embedding the query and finding similar chunks, you traverse the graph: start at the entities the question mentions and follow relationships across documents. That's what makes multi-hop questions answerable. 'Which inventors worked on patents cited by X?' isn't a similarity query — it's a walk across edges, and no single chunk contains the answer.
When graph beats vector
- Multi-hop questions — the answer connects several facts across several documents.
- Provenance and 'why is this relevant' — the path through the graph is itself the explanation, which matters when you must justify a result.
- Global questions — 'what are the main themes across the whole corpus?' is answered by community summaries, not by any one retrieved passage.
The cost, and the hybrid
GraphRAG front-loads work: the extraction and graph-building pass is an upfront LLM cost, and the graph needs maintenance as documents change. It earns that cost where relationships and provenance are the point — legal, research, and investigative work — and it's overkill for a simple 'find the passage that says X'. Often the best system is hybrid: vector retrieval for passages, the graph for relationships, with an agentic layer choosing per query (see the RAG-improvement post on agentic RAG).
Vector search finds what looks like the answer. A graph finds how the answer connects — and for the questions that are really about relationships, that's the difference between a guess and a justification.