Graph RAG Knowledge Graph Retrieval: Two Questions Vector Search Cannot Answer

Your chunk-and-embed pipeline answers "what is our refund window" perfectly. Then somebody asks "which of our suppliers are exposed to the same port delay as Acme," and the retriever hands back 5 chunks that each name one supplier and never connect them to each other. Graph RAG knowledge graph retrieval attacks that gap by building an entity-and-relationship graph out of your documents at index time and walking it at query time, instead of ranking isolated chunks by embedding distance.

What is graph rag knowledge graph retrieval, and how is it different from vector search?#

Graph retrieval builds a knowledge graph from your documents before any question arrives. A language model reads every chunk, pulls out entities and the relationships between them, and stores that structure alongside the text. At query time the retriever walks edges between related entities instead of ranking isolated chunks by embedding distance.

Microsoft's documentation describes the index in four moves: slice the corpus into text units, extract all entities, relationships, and key claims from those units, run hierarchical clustering over the resulting graph using the Leiden technique, then generate summaries of each community and its constituents from the bottom up. Only the first move is shared with a normal vector pipeline. If you want the baseline this is replacing, our guide to RAG chunking strategies covers the slicing step in detail.

Stages 2 and 4 are where the language model runs, and where the indexing bill comes from.

The query side splits into modes rather than one retriever. Local search reasons about specific entities by fanning out to their neighbors. Global search reasons about the whole corpus by mapping over the pregenerated community summaries. DRIFT search layers community context onto a local query, and basic search is a plain vector implementation the library keeps around so you can compare the two approaches on identical data.

Which questions can vector search not answer at all?#

Two shapes. Multi-hop questions need a fact from one document joined to a fact from another, where no single chunk holds both halves. Corpus-wide questions ask what the whole collection says, and a top-10 chunk list cannot summarize 5,000 documents it never retrieved in the first place.

The multi-hop failure is quiet and specific. Ask "which suppliers share a warehouse with the one that missed last quarter," and cosine similarity will happily return the chunk about the missed quarter and the chunk listing warehouse assignments. Neither chunk contains the answer. The answer lives in the join, and a similarity ranker has no mechanism for performing a join. Adding more chunks to the context window does not fix this, it just makes the model do the join badly.

The corpus-wide failure is louder. "What are the recurring complaint themes across our support archive" is a question about all 5,000 tickets, and retrieval returns 10. The original GraphRAG paper from Microsoft Research was written specifically around this case, which the authors call global sensemaking, and it reports substantial improvements over a conventional retrieval baseline on comprehensiveness and diversity for datasets in the 1 million token range.

Before you conclude you have one of these, rule out the cheaper explanation. A lot of what looks like a reasoning failure is a retrieval-quality failure, and our write-up on contextual retrieval covers the fix that costs a fraction of a graph index.

What does the benchmark data actually say?#

A benchmark called GraphRAG-Bench scored 9 retrieval methods across 4 task types on the same corpus. Graph methods beat reranked vector search by 10.5 points on complex reasoning and 13.1 points on contextual summarization. On plain fact retrieval, Microsoft GraphRAG scored 11.6 points below vector search.

Those are accuracy percentages from the paper's novel-corpus table, scored with GPT-4o-mini. Read the first row twice. On the task most production retrievers actually spend their day doing, the heaviest graph system in the test lost to a reranked vector index by more than 11 points, because the extra structure buys nothing when the answer sits in one paragraph and the retriever now has to route around a graph to find it.

One row deserves more attention than the rest. HippoRAG2 landed at 60.14 on fact retrieval against 60.92 for reranked vector search, close enough to call a tie, while winning every other task type in the test. That is the shape you want from a retrieval upgrade: no regression on the common case, real gains on the hard one. The heavier system did not manage it.

Contextual summarization is the task where graph retrieval pulls furthest ahead of vector search.

"Graph" is also not one thing. LightRAG, the lightest system in that table, scored 23.80 on creative generation against 38.26 for reranked vector search, so a cheaper graph is not automatically a safer default. Treat the whole table as directional rather than settled: it is one benchmark, on one corpus family, scored by one judge model, and it has not been widely replicated.

What does building the graph actually cost?#

Vector RAG reads your corpus once with a cheap embedding model that produces no output tokens. Graph retrieval runs a full language-model extraction pass over every chunk, then more calls to summarize every detected community. Microsoft's own README warns that indexing is expensive and tells you to start small.

Here is reproducible math with the assumptions on the table. Take a corpus of 1 million tokens. The extraction pass reads all 1 million and, if entities and relationships come back at roughly 30% of input volume, writes about 300,000 output tokens. Community summarization then re-reads that extracted graph and writes summaries at several levels; assume 600,000 tokens read and 300,000 written. At Claude Haiku 4.5's published rate of $1 per million input tokens and $5 per million output, that is $1.60 in and $3.00 out, so roughly $5 per million tokens of corpus. Swap in Sonnet 4.6 at $3 and $15 and the same pass lands near $14. Both figures are modeled, not measured, and the 30% output ratio is the assumption doing the most work.

Then double it, because you will run that pass more than once. Microsoft's documentation is explicit that using GraphRAG on your data out of the box may not yield the best possible results, and recommends a prompt-tuning step first, so budget for at least 2 index runs before the output is worth querying.

Every figure here is read off the source paper or the vendor's own published post.

Query time carries its own tax. GraphRAG-Bench measured a single Microsoft GraphRAG local query at 38,707 prompt tokens against 879 for vanilla vector RAG on the same corpus, roughly 44 times the context for every question asked. Microsoft evidently agrees the full pipeline is too heavy to be a default: its own LazyGraphRAG write-up reports indexing costs identical to vector RAG and 0.1% of full GraphRAG, with global queries more than 700 times cheaper at comparable quality.

Then there is freshness, which is where graph projects quietly die. The LightRAG paper measured the update path and found that adding a comparable new dataset costs GraphRAG roughly 1,399 community reports rebuilt at about 2 passes of 5,000 tokens each, close to 14 million tokens of model work for one refresh. A corpus that changes weekly turns that into a recurring line item, not a setup cost.

Which graph retrieval tool should you start with?#

Three are worth your time. Microsoft GraphRAG is the reference implementation and the most expensive to index. LightRAG is the lighter research-backed alternative with a single-call retriever. The Neo4j GraphRAG package for Python is the first-party route if a graph database already sits in your stack.

Microsoft GraphRAG is a Python package and command-line pipeline, at v3.1.1 as of July 2026 with roughly 35,100 GitHub stars. It is the most complete and the most opinionated: you get community summaries, 4 query modes, and a prompt-tuning step the docs recommend before you trust the output, which in practice means indexing more than once.

The split is mostly about how much work each one does before a single question is asked.

LightRAG came out of the HKU Data Science Lab, was published at EMNLP 2025, and sits near 34,000 stars. Its pitch is a dual-level retriever that answers with a single API call and under 100 tokens of keyword generation, plus incremental updates that merge new documents into the existing graph rather than rebuilding reports. The Neo4j package is the boring institutional choice: first-party, long-term supported, split into a graph-construction workflow and a querying workflow, and it is the renamed continuation of the deprecated neo4j-genai package.

One status note worth having before you plan around it. LazyGraphRAG has excellent published numbers, but Microsoft Research announced it in November 2024 and it still does not appear as a selectable query method in the open-source library's own query documentation. Treat it as a published result, not as something you can install this afternoon.

So should you build a graph, or stay on vectors?#

Stay on vectors unless you can name a real question your users ask that needs 2 documents joined, or one that needs the whole corpus summarized. If you cannot name one, graph retrieval is an indexing bill with no matching benefit. If you can name one, start with the lighter option.

Three checks decide it: whether you can name a multi-hop question, whether hybrid search already fell short, and whether the corpus holds still.

The framing that survives contact with production is graph retrieval alongside vectors, not instead of them. Route single-fact lookups to the vector index that already handles them at 879 tokens a query, and reserve the graph path for the joins and the summaries. Microsoft's library keeps a basic vector search mode in the box for exactly this reason, and it is the honest comparison to run before you migrate anything.

Whichever way you go, measure it rather than trusting the vendor chart. Score both retrievers on the same question set with the same rubric, and score them per question type instead of averaging, because a single averaged number will hide the fact-retrieval regression the benchmark above found. Our guide to RAG evaluation metrics covers the measurements that make that comparison mean something.

What should you do this week?#

Pull 20 real questions out of your logs and tag each one by shape: single fact, join across documents, or whole-corpus summary. That tally decides this for you. If single-fact lookups dominate, spend the week on reranking and metadata filters instead, and keep the graph idea on the shelf.

If the joins and the summaries are real, index a 5% sample with the lightest tool that fits your stack and score it against your current retriever on the same questions. A sample index costs a few dollars and an afternoon. A full index on a corpus you have not validated the approach against costs considerably more, and the benchmark says you might be buying a regression on the questions you ask most.

For the vector-side foundation this post keeps referring back to, start with chunking strategies, and if you are still deciding whether retrieval is the right architecture at all, long context versus RAG covers the case where the whole document fits in the window and none of this matters.