If you want to build anything worthwhile with LLMs, you can't just use them out of the box. They can "hallucinate" on unfamiliar topics and struggle with proprietary data on their own.
Yes, I know what you're thinking—RAG or fine-tuning can solve this. But both methods have their downsides—they often miss the real-world context and nuance that makes information feel complete and connected to what you already know. Plus, they usually leave you guessing about why you got that particular answer.
So what can we do to improve traditional RAG?
Enter GraphRAG — a new approach that uses knowledge graphs in addition to vector search to enhance answers at query time.
Microsoft LinkedIn have published compelling research showing that GraphRAG consistently delivers more accurate and richer answers across a broader range of questions compared to basic RAG.
This article is meant to demystify some of the questions around GraphRAG, and provide more information on the current architectures.
We especially appreciate the input from the founders who are building the infrastructure for GraphRAG: Vasilie Markovic, Guy Korland, and Kirk Marple — Thank you!
Now, let’s start at the begining — at the representation layer.
To enable the "R" in RAG we use vector search to find and pull semantically similar information from a vector database that matches the given query. Using vector search to add context to LLMs has proven to be very useful — it works with unstructured data, it’s fast, scalable and cheap.
It truly feels like the perfect solution—until you need to understand how results are retrieved and their vector connections between different parts of data for complex tasks.
Knowledge graphs in comparison, represent semantic information between two entities — more declarative representations. With this information, both computers and humans are able to understand how different entities (nodes) relate to each other (edges), along with their shared attributes.
They can address the challenges that vector retrieval faces, and modeling them is not as complex as it was in the past, thanks to LLMs.
This basic comparison of human, vector and knowledge view by Philip Rathle explains the representations perfectly. (source)
Vectors and knowledge graphs are two different ways to represent information, and they complement each other well. Because of this, there's a growing interest in combining them to enhance current RAG architectures.
GraphRAG is a new retrieval approach that uses knowledge graphs in addition to vector search in a basic RAG architecture.
"While Vector RAG focuses on narrow, context-specific retrieval, GraphRAG broadens the horizon, integrating diverse knowledge through graph structures. In essence, Vector RAG is AI myopia, limiting vision, whereas GraphRAG sees the bigger picture, enabling more holistic and insightful AI solutions."
Guy Korland, Co-founder at Falkordb
Instead of just providing text chunks to the LLM, GraphRAG also supplies structured entity information, including descriptions, properties, and relationships. This helps the LLM understand the data more deeply and accurately. Here’s one way of how to implement GraphRAG: