GraphRAG and Knowledge Graphs for Agents
Source: mortalapps.com- GraphRAG is a retrieval-augmented generation pattern that uses structured knowledge graphs to capture complex relationships and global document context.
- It solves the 'lost in the middle' and relational blindness issues inherent in standard vector-only similarity searches.
- Production systems use GraphRAG to enable multi-hop reasoning and high-level summarization across massive, interconnected datasets.
- Implementing this pattern results in significantly higher factual accuracy for queries requiring synthesis of disparate data points.
- The primary tradeoff is the high computational cost and token latency associated with the initial graph construction phase.
Why This Matters
Standard Retrieval-Augmented Generation (RAG) relies on vector similarity, which excels at finding local context but fails when an agent needs to understand the 'big picture' or follow a chain of relationships across multiple documents. This graphrag tutorial python guide addresses the fundamental limitation of vector databases: they treat data points as isolated coordinates in space rather than nodes in a relational network. In production engineering, ignoring the relational structure of enterprise data leads to hallucinations when agents are asked to compare entities, summarize themes across a corpus, or perform multi-hop reasoning. For example, a vector search might find a document about 'Project X' and another about 'Engineer Y,' but it may fail to connect them if the relationship is only implied through a third 'Contract' document. GraphRAG was developed to bridge this gap by using LLMs to pre-process text into a Knowledge Graph (KG). This approach is essential for complex use cases like fraud detection, legal discovery, and technical support where the answer is rarely contained in a single chunk. While vector RAG is faster and cheaper for simple Q&A, GraphRAG is the necessary alternative for agents that must act as domain experts, providing a structured memory that mirrors human relational understanding. Without this, enterprise agents remain limited to 'search-and-summarize' tasks rather than true cognitive reasoning.
Core Concepts
To implement GraphRAG effectively, engineers must master several structural concepts that differ from standard vector workflows:
- Entities and Relationships: The fundamental building blocks. Entities are nodes (e.g., People, Organizations, Concepts) and Relationships are directed edges (e.g., 'WORKS_FOR', 'LOCATED_IN').
- Triplets: The atomic unit of a knowledge graph, expressed as (Subject, Predicate, Object). For example, (Agent, USES, Tool).
- Community Detection: Algorithms like Leiden or Louvain that group related nodes into clusters. This allows the agent to summarize high-level 'communities' of information rather than individual nodes.
- Global vs. Local Search: Local search traverses the graph starting from a specific entity to find related facts. Global search aggregates summaries from pre-computed communities to answer broad questions like 'What are the main risks in this portfolio?'.
- Graph Embeddings: Numerical representations of graph structure that allow for hybrid retrieval, combining vector similarity with topological connectivity.
How It Works
The GraphRAG lifecycle consists of two distinct phases: Indexing (Graph Construction) and Querying (Retrieval).
Phase 1: Knowledge Graph Construction
- Source Chunking: Input documents are split into overlapping text chunks. Unlike vector RAG, these chunks are processed sequentially to maintain narrative flow.
- Entity and Relation Extraction: An LLM parses each chunk to identify entities and their relationships. This step uses a strict schema or a zero-shot prompt to generate triplets.
- Entity Resolution (Deduplication): The system identifies when different strings refer to the same entity (e.g., 'MSFT' and 'Microsoft') and merges them into a single node.
- Community Generation: Using hierarchical clustering algorithms, the graph is partitioned into communities. The LLM then generates a summary for each community, creating a multi-level index of the entire dataset.
Phase 2: The Retrieval Loop
- Query Analysis: The agent determines if the query is 'Local' (specific entity focus) or 'Global' (thematic focus).
- Traversing the Graph: For local queries, the agent identifies 'seed' nodes using vector search and then traverses edges to find neighboring context.
- Map-Reduce Aggregation: For global queries, the agent retrieves summaries from relevant communities (Map) and synthesizes them into a final answer (Reduce).
- Failure Handling: If no entities are found, the system falls back to standard vector retrieval. If the graph is too dense, the agent applies 'pruning' to remove low-weight edges that introduce noise.
Architecture
The GraphRAG architecture is a multi-stage pipeline that sits between the raw data and the LLM. It begins with a Data Ingestion layer that feeds into an LLM-based Extraction Engine. This engine outputs structured triplets into a Graph Database (typically Neo4j or FalkorDB). Simultaneously, a Community Detection service runs asynchronously to cluster nodes and trigger the LLM to write 'Community Summaries' back into the database. When a user query arrives, the Orchestrator (e.g., LangGraph or LlamaIndex) routes the request through a Hybrid Retriever. This retriever queries both a Vector Index (for semantic similarity) and the Graph Database (for relational context). The results are merged in a Context Assembler, which formats the graph traversals and community summaries into a prompt for the final LLM generation. The execution starts at the Orchestrator and ends with a grounded response that cites specific nodes and relationships as evidence.
Entity Extraction and Schema Alignment
The quality of GraphRAG is entirely dependent on the extraction phase. In production, using a generic 'extract entities' prompt leads to inconsistent schemas. Engineers should implement a Pydantic-based extraction layer where the LLM is forced to output JSON matching a predefined ontology. This ensures that 'Company' nodes always have a 'ticker' property and 'Person' nodes always have a 'role'. For unstructured domains, a 'Schema-First' approach is recommended where the LLM first proposes a schema for a sample of the data, which is then reviewed and locked before full-scale indexing.
Multi-Hop Traversal Logic
Multi-hop reasoning allows an agent to answer questions like 'Which vendors of our primary supplier have been affected by the recent port strike?'. This requires a 2-hop traversal: (Primary Supplier)-[SUPPLIED_BY]->(Vendor)-[AFFECTED_BY]->(Strike). In Cypher (Neo4j's query language), this is expressed as a path search. To prevent state explosion, traversals must be limited by 'hop count' (usually 2-3) and edge weight. Edge weights are calculated during extraction based on the frequency of the relationship appearing in the source text.
Community Summarization for Global Context
One of the most powerful features of GraphRAG is its ability to handle 'Global' queries that vector RAG cannot. By applying the Leiden algorithm, the graph is divided into hierarchical clusters. At the lowest level, clusters might represent specific product features. At the highest level, they represent entire business units. The system generates a summary for every cluster. When a user asks a broad question, the agent retrieves the top-level summaries, providing a comprehensive overview that doesn't miss 'orphaned' chunks of text that a vector search might ignore due to low similarity scores.
Hybrid Retrieval Strategies
Combining vector search with graph traversal provides the best of both worlds. A common pattern is to use vector search to find the 'entry point' entities in the graph. Once the entry points are identified, the agent performs a 'Graph Walk' to collect surrounding context. This context is then ranked using a Cross-Encoder to ensure only the most relevant relational facts are included in the final prompt. This prevents the 'context stuffing' problem where too many irrelevant graph edges dilute the LLM's focus.
Code Example
import os
from typing import List
from pydantic import BaseModel, Field
from openai import OpenAI
# Define the schema for graph extraction
class Relationship(BaseModel):
subject: str = Field(..., description='The source entity')
predicate: str = Field(..., description='The relationship verb (e.g., WORKS_AT)')
object: str = Field(..., description='The target entity')
class GraphExtraction(BaseModel):
relationships: List[Relationship]
api_key = os.environ.get("OPENAI_API_KEY")
if not api_key:
raise ValueError("OPENAI_API_KEY not set")
client = OpenAI(api_key=os.environ.get('OPENAI_API_KEY'))
def extract_graph_data(text: str):
# Use structured outputs to ensure valid triplets
completion = client.beta.chat.completions.parse(
model='gpt-4o-2024-08-06',
messages=[
{'role': 'system', 'content': 'Extract entities and relationships as triplets.'},
{'role': 'user', 'content': text}
],
response_format=GraphExtraction,
)
return completion.choices[0].message.parsed
text_sample = 'TechCorp acquired DataViz in 2023 to expand its AI capabilities.'
graph_data = extract_graph_data(text_sample)
print(graph_data.model_dump_json(indent=2)){
"relationships": [
{
"subject": "TechCorp",
"predicate": "ACQUIRED",
"object": "DataViz"
},
{
"subject": "TechCorp",
"predicate": "HAS_GOAL",
"object": "expand AI capabilities"
}
]
}
from neo4j import GraphDatabase
import os
class KnowledgeGraphManager:
def __init__(self):
self.driver = GraphDatabase.driver(
os.environ.get('NEO4J_URI'),
auth=(os.environ.get('NEO4J_USER'), os.environ.get('NEO4J_PASSWORD'))
)
def close(self):
self.driver.close()
def get_multi_hop_context(self, entity_name: str, hops: int = 2):
# Cypher query to find related nodes up to N hops away
query = (
"MATCH (e {name: $name})-[r*1.." + str(hops) + "]-(related) "
"RETURN e.name as start, type(last(r)) as rel, related.name as end"
)
with self.driver.session() as session:
results = session.run(query, name=entity_name)
return [f"{record['start']} {record['rel']} {record['end']}" for record in results]
# Usage in an agent loop
kg = KnowledgeGraphManager()
try:
context = kg.get_multi_hop_context('TechCorp')
print('
'.join(context))
finally:
kg.close()TechCorp ACQUIRED DataViz DataViz DEVELOPED VisionPro TechCorp HAS_GOAL expand AI capabilities