Beyond Retrieval: Adding a Reasoning Layer to RAG with Neo4j

Session Track: App Dev

Session Time: 03:30 am - 04:00 am November 6

Session description

The reasoning layer in my RAG system is powered by Neo4j’s knowledge graph, which acts as a structured context source beyond flat document retrieval. Here’s how it works: 1. Entity Linking—When a query comes in, we extract key entities and link them to nodes in the graph. 2. Multi-Hop Traversal—We use Cypher queries to traverse related entities, uncover hidden relationships, and fetch context that traditional vector search misses. 3. Query Enrichment—The insights from the graph are then used to expand or reframe the query before hitting the LLM, enabling more relevant and accurate responses. 4. Reasoned Prompt Injection—The LLM receives not just retrieved docs but structured knowledge paths (e.g., "Company A is Invested in Company B and Partnered with Startup C"), which it can reason over. This hybrid system lets the LLM “think” in terms of relationships, not just keywords, adding explainability and depth to its answers.