When RAG Hallucinates Numbers: Graph-RAG for Precise Answers

Session track: AI Engineering

Session time: 09:30 pm - 10:00 pm UTC November 12

Session description:

Your RAG agent seems smart until you ask it to count. "How many items match X?" It fabricates "about 45-50" when the real answer is 133. Vector similarity cannot count, aggregate, or reason across relationships. The root cause is architectural. RAG retrieves text chunks by similarity, then asks the LLM to synthesize an answer. That works for lookups but fails on four query types: counting, aggregation, multi-hop reasoning, and out-of-domain detection. Graph-RAG fixes this by building a knowledge graph automatically (no manual schema design) and using the Text2Cypher pattern to turn natural language into precise queries the LLM cannot fabricate. You will see two agents answer identical queries side by side: RAG invents results, Graph-RAG answers correctly every time. You'll walk away with: • Build Graph-RAG over your own documents • Decide when to use RAG versus Graph-RAG • Combine both in a hybrid retrieval system All code is open source.