Tracing Agent Decisions with Graph Evals and Neo4j

Session Track: Graph Memory & Agents

Session Time: 08:00 am - 08:30 am April 15

Session description

AI agents don’t just need to perform, they need to be understood, trusted, and improved.
Traditional evals only look at inputs and outputs, ignoring the messy middle where most agent failures happen.

This session introduces Graph Evals, a practical technique where every agent step (actions, states, tool calls, reasoning hops, failure points) is stored as a knowledge graph. By modeling an agent’s internal decision journey in Neo4j, we can analyze its reasoning patterns, detect blindspots, identify loops, and understand why it behaved the way it did.

Attendees will learn how to build a graph-based eval pipeline, visualize agent reasoning paths, run structural queries to catch failure modes, and continuously refine agent policies using graph insights.

Perfect for teams deploying production-grade agents and anyone who wants their AI to act less like a black box.