Optimizing Context Graph Agents with Deterministic Queries and Fast Decision Models

Session track: Modern Applications

Session time: 10:30 pm - 11:00 pm UTC November 12

Session description:

Context graphs give AI agents access to an organization's shared memory. Up to date facts, decisions, agent trajectories, and the relationships between them that provide context that no frontier model has. To access the various facets of the knowledge in a context graph, we typically provide an agent with many tools which preform specialized queries to bring the right context to an LLM. We often allow the agent to orchestrate itself: to choose its own tools in any order. We might even allow an agent to write its own queries as it attempts to reason through a tough question. While this approach can yield success, especially if guided by prompts or skills, it is often not the most efficient in terms of time or tokens, and tool order is not deterministic. At worst, the same question might be answered completely differently every time. A new class of decision models, such as Jev from TypeSafe AI, can give fast, clear answers if provided with the right context. Once your agent discovers a recipe for obtaining the right context for a given decision type, consider making it more deterministic by running that recipe deterministically and asking for a simple yes or no from a decision model. In this talk we'll touch on context graphs, decision models, and demonstrate a tool-use agent versus a decision-model agent and how the approaches can be combined.