Graph technology is made for managing multi-agent AI environments

As more businesses incorporate AI agents into their operations, visibility and auditability around their usage is critical to building acceptance and trust.

It’s increasingly apparent that the next wave of artificial intelligence adoption will be driven by multi-agent systems.

While generative AI and single AI agents are producing value for enterprise and government, the reality is that organisations have access to a multitude of different data sources. It does not make sense to have one agent trying to interact with all applications and data repositories. Rather, a number of domain-specific agents should ideally work together to autonomously execute complex tasks.

This is where the challenges begin.

While multi-agent systems are technically feasible and being implemented today, broad adoption will require effective governance to ensure that the systems are being used ethically and in line with a set of defined corporate, government, and regulatory standards.

It is already hard to build a trustworthy, explainable AI system where only one AI agent is involved. It’s doable – there are existing frameworks that could be leveraged – but the fact is that it’s not entirely trivial.

The challenge is exponentially more difficult with multi-agent systems.

That leads multi-agent AI systems to often operate as mysterious “black boxes”. With a large number of agents communicating with one another, there is so much more to think about – more moving parts, more data sources, and potentially less transparency around how each agent takes into account all information in front of it to reach a specific decision or conclusion. The workflows and reasoning employed by the agents – including how a complex question is broken down into parts, and which agent is called upon to handle each part of the question – will likely be masked from view.

Even for an in-house built, multi-agent system with dozens of agents, each capable of consulting dozens of data sources or tools, without a strong framework for audit and traceability in place, it is almost impossible to clearly visualise how an answer was reached and whether the output can be fully trusted.

The three internal cohorts seeking assurance

Three distinct sets of stakeholders exist within organisations that will require their own separate view of governance: developers, the security/legal team, and then the users/stakeholders. Each cohort has differing needs, and these need to be catered to.

For developers, their requirements boil down to traceability, logging, and monitoring of multi-agent applications. The goal is to help them troubleshoot multi-agent systems: debug what happens, improve accuracy and responses, understand what happens when a user asks the multi-agent system a question, and how it retrieves data in response.

Security and legal personnel will have more interest in what data has specifically been accessed, the transparency of the end-to-end data processing workflow, and in any logs the multi-agent system produces. This is the foundation for auditability and traceability of the organisation’s data.

This covers the content of user prompts – who asked what question, and which agent that question triggers; the data, documents, and entities retrieved by the multi-agent system to generate an answer; and metrics around the total processing cost each time the multi-agent system runs, which is measured in tokens, a unit of data processing by AI models. The latter facilitates comparisons of the running costs of different prompts against different AI models, which may influence architectural decisions regarding what lies behind the multi-agent system.

Finally, the goal for users is to understand how the response they receive from the multi-agent system was generated, using which data and agents, and what references are available to verify the source material used by the multi-agent system to produce the end response. This explainability is important for trust, since an AI system can often produce a different answer every time a question is asked, even if the question is largely the same each time.

What graph technology brings to the table

Graph database technology is already helping AI systems to produce more reliable outputs that are less prone to hallucinations. It achieves this by considering the relationships between various pieces of data and data sources, far more than standard databases. This extra correlation is valuable to improving answer quality and explainability.

However, graph technology has an additional role to play: as a means to orchestrate, manage, trace, and log all the steps involved in generating that response. Graph-based visualisation is well-suited to understanding everything that happens in a multi-agent system once a question is posed: how the question is broken down into executable parts, which agent takes on which part, what tools or data sources are used in formulating the response, and so on.

Specifically, it is useful for auditing who accesses what documents or information using AI agents, which is particularly important if the information is confidential. For added integrity and transparency, the graph will visualise logs of how every single AI agent interacts with other agents, applications, and its data domain, all of which will be transparently stored in the graph.

Because the orchestration of all the agentic tools is also stored in the graph, this acts as a memory layer that can be recalled when a similar question or subset of a question is asked of the agentic system in the future. This aids in the repeatability and predictability of outputs, given that agentic systems are otherwise prone to answering the same question differently each time it’s asked.

Finally, the graph is able to store metrics for every action or run of the multi-agent system, such as execution time, the model, and the number of tokens used. This provides a point of comparison, enabling the cost and processing efficiency of tweaks in a multi-agent setup to be tested and measured.