Investor IQT Uses Neo4j to Help U.S. Intelligence Agencies Find the Best Cutting-Edge Technology
The ChallengeIQT bridges the gap between its government customers, the startup world and the venture capital (VC) community funding them. It uses its deep understanding of these diverse communities to identify and support the tech innovations that best match America’s intelligence missions – the most enduring and difficult problems facing US security agencies.
IQT invests in approximately 50 companies a year. But to do so successfully, its staff must maintain a network of connections with VCs, startups, universities and technology centers. They must also evaluate tech innovations from a wide range of sectors – including biology, space systems, communications, cybersecurity, analytics, infrastructure, IoT, robotics, artificial intelligence, materials and energy.
IQT’s ultimate aim is to break down (or “decompose”) these often very different product sets – anything from a drone to a 5G wireless communication system, for example – into their core complementary capabilities. It will then mix-and-match these features to create new custom technology “stacks” that solve its customers’ complex problems.
But IQT faced a significant problem in achieving this mission.
“We had no way to automate these exercises,” explained Chief Architect Ravi Pappu. “Technology evaluation and decomposition was done manually in spreadsheets and presentation diagrams. Tech suppliers were matched manually, and the process of identifying new product combinations was slow and generated few ideas.”
The SolutionPappu recognized that IQT had a series of connected data challenges: mapping the connections between intelligence agencies, their mission problems and startups; integrating masses of information drawn from different suppliers and other sources; and quickly pinpointing significant links between the different tech products to create new solutions.
And from his experience of using Neo4j in a previous role, he recognized the best way to solve these issues was by a graph database.
“Our tools didn’t reflect the connectedness of our data,” Pappu said. “That’s what we solved with Neo4j.”
“The fundamental reason for us to choose a graph database over other systems is that there is enormous value in the relationships between different objects,” he explained. “Also, we have many different data silos in our organization, and we wanted to do a JOIN across all of them. Graph databases are the best way to do this – and we picked Neo4j because of its maturity, commercial support, my prior experience, and the willingness of the company to work with us on pricing.”
In January 2017, IQT began building a 100,000-node Neo4j database accessed by ARQ, a bespoke Neo4j-based front-end written in Go. This went live in mid-2017, with new releases every three months since. IQT is also in the process of integrating Elasticsearch to improve the relevance of search queries.