Flexible Data Pipelines: 5-Minute Interview with Pat Patterson, Technical Director, StreamSets

“With Neo4j, we can run analyses across relationships. These analyses are not feasible or even possible when the data is spread out among different systems,” said Pat Patterson, Technical Director, StreamSets.

In this week’s five-minute interview, we discuss how StreamSets uses Neo4j to build flexible data pipelines that bring together continuous and streaming data from multiple sources.

Tell us about how StreamSets is using Neo4j.

Pat Patterson: StreamSets was founded about four years ago, to create a new generation of data integration tools. We’re about moving data from source to destination systems in the world of continuous streaming and big data. I work with our open source community, helping enterprises move data between systems.

As we’ve moved away from relational databases, we’ve moved away from the traditional ETL world of static schema. That’s where we find ourselves now: working with changing data, changing metadata, and being a flexible tool to build those pipelines that allow companies to run their data flow operations in an efficient way.

What made you choose Neo4j?

Patterson: With master data management, we need to integrate data from a variety of sources and join it together in Neo4j so that we can run analyses across those relationships. These analyses are not feasible or even possible when the data is spread out among different systems.

What do you think is in store for the future of graph technology?

Patterson: Emil spoke this morning about the Neo4j graph database becoming a data store in its own right, rather than just a point tool for visualization and analysis.

That’s really interesting because it means that enterprises are going to want to move data into that data store, read it from that data store and it becomes part of their data operations.

At StreamSets, we see Neo4j becoming more and more important as a data store that we work with.

Anything else you’d like to add?

Patterson: I’ve been working with Neo4j for about 18 months now, and it’s always a pleasant experience. The tools are very stable, very performant, and it’s a joy to build an integration with Neo4j.

Want to share about your Neo4j project in a future 5-Minute Interview? Drop us a line at content@neo4j.com