Streaming Data from Confluent Cloud to AuraDB for Real-Time Graph Analytics (Part 1)


Neo4j now supports Confluent’s Custom Connectors so that customers can stream data from Confluent Cloud into Neo4j AuraDB to perform real-time graph analytics.

This integration comes as a result of our ongoing partnership with Confluent and our joint commitment to empowering developers and organizations to harness the power of connected data.

Neo4j Connector running as a Custom Connector inside Confluent Cloud, sinking data to AuraDB

Combining knowledge graphs with streaming updates unlocks valuable use cases to get answers to end users for the following use cases; network analysis, digital twins, cybersecurity, fraud detection, and trustworthy AI experiences.

Confluent Cloud is a fully-managed, cloud-native event streaming platform powered by Apache Kafka®. It enables organizations to build real-time, scalable data pipelines and applications effortlessly.

Custom Connectors, introduced by Confluent at Kafka Summit London in May 2023, provide users with the flexibility to extend Confluent Cloud beyond the fully managed connectors available on their platform. Custom connectors are created as Kafka Connect plugins, and users can upload their own plugins, modified open-source connector plugins, or third-party connector plugins like the Neo4j Connector for Confluent.

Neo4j’s Connector for Confluent has been available for many years from Confluent Hub, and hundreds of customers trust it to move data between self-managed Confluent Clusters and Neo4j AuraDB, our fully managed database-as-a-service. With Custom Connectors in Confluent Cloud, customers can seamlessly integrate AuraDB, Neo4j’s cloud database, with Confluent Cloud’s event streaming platform without needing to manage Kafka Connect infrastructure. This powerful combination allows you to use the strengths of both platforms, enabling you to build graph-based streaming applications and unlock deeper insights from your connected data with a wide range of possibilities, including real-time recommendations, fraud detection, social network analysis, and much more. For customers looking to migrate from self-managed Confluent, and Apache Kafka deployments to Confluent Cloud, Custom Connectors provide the means to start making the switch.

Benefits of Neo4j’s Connector for Confluent Cloud

Real-Time Insights

Stream data from Confluent Cloud to Neo4j in real time, enabling you to derive actionable insights from connected data immediately.

Simplified Data Integration

Seamlessly integrate Neo4j’s graph database with Confluent Cloud’s event streaming platform, simplifying your data integration architecture.

Enhanced Data Pipelines

Build robust and scalable data pipelines by combining the power of Confluent Cloud’s event streaming platform with Neo4j’s graph database.

Graph-Powered Applications

Leverage the strengths of Neo4j’s graph database and Confluent Cloud to build graph-powered applications that uncover hidden relationships and deliver enhanced user experiences.

The current requirements and limitations of the Custom Connectors are described here.

Support

While Neo4j has partnered with Confluent to bring support for Custom Connectors to our joint customers, support is through your Neo4j Support Contract, and not your Confluent Contract as stated here. Please raise support requests through Neo4j’s support processes, in the same way you would when running a self-managed Confluent or Apache Kafka Cluster. You will need to obtain your logs from the Confluent Cloud and provide them to our Support team, along with the usual logs for Neo4j.

Summary

The integration of Neo4j’s Custom Connector with Confluent Cloud brings together the strength of graph databases and event streaming platforms, empowering developers to build powerful graph-based applications with real-time insights. We are thrilled to offer this integration and look forward to seeing the innovative solutions that developers will build using Neo4j and Confluent Cloud together.

Unlock the full potential of your connected data with Neo4j’s Custom Connector for Confluent Cloud. Start building your graph-powered cloud applications today by checking out the following resources on our Connector for Confluent. Start for free with Aura here and read our documentation.

Stay tuned for part 2 of this blog next week when I will show you how quick and easy it is to deploy Neo4j’s Connector for Confluent into Confluent Cloud as a Custom Connector and create a data sink from a Confluent Topic into a Neo4j AuraDB database.


Streaming Data from Confluent Cloud to AuraDB for Real-Time Graph Analytics (Part 1) was originally published in Neo4j Developer Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.