1.2. Designing for the enterprise

When designing your solution, some of your first considerations will concern your functional requirements and the type of technology choices you make to meet them. Some of those functional requirements likely will include a need to scale to many concurrent users, maintain consistent uptime, or the ability to recover from a system failure and maintain availability. These are important production related questions that help drive your technical decisions and can ultimately guide you to choose to cluster Neo4j.

This covers four major advantages of using Neo4j clustering:

  1. Read Scalability
  2. High Availability
  3. Disaster Recovery
  4. Analytics

1.2.1. Read scalability

Clustering Neo4j allows you to distribute read workload across a number of Neo4j instances. You can take two approaches to scaling your reads with Neo4j:

Distribute load balance reads to any instance in the cluster

Neo4j’s clustering architecture replicates the entire database to each instance in your cluster. Therefore you are able to direct any read from your application to any instance without much concern for data locality.

Figure 1.1. Distribute load balance reads to any instance in the cluster
cluster w lb neo styled
When would you choose this method?
  • You need to scale up the number of concurrent read requests
  • Your data has no natural or obvious way of partitioning reads
  • A significant portion of the data that needs to be read can reasonably be expected to already be in memory on any instance in the cluster.
Distribute direct reads to specific instances in the cluster

This is sometimes referred to as "cache sharding". The strategy simply allows you to take advantage of natural partitions in your data to direct reads to particular instances where the system will already have those datasets in memory. This approach is significantly beneficial when your total active dataset is much larger than can fit in memory in any particular instance.

Figure 1.2. Cache sharding
cache sharding neo
When would you choose this method?
  • Your total active data set is larger than can reasonably be expected to fit in memory in any single instance in your cluster.
  • A natural or obvious partition can be identified in your dataset
  • You have the application and operations ability to direct which instances are read from.

1.2.2. High availability

Figure 1.3. High availability cluster
cluster failover neo styled

A significant and fundamental functional requirement for any service or application is the requirements for overall availability. Very often this question is answered more by the demands of the users, the times they would be interacting with the solution, the impact downtime would have on the business or users of the system to complete their roles, or the financial impact of a system failure. These are not always customer-facing solutions and can be critical internal systems.

Availability can often be addressed with various strategies for recovery or mirroring. However, Neo4j’s clustering architecture is an automated solution for ensuring Neo4j is consistently available to your application and end-users.

How do you know if you need Neo4j’s clustering for high availability reasons?
  • Neo4j is serving data for a critical business or consumer-facing solution that would impact the ability for the company to conduct business if the component were down.
  • Global end-users with random access behavior are depending on the data stored in Neo4j.
  • Business continuity must be ensured by availability of disaster recovery features.

1.2.3. Disaster recovery

Disaster recovery, in general terms, defines your ability to recover from major outages of your services. The most common example is whole-datacenter outages where many services are disrupted. In these cases a disaster recovery strategy can define a failover datacenter along with a strategy for bringing services back online.

Neo4j clustering can accommodate disaster recovery strategies that require very short-windows of downtime or low tolerances for data loss in disaster scenarios. By deploying a cluster instance to an alternate location, you have an active copy of your database up and available in your designated disaster recovery location that is consistently keeping up with the transactions against your database.

Why would you choose Clustering in support of Disaster Recovery?
  1. Minimize downtime: Your application availability demands are very high and you cannot sustain significant periods of downtime.
  2. Require real-time: You already employ a disaster recovery strategy for other application or service components that are near real-time.
  3. Minimize data loss: You have a significantly large database that changes frequently and have low tolerance for data loss in a disaster scenario.

1.2.4. Analytics

Your application needs to access data for its purposes. It reads data, writes data, and is generally keeping your application service or end-users happy. Then comes the analytics team that wants to collect and aggregate data for their reports. Next thing you know, you have a set of long-running compute queries running against your production databases and disrupting your service or end-users' happiness.

You can’t avoid servicing the needs of the analytics requests, but you can box in the impact their queries have on your service. Neo4j clustering can be used to include separate instances entirely in support of query analytics, either from end users or from BI tools. Using clustering means the data is always up to date for analytics queries as well.

When would you decide to use clustering to support analytics needs?
  • You have regular BI users that consistently need to run analytics against the most recent versions of the data
  • Your analytics includes queries that aggregate over large or entire sets of data
  • Your analytics processes include complex compute algorithms for predictive or modeling purposes