Leadership, routing, and load balancing
Enterprise Edition

Elections and leadership

The servers in a cluster use the Raft protocol to ensure consistency and safety. An implementation detail of Raft is that it uses a Leader role to impose an ordering on an underlying log with other instances acting as Followers which replicate the leader’s state. Specifically in Neo4j, this means that writes to the database are ordered by the server currently playing the Leader role for the respective database.

Only servers hosting a database in primary mode can be elected leaders for that database, provided that the cluster contains more than one primary server. If a Neo4j DBMS cluster contains multiple databases, each one of those databases operates within a logically separate Raft group, and therefore each has an individual leader. This means that a server may act both as Leader for some databases, and as Follower for other databases.

If a follower has not heard from the leader for a while, then it can initiate an election and attempt to become the new leader. The follower makes itself a Candidate and asks other servers to vote for it. If it can get a majority of the votes, then it assumes the leader role. Servers do not vote for a candidate which is less up-to-date than itself. There can only be one leader at any time per database, and that leader is guaranteed to have the most up-to-date log.

Elections are expected to occur during the normal running of a cluster and they do not pose an issue in and of itself. If you are experiencing frequent re-elections and they are disturbing the operation of the cluster then you should try to figure out what is causing them. Some common causes are environmental issues (e.g. a flaky networking) and work overload conditions (e.g. more concurrent queries and transactions than the hardware can handle).

Leadership balancing

Write transactions are always routed to the leader for the respective database. As a result, unevenly distributed leaderships may cause write queries to be disproportionately directed to a subset of servers. By default, Neo4j avoids this by automatically transferring database leaderships so that they are evenly distributed throughout the cluster. Additionally, Neo4j automatically transfers database leaderships away from instances where those databases are configured to be read-only using server.databases.read_only or similar.

Client-side routing

Client-side routing means that the application decides which cluster server to send specific requests to. Typically, this ensures that write operations are sent to the server currently acting as the writer for the target database, while read operations are sent to other servers.

Client-side routing is based on getting a routing table from a cluster server, and then using that information to make the routing decisions. Use the dbms.routing.getRoutingTable() procedure to obtain a routing table.

A routing table contains information about the servers and their roles as writers, readers, and routers for a specific database. There is usually one writer, though there may be none if the database is read-only or unhealthy. With the default configuration, all other servers are considered readers, i.e. the writer is not in the list of readers. This is to let it focus on the write load and not have to manage two kinds of interactions. Typically, all servers that host the database are listed as routers, which are servers that can be contacted to get a new routing table for that database.

Neo4j Drivers retrieve a routing table the first time they attempt to connect to a database, and fetch a fresh one after the configured time-to-live, or if it seems the routing table has got out of date.

For example, if the routing table lists server-3 as the writer for the database, but write requests are rejected with a not able to write error, the driver may request a new routing table, since the writer role may have moved to a different server.

Routing policies

You can control the routing table that servers provide by using routing policies. Policies filter the full set of possible servers for each category according to the rules you define. For example, this can be used to preferentially route to a local data centre, or to specific large machines, depending on your policies.

Server-side routing

Server-side routing is a complement to the client-side routing.

In a cluster deployment of Neo4j, Cypher queries may be directed to a cluster member that is unable to run the given queries. With server-side routing enabled, such queries are rerouted internally to a cluster member that is expected to be able to run them. This situation can occur for write-transaction queries when they address a database for which the receiving cluster member is not the leader.

The cluster role for cluster members is per database. Thus, if a write-transaction query is sent to a cluster member that is not the leader for the specified database (specified either via the Bolt Protocol or with Cypher USE clause), server-side routing is performed if properly configured.

Server-side routing is enabled by the DBMS, by setting dbms.routing.enabled=true for each cluster member. The listen address (server.routing.listen_address) and advertised address (server.routing.advertised_address) also need to be configured for server-side routing communication.

Client connections need to state that server-side routing should be used and this is available for Neo4j Drivers and HTTP API.

Neo4j Drivers can only use server-side routing when the neo4j:// URI scheme is used. The Drivers do not perform any routing when the bolt:// URI scheme is used, instead connecting directly to the specified host.

On the cluster-side you must fulfill the following prerequisites to make server-side routing available:

Set dbms.routing.enabled=true on each member of the cluster.
Configure server.routing.listen_address, and provide the advertised address using server.routing.advertised_address on each member.
Optionally, you can set dbms.routing.default_router=SERVER on each member of the cluster.

The last prerequisite enforces server-side routing on the clients by sending out a routing table with exactly one entry to the client. Therefore, dbms.routing.default_router=SERVER configures a cluster member to make its routing table behave like a standalone instance. The implication is that if a Neo4j Driver connects to this cluster member, then the Neo4j Driver sends all requests to that cluster member. Please note that the default configuration for dbms.routing.default_router is dbms.routing.default_router=CLIENT. See dbms.routing.default_router for more information.

The HTTP-API of each member benefits from these settings automatically.

Server-side routing connector configuration

Rerouted queries are communicated over the Bolt Protocol using a designated communication channel. The receiving end of the communication is configured using the following settings:

Server-side routing driver configuration

Server-side routing uses the Neo4j Java driver to connect to other cluster members. This driver is configured with settings of the format:

dbms.routing.driver.<setting>

Server-side routing encryption

Encryption of server-side routing communication is configured by the cluster SSL policy. For more information, see Cluster Encryption.

Glossary

allocator: A component in the cluster that allocates databases to servers according to the topology constraints specified and an allocation strategy.
asynchronous replication: Asynchronous replication is used by secondary copies to poll for new transactions, which means they cannot be guaranteed to have received the most recent transactions. This enables efficient scale-out of read-performance.
Aura instance: A fully-managed DBMS represented by a single instance ID, that is running in the Neo4j Aura cloud.
auto-commit transaction: An automatically committed transaction that contains a single query.
Bolt protocol: Bolt is a protocol used for interaction between Neo4j instances and drivers.
bookmark: A marker the client can request from the cluster to ensure that it is able to read its own writes so that the application’s state is consistent and only databases that have a copy of the bookmark are permitted to respond.
category (Bloom): A category is based on a node label and is defined in a Perspective as a way of visually distinguishing nodes with the same label(s).
causal consistency: All servers in a cluster agree on the order in which transactions take place. The position of a server on the causal chain can be guaranteed using a bookmark.
cluster: A Neo4j DBMS that spans multiple servers working together to increase fault tolerance and/or read scalability. Databases on a cluster may be configured to replicate across servers in the cluster thus achieving read scalability or high availability.
client application: Software that interacts with a Neo4j server.
commit: A commit is the successful completion of a transaction, which ensures durability of any changes made. For more details, visit Operations Manual → Transaction management.
composite database: Composite databases are the means to access partitioned graph data with a single Cypher query.
constraint: Constraints are sets of data modeling rules that ensure the data is consistent and reliable.
Cypher®: Neo4j’s graph query language.
data model: A data model defines how information is organized in a database. A good data model will make querying and understanding your data easier. In Neo4j, the data models have a graph structure.
database: A database is a container used by the DBMS to manage and store graph data. The physical structure of data is controlled by the database.
database vs graph: Databases are the physical containers of graph data. Graphs are the logical structure of data in Neo4j.
Database Management System: Database Management System, or DBMS, capable of managing multiple databases. A DBMS may run on a single server, or span several servers configured as a cluster.
database schema: The prescribed property existence and datatypes for nodes and relationships.
deallocate: An act of removing a database from a server or a server from a cluster without loss of data or reduced fault tolerance.
degree (of a node): The number of relationships of a specific node; loops are counted twice.
disaster recovery: A manual intervention to restore availability of a cluster, or databases within a cluster.
driver: A software library that provides access to Neo4j from a particular programming language.
election: In the event that the Raft leader becomes unresponsive, followers automatically trigger an election and vote for a new leader.
entity: A node or a relationship.
expression (Cypher): A component of a Cypher query which produces values. It may be used in projections, as a predicate, or when setting properties on graph elements.
fabric: Fabric is the architectural design of a unified system that provides a single access point to local or distributed graph data.
fault tolerance: A guarantee that a cluster can maintain a database’s persistence and availability in the event of one or more servers failing.
follower: A primary copy of a database acting as a follower, receives and acknowledges synchronous writes from the leader.
Generative AI (GenAI): A type of artificial intelligence (AI) system that generates text, images, or other media in response to prompts.
graph: A logical representation of a set of nodes where some pairs are connected by relationships.
index: Data structure that improves read performance of a database.
knowledge graph: A specific type of graph that has an organizing principle so that a user (or a computer system) can reason about the underlying data. The organizing principle provides an additional layer of structure that adds context to support knowledge discovery.
label: Marks a node as a member of a named and indexed subset. A node may be assigned zero or more labels.
leader: A single primary copy of a database is designated as the leader. It receives all write transactions from clients and replicates writes synchronously to followers and asynchronously to secondary copies of the database.
main database: In terms of Neo4j Enterprise Studio, the database(s) containing the user’s data. Can exist in the same Neo4j deployment as the tool asset database.
motif: A description of a specific pattern within a graph.
node: A node represents an entity or discrete object in your graph data model. Nodes can be connected by relationships, hold data in properties, and are classified by labels.
operator: A symbol representing a mathematical or logical operation.
parameter: Named value provided when running a Cypher statement.
path: A sequence of nodes and the relationships connecting them, that does not contain duplicate relationships. Several paths can match a pattern.
pattern: A specific arrangement of nodes and relationships that can be matched in a graph. A pattern follows a motif.
perspective (Bloom): A Perspective defines a certain business view or domain that can be found in the target Neo4j graph. A single Neo4j graph can be viewed through different Perspectives, each tailored for a different business purpose.
primary: A copy of the database that is able to process write transactions and is eligible to be elected as a leader. It participates in fault tolerant writes as it is part of the majority required to acknowledge and commit write transactions.
primary vs secondary: In a cluster, databases can operate in either primary or secondary mode. Primary databases are able to process write and read transactions, ensuring fault tolerance. Secondary databases are replicated asynchronously from primaries, and their main purpose is to provide read scaling within the cluster.
project (Aura): An isolated environment in the unified Aura console that contains its own database instances, configurations, and resources. Preceded by tenant in the classic Aura console.
property: Properties are key-value pairs that are used for storing data on nodes and relationships.
query (Cypher): A statement that retrieves or writes information to a database.
Raft group: A group of servers that are participating in hosting a particular database in primary mode.
Raft group member: A server that is participating in a Raft group. A server can be a member of one or more groups.
Raft log: A shared log between all Raft group members that is guaranteed to be consistently updated and viewed by those members. The log contains both database data and operational state of the Raft group.
Raft protocol: The networking mechanism that enables a database to replicate its data across multiple servers to give high availability for accessing the data and high durability to the data stored.
read scaling: Distributing query load by creating additional database copies hosted in secondary mode (read-only).
relationship: A relationship represents a connection between nodes in your graph data model. Relationships connect a source node to a target node, hold data in properties, and are classified by type.
secondary: An asynchronously replicated copy of the database that provides read scaling within the cluster.
seed: A seed is a database dump or a full backup used to create a database on a cluster. This is sometimes called seeding.
server: A physical machine, a virtual machine, or a container running an instance of Neo4j. Servers can be standalone or part of a cluster.
session: A causally linked sequence of transactions.
session consistency: An alternative name for Neo4j’s causal consistency.
standalone: A single server running Neo4j and not part of a cluster.
synchronous replication: Synchronous replication requires the leader primary to replicate a transaction and block the commit until a quorum of the follower primaries acknowledges that the transaction is successfully replicated. Once the transaction is replicated, the commit is allowed to proceed. This ensures data durability and consistency within the cluster.
system database: A database used by Neo4j to store system information.
tenant (Aura): An isolated environment in the classic Aura console that contains its own database instances, configurations, and resources. Replaced by project in the unified Aura console.
tool asset database: In terms of Neo4j Enterprise Studio, the database where tools' assets are stored. This can be in the same Neo4j deployment as the main database(s) or in a separate deployment.
topology: A configuration that describes how the copies of a database should be spread across the servers in a cluster, see primary mode and secondary mode.
transaction: A transaction comprises a unit of work performed against a database. It is treated in a coherent and reliable way, independent of other transactions. Transactions comply with the ACID consistency model (atomic, consistent, isolated, and durable).

Leadership, routing, and load balancingEnterprise Edition