In Neo4j, administrative operations such as database creation do not happen synchronously.
Instead, the changes are first written to the
system database, recording the requested state of the DBMS.
Each server has a reconciler, an internal component which observes the requested state and makes changes to the local server to match that state.
For example, the
system database may record that the database
bar is expected to be running on
When the reconciler on
server-1 becomes aware of that, it starts
This means the loss of any one server cannot prevent all progress on an operation.
Executing an operation and getting back a successful response means that the request is safely committed to the
system database, and will be processed by each member of the cluster at some point, assuming the server is healthy enough.
Servers can become aware the same operation at different times, since the
system database is just another Raft group, where followers and replicas can lag behind the leader in some situations.
Eventually, the new state gets propagated to all healthy servers, and they take action on it.
If you want to be sure that each server has processed an administrative operation before running the next action, you can use the
WAIT makes the statement not return until all servers have seen the transaction, and their reconciler has finished processing it.
CREATE DATABASE foo WAIT
When a statement with
An operation might only succeed on some servers. For instance, some servers might be offline when a database is created, or a disk error might cause a server to fail to start a database. In this situation, the operation has not failed as a whole, since the database may even be running (although with less fault tolerance than you intended).
Some failed operations can just be re-tried: for example,
START DATABASE can be re-run if not all members started successfully, and you think the cause of that has been resolved.
Other failures only require the underlying problem to be resolved.
When the issue is fixed, the reconciler will make the intended change, because it is still trying to achieve the requested state.
For example, if a new server fails during a
DEALLOCATE DATABASES FROM SERVER (meaning a database cannot be safely moved), fixing the new server should be enough to resolve the problem, as the new server will start its copy of the database as soon as the target is ready, allowing the deallocating server to shut down its copy.