Migrate Neo4j drivers

This chapter describes the necessary information to migrate Neo4j drivers from 1.7 to 4.0.

The 4.0 drivers have been designed to work with Neo4j 4.0. In 4.0, the drivers are built to provide a user-friendly and unified API across all languages, to take advantage of all new features and services introduced in Neo4j 4.0.

In previous versions of Neo4j, client-server communication used encrypted local connections and generated a self-signed certificate out of the box. In 4.0 however, the default is set to unencrypted. Please see Driver Manual → Connection URIs for more information.

Neo4j 4.0 introduces a reactive API, compatible with the Reactive Streams standard. This enables fine-grained control of the data flow for Cypher query results, including the ability to pause or cancel part-way through. Read more in Driver Manual → Queries and results.

When using the 4.0 driver to connect to a 4.0 database, it is possible to work with multiple databases. From a driver API perspective, this means that one database must be selected for use as an execution context for transactions within a session. This can be configured on session construction. If no database is selected, the driver will connect to the server’s default database.

Drivers 1.7 work in fallback mode with Neo4j 4.0. They do not support features introduced in Neo4j 4.0, such as multiple databases, Neo4j Fabric, and fine-grained access control. To be able to run multiple databases online concurrently and to do distributed queries over them, you must migrate from 1.7 to 4.0.

The examples in this chapter are mainly written in Java, using the Java driver. However, similar code can be translated to other languages.

1. New driver releases

Starting with Neo4j 4.0, the versioning scheme for the database, driver and protocol are all aligned. For supported drivers, this means that the version number will go from 1.7 to 4.0.

The new 4.0 drivers for different languages can be found with the links below:

The Go Driver 4.0 is under construction.

The current stable (version 1.8) for the Go Driver will work in fallback mode with Neo4j 4.0. Therefore, all functionality that exists in Neo4j 3.5 will also be available in Neo4j 4.0, but new functionality introduced in 4.0 will not.

Note that a 1.7+ driver communicating with a 4.0 server may need to be have encryption explicitly switched off. This is due to a change in the defaults between Neo4j 3.x and 4.0.

2. Compatibility

The compatibility between Neo4j 3.5 and 4.0, and 4.0 Bolt drivers is illustrated in the tables below:

Table 1. Protocols
Neo4j 4.0 Neo4j 3.5

Bolt v4.0

All features fully supported.

Not supported.

Bolt v3

All features fully supported, but the support may be removed in next version.

All features fully supported.

Bolt v2

Not supported.

All features fully supported, but the support may be removed in next version.

Bolt v1

Not supported.

All features fully supported, but the support may be removed in next version.

Table 2. Drivers
Neo4j 4.0 Neo4j 3.5

Bolt version

Support

Bolt version

Support

Java Driver 4.0

Bolt v4.0

All features fully supported.

Bolt v3

All features fully supported, but the support may be removed in next version.

.NET Driver 4.0

Bolt v4.0

All features fully supported.

Bolt v3

All features fully supported, but the support may be removed in next version.

JavaScript Driver 4.0

Bolt v4.0

All features fully supported.

Bolt v3

All features fully supported, but the support may be removed in next version.

Python Driver 4.0

Bolt v4.0

All features fully supported.

Bolt v3

All features fully supported, but the support may be removed in next version.

Go Driver 1.81

Bolt v3

All features partially supported.

Bolt v3

All features fully supported.

Go Driver 1.71

Bolt v3

All features partially supported.

Bolt v3

All features fully supported.

[1]Neo4j The Go Driver 4.0 is still under construction. Please refer to https://github.com/neo4j/neo4j-go-driver for the latest versions that are available.

3. What’s new?

  • Bolt v4.0 is implemented in both 4.0 drivers and 4.0 servers.

  • Reactive API is now available with 4.0 servers. To make use of the reactive API, the starting point is RxSession on the driver object.

  • With 4.0 servers, session instances should now be acquired against a specific database. Causal chaining is still respected on each database (transactions cannot span across multiple databases). The driver itself connects to Neo4j DBMS.

  • A new feature detection method driver.supportsMultiDb() is added for querying whether the remote database supports multiple databases.

  • A new driver.verifyConnectivity() method is introduced for connectivity verification purposes. The driver instances by default will not verify DBMS availability after construction.

  • New connection URI schemes with variants that contain extra encryption and trust information - neo4j+s, bolt+s, neo4j+ssc and bolt+ssc. The +s variants enable encryption with a full certificate check, and the +ssc variants enable encryption, but with no certificate check. This latter variant is designed specifically for use with self-signed certificates. For more information, see Additional URI Schemes.

Example 1. Detecting multiple database support
import org.neo4j.driver.Driver;
import org.neo4j.driver.Result;
import org.neo4j.driver.Session;
import org.neo4j.driver.SessionConfig;
import org.neo4j.driver.Values;

...
private final Driver driver;
...

public void printGreeting( final String message )
{
   SessionConfig sessionConfig = driver.supportsMultiDb() ? SessionConfig.forDatabase( "neo4j" )
                                                          : SessionConfig.defaultConfig();

   try ( Session session = driver.session( sessionConfig ) )
   {
       String greeting = session.writeTransaction( tx -> {
           Result result = tx.run( "CREATE (a:Greeting) SET a.message = $message RETURN a.message + ', from node ' + id(a)",
                                   Values.parameters( "message", message ) );
           return result.single().get( 0 ).asString();
       } );
       System.out.println( greeting );
   }
}

4. Breaking changes

  • The driver’s default configuration for encrypted is now false (meaning that driver will only attempt plain text connections by default). Connections to encrypted services (such as Neo4j Aura) should now explicitly be set to encrypted.

  • When encryption is explicitly enabled, the default trust mode is to trust the CAs that are trusted by operating system. This means that encrypted connections to servers holding self-signed certificates will now fail on certificate verification by default.

  • Hostname verification is turned on by default when encryption is turned on.

  • v1 is removed from drivers’ package name. For example, in the Java driver, all public APIs are in the package org.neo4j.driver instead of the old org.neo4j.driver.v1.

  • The neo4j:// scheme replaces bolt+routing:// and can be used for both clustered and single-instance configurations. This is a rename only, and neo4j:// URIs can still be used to communicate with Neo4j 3.x clusters. Please note though that Neo4j 3.x standalone instances do not expose a routing interface.

    The bolt:// scheme is used for direct connection to a particular Neo4j server. This scheme is no longer required for standalone machines, however. Neo4j 4.0 now exposes a routing interface for all deployment topologies, allowing neo4j:// URIs to be used for all deployments. The bolt:// scheme is now mainly only useful when targeting a specific machine, rather than an entire service. This can be a certain server in a Causal Cluster or the one server in a single-instance environment.

  • For drivers where synchronous and asynchronous methods are both implemented, asynchronous methods have been extracted out and put in AsyncSession, whereas synchronous methods remain in Session. This change ensures that blocking and non-blocking APIs can never be mixed together.

  • Driver#session method now makes use of a session configuration object or option builder, rather than method arguments.

  • Bookmark has changed from a string, and/or a list of strings, to a Bookmark object.

  • For synchronous Transaction API, Transaction#success and Transaction#failure have been removed.

    The success/close pattern for Transaction objects is now obsolete and has been fully superseded by commit and rollback methods. However, unlike Transaction#success, which only marks the transaction to be successful and then waits for Transaction#close to actually perform the real commit, Transaction#commit commits the transaction immediately.

    A transaction in 4.0 can only be committed or rolled back once. If a transaction is not committed explicitly using Transaction#commit, Transaction#close will roll back the transaction.

  • Statement has been renamed to Query. StatementResult has been renamed to Result. Similarly, StatementResultCursor has been renamed to ResultCursor.

  • A result can only be consumed once.

    A result is consumed if either the query result has been discarded by invoking Result#consume, and/or the outer scope where the result is created, such as a transaction or a session, has been closed. Attempts to access consumed results will be responded with a ResultConsumedException.

  • The experimental StatementRunner.typeSystem() has moved to Driver.defaultTypeSystem().

  • LoadBalancingStrategy is removed from Config class, and the drivers always default to LeastConnectedStrategy.

  • The recommended Driver Connection URI scheme is as follows:

    Table 3. Recommended Driver Connection URI scheme.

    4.0 drivers

    1.7 drivers

    4.0 Neo4j

    Single instance

    neo4j

    bolt

    Cluster core members

    neo4j

    neo4j (bolt+routing)

    Cluster read replicas

    neo4j

    bolt

    3.5 Neo4j

    Single instance

    bolt

    bolt

    Cluster core members

    neo4j

    neo4j (bolt+routing)

    Cluster read replicas

    bolt

    bolt

4.1. Driver-specific breaking changes

In addition to the breaking changes mentioned above, which apply in general for all drivers, the following drivers have further breaking API changes:

.NET Driver

  • The Neo4j.Driver package contains only the asynchronous API.

  • The IDriverLogger has been renamed to ILogger.

  • TrustStrategy is replaced with TrustManager.

Example 2. Migrating from the 1.7 .NET Driver to the 4.0 .NET Driver.
Example code for the 4.0 .NET driver Example code for the 1.7 .NET driver
using Neo4j.Driver.Simple;
...
private readonly IDriver _driver;
private readonly string _previousNeo4jSessionBookmark;
...
public void PrintGreeting(string message)
{
   using (ISession session = _driver.Session(o =>
      o.WithDatabase("neo4j")
       .WithDefaultAccessMode(AccessMode.Write)
       .WithBookmarks(_previousNeo4jSessionBookmark)))
   {
       using (ITransaction transaction = session.BeginTransaction())
       {
           Query query = new Query("CREATE (a:Greeting) SET a.message = $message RETURN a.message + ', from node ' + id(a)", new Dictionary<string, object>{{"message", message}});
           IResult result = transaction.Run(query);

                string greeting = result.Single()[0].As<string>();
                Console.WriteLine(greeting);
                transaction.Commit(); // commit immediately here
            }
        _previousNeo4jSessionBookmark = session.LastBookmark;
        }
     }
using Neo4j.Driver;
...
private readonly IDriver _driver;
private readonly string _previousSessionBookmark;
...
public void PrintGreeting(string message)
{
   using (ISession session =_driver.Session(
      AccessMode.Write, _previousSessionBookmark))
   {


      using (ITransaction transaction = session.BeginTransaction())
      {
         Statement query = new Statement("CREATE (a:Greeting) SET a.message = $message RETURN a.message + ', from node ' + id(a)", new Dictionary<string, object>{{"message", message}});
          IStatementResult result = transaction.Run(query);
          transaction.Success(); // mark success, actually commit will happen in transaction.Dispose()
          var greeting = result.Single()[0].As<string>();
          Console.WriteLine(greeting);
   }
   _previousSessionBookmark = session.LastBookmark;
   }
}

JavaScript Driver

  • session#close() and driver#close() both now return Promises, and no longer accept callback function arguments.

  • driver.onError and driver.onCompleted callbacks have been completely removed. Errors should be monitored on related code paths (i.e. through Promise#catch, etc.).

Example 3. Migrating from the 1.7 JavaScript Driver to the 4.0 JavaScript Driver.
Example code for the 4.0 JavaScript driver Example code for the 1.7 JavaScript driver
var neo4j = require('neo4j-driver')
...
const driver = neo4j.driver(uri, neo4j.auth.basic(user, password))
...

const session = driver.session()
try {
  const tx = session.beginTransaction()
  const result = await tx.run('CREATE (a:Greeting) SET a.message = $message RETURN a.message + ", from node " + id(a)', { message: 'hello, world' })
  const greeting = result.records[0].get(0)
  console.log(greeting)
  await tx.commit()
} finally {
  await session.close()
}
var neo4j = require('neo4j-driver').v1
...
const driver = neo4j.driver(uri, neo4j.auth.basic(user, password))
...

const session = driver.session()
try {
  const tx = session.beginTransaction()
  const result = await tx.run('CREATE (a:Greeting) SET a.message = $message RETURN a.message + ", from node " + id(a)', { message: 'hello, world' })
  const greeting = result.records[0].get(0)
  console.log(greeting)
  await tx.commit()
} finally {
  session.close(callback) // another session can be chained in callback
}

Java Driver

Example 4. Migrating from the 1.7 Java Driver to the 4.0 Java Driver.
Example code for the 4.0 Java Driver Example code for the 1.7 Java Driver
import org.neo4j.driver.Bookmark;
import org.neo4j.driver.Driver;
import org.neo4j.driver.Query;
import org.neo4j.driver.Result;
import org.neo4j.driver.Session;
import org.neo4j.driver.SessionConfig;
import org.neo4j.driver.Transaction;
import org.neo4j.driver.Values;
...

private final Driver driver;
...
public void printGreeting( String message, Bookmark bookmark )
{
   SessionConfig sessionConfig = SessionConfig.builder()
     .withDatabase( "neo4j" )
     .withDefaultAccessMode( AccessMode.WRITE )
     .withBookmarks( bookmark ).build();

   try ( Session session = driver.session( sessionConfig );
         Transaction transaction = session.beginTransaction() )
   {
      Query query = new Query( "CREATE (a:Greeting) SET a.message = $message RETURN a.message + ', from node ' + id(a)", Values.parameters( "message", message ) );

      Result result = transaction.run( query );
      String greeting = result.single().get( 0 ).asString();
      System.out.println( greeting );
      transaction.commit(); // commit immediately here
   }
}
import org.neo4j.driver.v1.AccessMode;
import org.neo4j.driver.v1.Driver;
import org.neo4j.driver.v1.Session;
import org.neo4j.driver.v1.Statement;
import org.neo4j.driver.v1.StatementResult;
import org.neo4j.driver.v1.Transaction;
import org.neo4j.driver.v1.Values;

...

private final Driver driver;
...
public void printGreeting( String message, String bookmark )
{




   try ( Session session = driver.session( AccessMode.WRITE, bookmark );
         Transaction transaction = session.beginTransaction() )
   {
       Statement query = new Statement( "CREATE (a:Greeting) SET a.message = $message RETURN a.message + ', from node ' + id(a)", Values.parameters( "message", message ) );


       StatementResult result = transaction.run( query );
       transaction.success(); // mark success, actually commit will happen in transaction.close()
       String greeting = result.single().get( 0 ).asString();
       System.out.println( greeting );
   }
}

Python Driver

  • Renamed Configuration max_retry_time is renamed to max_transaction_retry_time.

  • Renamed Configuration access_mode to default_access_mode and this is now a keyword argument.

  • Renamed Exception neo4j.exceptions.CypherError to neo4j.exceptions.Neo4jError.

  • Removed Exception neo4j.exceptions.ConnectionExpired.

  • Removed transaction.success flag.

  • neo4j.Record helper function Result.value(key=0, default=None), the key parameter have changed name from item to key.

  • neo4j.Record helper function Result.values(*keys), the *keys paramatere have changed name from *items to *keys.

  • neo4j.Record helper function Result.data(*keys), the *keys paramatere have changed name from *items to *keys.

  • neobolt is not a dependency any more.

  • neotime is not a dependency any more.

  • pytz is a dependency now.

  • Transaction.sync() has been removed. Use Result.consume() if the behaviour is to exhaust the result object.

  • Transaction.success has been removed.

  • Transaction.close() behaviour changed. Will now only perform rollback if no commit have been performed.

  • Session.sync() has been removed. Use Result.consume() if the behaviour is to exhaust the result object.

  • Session.detach() has been removed. Use Result.consume() if the behaviour is to exhaust the result object.

  • Session.next_bookmarks() has been removed.

  • Session.has_transaction() has been removed.

  • Session.closed() has been removed.

5. Back-pressure

Neo4j 4.0 introduces client-side back-pressure. The concept of client-side back-pressure is as follows; the client will communicate with the remote server regarding how much data it is able to process, and will only request additional data when it is ready to consume more.

The back-pressure concept is naturally compatible with Reactive programming. As a result, Reactive API support is added into all language drivers in 4.0 driver releases.

The Java driver Reactive API exposes a raw Publisher-Subscriber API, which is defined by reactive streams. When using Java driver’s Reactive API, it is anticipated that it is used with a reactive library, such as Project Reactor and/or RxJava.

For the .NET and JavaScript drivers Reactive API, the drivers are already supplied with reactive libraries. The built-in System.Reactive has been used in .NET driver. As for JavaScript driver, the popular RxJs library is adopted. These two libraries, as well as RxJava all belong to the same reactive framework ReactiveX.

To use the drivers’ Reactive API, a preliminary knowledge of reactive programming is necessary. Details of how to use Neo4j Reactive Driver API can be found in the Neo4j Driver Manual.

However, back-pressure is not only limited to the driver’s Reactive APIs. All other APIs, such as simple and async, by default have back-pressure enabled when handling query execution results.

Table 4. How back-pressure is implemented in the different language driver session APIs
Simple API Async API Reactive API

Java driver

Record buffer

Record buffer

Raw Publisher-Subscriber API

.NET driver

Record buffer

Record buffer

Record buffer

Javascript driver

Not applicable

Record buffer

Record buffer

5.1. Back-pressure with Bolt v4.0 and record buffer

The Neo4j 4.0 server and drivers implement Bolt v4.0. One of the main features introduced in this Bolt version is pulling query results (records) in batches. In previous Bolt versions, the complete result set is always pulled in one batch from a server to a driver. Bolt v4.0 enables us to pull these results in multiple batches where the size of each can be defined by a fetchSize. By default, the drivers use a fetchSize of 1000 records.

With the introduction of batching of records, drivers could now implement client-side back-pressure. For each result, the driver keeps a record buffer of unconsumed records. The buffer size is the same as fetchSize for each batch. The pulling of records from the server will be paused when the buffer is more than 70% full, and the record pulling is re-enabled once the buffer is less than 30% full. With the default fetchSize of 1000 records, record pulling is paused when more than 700 records are in the buffer and is resumed when the buffer drops below 300.

Example 5. Set default fetchSize on driver and alter the default value on session.
import org.neo4j.driver.AuthTokens;
import org.neo4j.driver.Config;
import org.neo4j.driver.Driver;
import org.neo4j.driver.GraphDatabase;
import org.neo4j.driver.Session;
import org.neo4j.driver.SessionConfig;
...

Config config = Config.builder().withFetchSize( 2000 ).build();
Driver driver = GraphDatabase.driver( uri, AuthTokens.basic( user, password ), config );

SessionConfig sessionConfig = SessionConfig.builder()
                                          .withDatabase( "neo4j" )
                                          .withFetchSize( 100 )
                                          .build();
try ( Session session = driver.session( sessionConfig ) ) {...}

5.2. Java driver Reactive API

The Java driver’s Reactive API exposes a very low level Publisher-Subscriber API. As a result, it will not perform any kind of back-pressure by default. Instead, we expect driver users to make use of a reactive framework to utilise back-pressure. Depending on the choice of the reactive framework, the framework may apply back-pressure by pausing the pulling of the data from a Neo4j server, or dropping data when there is too much to process.

6. Multiple databases

With the addition of multiple databases in 4.0, you can now specify which database to work with. When constructing a session you can specify in the session configuration which database the session is linked to. Queries will then be executed against that database for the duration of the session. Not specifying a database will result in the session being linked to the default database as specified in the server configuration, see Operations manual → The default database. When using 4.0 drivers with 4.0 Neo4j Servers, we always recommend to specify the database of each session explicitly.

Example 6. Selecting a database for a session.
import org.neo4j.driver.Driver;
import org.neo4j.driver.Session;
import org.neo4j.driver.SessionConfig;
...

try ( Session session = driver.session( SessionConfig.forDatabase( "neo4j" ) ) ) {...}

While managing multiple databases is primarily a feature for Neo4j Enterprise Edition, users of Neo4j Community Edition will still need to use the system database when carrying out administrative operations on the database. See Operations manual → The system database and Cypher manual → Administration for more information.

6.1. Bookmarks

Bookmarks are generally handled internally by the driver. Applications typically only need to work with bookmarks directly when chaining sessions.

When using bookmarks in a multiple database context, the base rule is that bookmarks can only be passed among sessions for the same database. This is because the bookmarks (and/or transactions) cannot cross multiple databases in Neo4j 4.0. There is one exception however, the bookmarks generated by the system database can be used with other databases.

Example 7. Using system bookmark with another database to ensure the updated system status.
import org.neo4j.driver.Bookmark;
import org.neo4j.driver.Driver;
import org.neo4j.driver.Result;
import org.neo4j.driver.Session;
import org.neo4j.driver.SessionConfig;
...

Bookmark sysBookmark;
try ( Session session = driver.session( SessionConfig.forDatabase( "system" ) ) )
{
   session.writeTransaction( tx -> {
       Result result = tx.run( "CREATE database foo" );
       return result.consume();
   } );
   sysBookmark = session.lastBookmark();
}

try ( Session session = driver.session( SessionConfig.builder().withDatabase( "foo" ).withBookmarks( sysBookmark ).build() ) )
{
   session.writeTransaction( tx -> {
       Result result = tx.run( "CREATE (n)" );
       return result.consume();
   } );
}

7. Configure SSL Policy for Bolt server and HTTPS server

Neo4j 3.5 always allows encrypted connections with the default configuration. In case no certificate is installed before a server starts, self-signed certificates will be automatically generated. However, in 4.0 the default encryption setting is off and Neo4j will no longer generate certificates when none is provided. As a result, Bolt server only allows plaintext connections, and HTTPS server is not enabled by default. The table below summarizes the default behaviour change between 3.5 and 4.0 regarding encryption and certificates.

Table 5. Encryption and certificates differences between 3.5 and 4.0 servers.
3.5 Neo4j Bolt Server 4.0 Neo4j Bolt Server 3.5 Neo4j HTTPS Server 4.0 Neo4j HTTPS Server

Server Enabled

Yes

Yes

Yes

No

Encryption on client connections

Optional

Not allowed

Always

Always

Certificates

Auto-generated self-signed certificates if not provided

None

Auto-generated self-signed certificates if not provided

None

Default Certificates Path

$neo4jHome/certificates

None

$neo4jHome/certificates

None

Default Certificate Names

neo4j.key

neo4j.cert

private.key

public.crt

neo4j.key

neo4j.cert

private.key

public.crt

In order to re-enable encryption in 4.0, we need to configure the SSL policy in the Neo4j config file. Given certificates named public.crt and private.key in folder $neo4jHome/certificates/bolt for Bolt server, and certificates with the same file names in folder $neo4jHome/certificates/https for HTTPS server. The example below shows how to turn encryption back on for the Bolt server and re-enable the HTTPS server.

Example 8. Turn encryption on for Bolt v4.0 server.
dbms.connector.bolt.enabled=true
dbms.connector.bolt.tls_level=OPTIONAL 			# allows both encrypted and unencrypted driver connections

dbms.ssl.policy.bolt.enabled=true
dbms.ssl.policy.bolt.base_directory=certificates/bolt
#dbms.ssl.policy.bolt.private_key=private.key 	# Optional if the file name is the same as the default.
#dbms.ssl.policy.bolt.public_certificate=public.crt 	# Optional if the file name is the same as the default.
Example 9. Enable the HTTPS 4.0 server.
dbms.connector.https.enabled=true

dbms.ssl.policy.https.enabled=true
dbms.ssl.policy.https.base_directory=certificates/https
#dbms.ssl.policy.https.private_key=private.key	# Optional if the file name is the same as the default.
#dbms.ssl.policy.https.public_certificate=public.crt	# Optional if the file name is the same as the default.

8. Additional URI Schemes

Since v4.0.1 of the Java and .NET drivers, and v4.0.2 of the JavaScript driver, you are able to configure the encryption and trust settings of the driver directly through the connection URI.

The neo4j+s and bolt+s schemes enable encryption and full certificate checks against the system’s local CA store. The neo4j+ssc and bolt+ssc schemes also enable encryption with no certificate checks, typically for use with self-signed certificates.

Table 6. Available URIs
URI Routing Description

neo4j

Yes

Unsecured

neo4j+s

Yes

Secured with full certificate

neo4j+ssc

Yes

Secured with self-signed certificate

bolt

No

Unsecured

bolt+s

No

Secured with full certificate

bolt+ssc

No

Secured with self-signed certificate

Using these new URI schemes is not compatible with configuring encryption and trust with the Configuration API. Otherwise, this does not effect the behaviour of the existing neo4j and bolt schemes.

For more information, see Driver Manual → Connection URIs.