While we've worked hard to make updates to newer versions of Neo4j as pleasant as possible, there are a few things to be aware of, particularly when switching to a major revision.
Neo4j 3.1 Upgrade FAQWhat is in 3.1?
- New Clustering Technology based on state-of-the-art Raft protocol:
- Cluster-aware stack with built-in load balancing
- Causal consistency enabling applications to decide on cost/consistency tradeoff at query time
- New Security Foundation ensuring that authenticated users and applications can perform only authorized actions on data graph stored in Neo4j:
- Role-Based Access Control for built-in native and federated users (via LDAP connector)
- Sub-graph authorization achievable via User-Defined Procedures
- Query listing and termination using :queries command in Neo4j Browser
- Security event logging
- Database Kernel and Operations Advances:
- New execution guard to terminate long-running, end-user queries
- Ability to reallocate space from deleted records while the cluster is online
- New command line tools: cypher-shell and improved neo4j-admin
- Schema Viewer: Displays the graph model at the current point in time
- IBM Power8 CAPI Flash Support: Enables ultra-large in-memory graphs
Where are the Release Notes?Release notes for all Neo4j releases can be found here: http://neo4j.com/release-notes/
Is 3.1 an API-Breaking Release?No. Minor releases are a place to deprecate features and functionality, which includes public APIs. However, we will not remove deprecated functionality until a major release. Note: upgrading from a version prior to Neo4j 3.0 means that the 3.0 Upgrade FAQs apply, and this would then be a potential breaking release. If you are upgrading from a 2.x release, see the 3.0 Upgrade FAQs below.
A few things you should know about 3.1:
- While Causal Clustering is introduced, HA still works as well.
- Causal Clustering is available for server only, not embedded at this time.
- To keep local security credentials in sync across a cluster, use rsync as described in this KB (Neo4j support portal access only): https://support.neo4j.com/hc/en-us/articles/236060187-Cluster-synchronization-of-neo4j-native-security
- The neo4j.conf and neo4j-wrapper.conf are merged in 3.1. neo4j-wrapper.conf is deprecated.
- See this KB (public access) for example script: https://neo4j.com/developer/kb/manually-merging-neo4jwrapperconf-into-neo4jconf-in-neo4j-31/
- neo4j-shell is deprecated in favor of the new cypher-shell
Migrate configurationWhen upgrading to Neo4j 3.1 from Neo4j 3.0, the conf/neo4j.conf and conf/neo4j-wrapper.conf files have now been merged into one. The configuration settings have not changed, so it is simply a matter of copying the contents of conf/neo4j-wrapper.conf, pasting into the end of the conf/neo4j.conf file, and then removing conf/neo4j-wrapper.conf. Use the script in the KB entitled Manually Merging neo4j-wrapper.conf into neo4j.conf in Neo4j 3.1 to automate this merge. Note: conf/neo4j-wrapper.conf will be deprecated, and not removed yet, so will still work for backward compatibility.
When migrating to Neo4j 3.1 from a release prior to 3.0 please note that most configuration settings have been renamed to support a more consistent naming scheme. Also note that neo4j.properties and neo4j-server.properties have been merged into a single neo4j.conf file. Since many settings have been changed between Neo4j 2.x and Neo4j 3.0, it is advisable to use the config-migrator utility to migrate the config files and settings for you. The config-migrator can be found in the bin/tools directory of Neo4j 3.0, and can be invoked with a command like: java -jar config-migrator.jar path/to/neo4j2.3 path/to/neo4j3.0. Take note of any warnings printed, and manually review the edited config files produced. The warnings will alert you to deprecated settings, or settings it did not know about (spelling mistakes, etc) that require manual intervention. Also note that the following configuration settings have been deprecated between Neo4j 3.0 and 3.1: dbms.index_sampling.buffer_size
Where in the Documentation Can I Learn More about Upgrades?The Upgrade section of the Neo4j Operations Manual can be found here, and also covers upgrades specific to 3.1. The store upgrade happens by starting your Neo4j data store (after making the backup) with Neo4j 3.1 and the configuration dbms.allow_format_migration=true enabled in the configuration file conf/neo4j.conf
Are there any Backwards-Compatibility Options?Yes. Cypher lets you specify what version of the Cypher compiler you would like to use. See Operations Manual for details. So it is possible to run your 2.x queries without changing them. Of course you will eventually want to upgrade your queries to take advantage of the new features in Cypher and Neo4j 3.1.
Also of note is the new Causal Clustering architecture introduced in 3.1. Note that 3.1 will continue to support the previous Master/Slave clustering architecture available in 3.0 and prior versions. You can upgrade to 3.1, and save the cluster changes for a later date to simplify the process.
Is the Store Upgrade Reversible?No, once your data store is upgraded, it cannot be rolled back. For this reason, Neo4j makes a backup of each store file as part of the upgrade to 3.1. (And as an extra precaution, it doesn’t hurt to take a second backup before you start the upgrade process. Just make sure you have enough disk space to accommodate this.) Make sure you have at least double the size of your Neo4j store available as free space on your disk, before you upgrade. After the upgrade is successful, those backup files will be removed.
How Long Will the Upgrade Take?When you initiate the upgrade of your database to 3.1 for the first time, Neo4j will run through the store files and update them to the new format. How much time this takes depends on the size of your data store. We recommend you try this out in a test environment first, to get an idea of the timing, and help you plan your upgrade. Don’t forget though that you will first need to test your application to make sure it works! Chances are that that you may have to make some changes. How much development time required to update your code will depend on your application. Applications using Neo4j’s native Java APIs will be affected more than others.
When Should I Upgrade?If you are planning a new release of your app, and adding new features, then we recommend you upgrade now: since you are likely to save a lot of time with the new features. Even if you aren’t adding features, we suggest you start planning your upgrade soon, in the coming months, as the 2.3 and prior releases will now be going into maintenance mode. As far as when to schedule your upgrade? Upgrading an already-running application to 3.1 just for the sake of it may not yield huge benefits right now. One good reason to start planning an upgrade besides new features is the upcoming 3.1 release. Neo4j 3.1 is planned to be a minor/API-compatible release of Neo4j 3.x, focused primarily on scalability and security. Neo4j 3.1 is expected mid-late 2016, and like 3.0, will require a store upgrade. Customers not in a hurry to upgrade, and/or seeking to minimize their Neo4j upgrade overhead in 2016, might want to start developing on 3.0 and go live with 3.1. As always, we encourage you to contact Support to discuss your actual situation. We are happy to offer advice as to when to time your upgrade.
Are Rolling Upgrades Supported?No. Rolling upgrades let you upgrade a cluster while it is running. While we do our best to support rolling upgrades between patch releases – the scope and nature of the changes made in major/minor versions is such that rolling upgrades are not possible. However, for Enterprise customers looking for guidance on minimizing the impact of this upgrade on your system, refer to the article Upgrading a Neo4j Cluster with Minimal Downtime
What Version Can I Upgrade From?Neo4j 3.1 supports direct upgrades from 2.0 and newer. It is generally recommended to upgrade from the most recent patch, as that is the tested upgrade path from pre-3.1 releases to 3.1. Previous versions, such as 1.9, have to be upgraded to 2.x first.
How Long Can I Continue Using 3.0?3.0 will continue to be supported for at least 12 months following its release (April 26, 2016) per the terms of your Support Agreement. Any fixes deemed applicable to the 3.0 release will be released in a 3.0.x release. As of the time of this writing, the latest stable 3.0.x release is 3.0.7.
Neo4j 3.0 Upgrade FAQ
What is in 3.0?Neo4j 3.0 substantially improves installation, application development, and operational management. On disk, Neo4j has a reorganized directory layout which is easier to understand and work with; take note of the new locations and names. Application development has been vastly simplified and optimized, featuring a single, coherent, high-speed programming interface called Bolt. With Cypher’s new ability to mix queries and calls to Java Stored Procedures, even Java developers will love working over-the-wire.
Where are the Release Notes?Release notes for all Neo4j releases can be found here: http://neo4j.com/release-notes/
Is 3.0 an API-Breaking Release?Yes. Major releases are the time when we remove features that were marked deprecated in earlier releases. Further: to benefit from the upgrade to Neo4j 3.0, you will need to change your code to take advantage of the new features, such as the new language drivers over the Bolt protocol.
A few things you should know about 3.0:
- To use Neo4j 3.0, please install Java 8, preferably the OpenJDK 8, Oracle Java 8 or IBM Java 8. Install at least the full Java Runtime Environment (JRE) with command-line tools.
- The store format for Neo4j Enterprise has changed considerably, as is the case with a major release. See section on Is the Store Upgrade Reversible for more details.
- Lucene has been upgraded from 3.6.2 to 5.4.0. As a result, all indexes will be rebuilt on upgrade, which means the upgrade will take some additional time.
- The directory structure and configuration parameters have been refactored. Please make yourself familiar with the new location and names of files.
Migrate configurationWhen migrating to Neo4j 3.0 from a previous release, please note that most configuration settings have been renamed to support a more consistent naming scheme. Also note that neo4j.properties and neo4j-server.properties have been merged into a single neo4j.conf file. Since many settings have been changed between Neo4j 2.x and Neo4j 3.0, it is advisable to use the config-migrator utility to migrate the config files and settings for you. The config-migrator can be found in the bin/tools directory of Neo4j 3.0, and can be invoked with a command like:
java -jar config-migrator.jar path/to/neo4j2.3 path/to/neo4j3.0. Take note of any warnings printed, and manually review the edited config files produced. The warnings will alert you to deprecated settings, or settings it did not know about (spelling mistakes, etc) that require manual intervention. Also note that the following configuration settings have been removed between Neo4j 2.3 and 3.0:
consistency_check_execution_order consistency_check_report_file logging.threshold_for_rotation neo_store org.neo4j.server.database.location org.neo4j.server.db.tuning.properties org.neo4j.server.http.log.config org.neo4j.server.http.unsafe.content_log.enabled org.neo4j.server.properties org.neo4j.server.webadmin.rrdb.location org.neo4j.server.webserver.https.keystore.location org.neo4j.server.webserver.statistics relationship_grab_size store.internal_log.location wrapper.user