Usage

Common operations

Clean

clean applies by default to the schema database. It will remove Neo4j-Migrations related nodes and relationships. If there is no schema database selected, it works on the optional target database. If this isn’t configured either, the users home database will be used.

The clean operation will search for

  • Migration chains (those are the nodes containing information about the applied migrations)

  • Any log from this Neo4j-Migrations

  • Any constraints created by Neo4j-Migrations

and will delete and drop them in that order. This is a destructive operation, so make sure not to apply it to your production database without thinking at least twice. It cannot be undone via Neo4j-Migrations.

The operation takes in a boolean parameter. When set to false, only the migration chain for the currently configured target database will be deleted. When set to true, all objects created by Neo4j-Migrations will be deleted.

Info

The info operations returns information about the context, the database, all applied and all pending applications.

Migrate / apply

The migrate command (or its underlying method apply in the Migrations Core API) does exactly that: It applies all locally resolved migrations to the target database and stores the chain of applied migrations in the schema database.

It returns the last applied version.

Validate

The validate operations resolves all local migrations and checks whether all have applied in the same order and in the same version to the configured database. A target database will validate as valid when all migrations have been applied in the right order and invalid in any cases where migrations are missing, have not been applied, applied in a different order or with a different checksum.

The validation result provides an additional operation needsRepair(). In case the result is invalid you might check if it needs repair. If not, you can just call the apply operation to turn the database into a valid state.

CLI

Please choose the version of Neo4j-Migrations-CLI fitting your operating system or target system as described in download. In the following we assume you downloaded and unzipped the architecture independent version. For that version to work, you need to have JDK 17 or higher installed:

Listing 1. Download and extraction of the JVM based version
java -version
curl -LO https://github.com/michael-simons/neo4j-migrations/releases/download/2.0.3/neo4j-migrations-2.0.3.zip
unzip neo4j-migrations-2.0.3.zip
cd neo4j-migrations-2.0.3
./bin/neo4j-migrations -V

Those commands should first print out your Java version, then download, extract and run Neo4j-Migrations-CLI to give you its version.

If you only deal with Cypher-based migrations and don’t have the need for any programmatic migrations, we provide a native binary for your platform, make sure to choose that. Its startup time is faster, and you don’t need to have a JVM installed.

All options and arguments

The CLI comes with a build-in help, accessible via neo4j-migrations -h or neo4j-migrations --help:

./bin/neo4j-migrations --help
Usage: neo4j-migrations [-hvV] [--autocrlf] [--validate-on-migrate] -p
                        [=<password>] [-p[=<password>]]... [-a=<address>]
                        [-d=<database>] [--impersonate=<impersonatedUser>]
                        [--schema-database=<schemaDatabase>]
                        [--transaction-mode=<transactionMode>] [-u=<user>]
                        [--location=<locationsToScan>]...
                        [--package=<packagesToScan>]... [COMMAND]
Migrates Neo4j databases.
  -a, --address=<address>   The address this migration should connect to. The
                              driver supports bolt, bolt+routing or neo4j as
                              schemes.
      --autocrlf            Automatically convert Windows line-endings (CRLF)
                              to LF when reading resource based migrations,
                              pretty much what the same Git option does during
                              checkin.
  -d, --database=<database> The database that should be migrated (Neo4j EE 4.0
                              +).
  -h, --help                Show this help message and exit.
      --impersonate=<impersonatedUser>
                            The name of a user to impersonate during migration
                              (Neo4j EE 4.4+).
      --location=<locationsToScan>
                            Location to scan. Repeat for multiple locations.
  -p, --password[=<password>]
                            The password of the user connecting to the database.
      --package=<packagesToScan>
                            Package to scan. Repeat for multiple packages.
      --schema-database=<schemaDatabase>
                            The database that should be used for storing
                              information about migrations (Neo4j EE 4.0+).
      --transaction-mode=<transactionMode>
                            The transaction mode to use.
  -u, --username=<user>     The login of the user connecting to the database.
  -v                        Log the configuration and a couple of other things.
  -V, --version             Print version information and exit.
      --validate-on-migrate Validating helps you verify that the migrations
                              applied to the database match the ones available
                              locally and is on by default.
Commands:
  clean           Removes Neo4j-Migration specific data from the selected
                    schema database
  help            Displays help information about the specified command
  info            Retrieves all applied and pending information, prints them
                    and exits.
  init            Creates a migration project inside the current folder.
  migrate, apply  Retrieves all pending migrations, verify and applies them.
  run             Resolves the specified migrations and applies them. Does not
                    record any metadata.
  show-catalog    Gets the local or remote catalog and prints it to standard
                    out in the given format.
  validate        Resolves all local migrations and validates the state of the
                    configured database with them.

If no values are given to either location or packages we check for a directory structure of neo4j/migrations inside the current working directory and use that as a default for location if such a structure exists.

The info command takes a mode option as an optional argument:

Usage: neo4j-migrations info [mode=<mode>]
Retrieves all applied and pending informations, prints them and exits.
      mode=<mode>   Controls how the information should be computed. Valid
                      options are COMPARE, LOCAL, REMOTE with COMPARE being the
                      default. COMPARE will always compare locally discovered
                      and remotely applied migrations, while the other options
                      just check what's there.

This means that we by default compare what has been discovered locally with what has been applied in the database: We check for missing or superfluous migrations and also compare checksums. At times, you might want to have just a quick look at what is in the database, without configuring a local filesystem. Use mode=remote in that case: We just look at what is in the database and assume everything is applied. Use mode=local to print out what has been discovered locally with the current settings and would be applied to an empty database.

neo4j-migrations looks in the current working directory for a properties file called .migration.properties which can contain all supported options. Use such a file to avoid repeating long command lines all the time. Use neo4j-migrations init to create a file with the default values. Any options passed to neo4j-migrations before the init command will also be store.

Output

Direct information coming from the CLI itself will always go to standard out. Information coming from core migrations will be locked with a timestamp on standard error. This allows for controlled redirection of different information.

Safe passwords in CI/CD usage

There are 4 ways to specify the password:

  1. interactive: Use --password without arguments and your shell will prompt you with a hidden prompt.

  2. direct: Use --password not-so-secret. The password will be visible in the shell history and in the process monitor.

  3. Via environment variable: Define an environment variable like MY_PASSWORD and use --password:env MY_PASSWORD. Note that the parameter is the name of the variable, not the resolved value.

  4. Via a file: Create a file in a safe space and add your password in a single line in that file and use --password:file path/to/your/passwordFile. The password will be read from this file.

The last two options are a safe choice in scripts or in a CI/CD environment.

Well-known Neo4j environment variables

Neo4j AuraDB provides .env files when creating new instances that look like this:

Listing 2. A Neo4j AuraDB .env file
# Wait 60 seconds before connecting using these details, or login to https://console.neo4j.io to validate the Aura Instance is available
NEO4J_URI=neo4j+s://xxxx.databases.neo4j.io
NEO4J_USERNAME=neo4j
NEO4J_PASSWORD=somepassword
AURA_INSTANCENAME=Instance01

Neo4j-Migrations will recognize those environment variables when present. If you didn’t specify a value for username, password or address and those variables are present and not empty, Neo4j-Migrations will use them.

Above file can be directly used in a command like this (on a *Nix-system):

set -o allexport (1)
(source ~/Downloads/credentials-xxx.env; neo4j-migrations info)
set +o allexport
1 Might not be needed in your shell

Enable autocompletion for Neo4j-Migrations in your shell

Neo4j-Migrations can generate a shell script providing autocompletion for its options in Bash, zsh and others. Here’s how to use it:

Listing 3. Generate autocompletion script
./bin/neo4j-migrations generate-completion > neo4j-migrations_completion.sh

The generated script neo4j-migrations_completion.sh can than be run via . neo4j-migrations_completion.sh or permanently installed by sourcing it in your ~/.bashrc or ~/.zshrc.

If you want to have autocompletion for Neo4j-Migrations just in your current shell use the following command

Listing 4. Add autocompletion to your current shell
source <(./bin/neo4j-migrations generate-completion)
Autocompletion for macOS is automatically installed when you use Homebrew.

Full example

Here’s an example that looks for migrations in a Java package, its subpackages and in a filesystem location for Cypher-based migrations. In this example we have exported the directory with our Java-based migrations like this: export CLASSPATH_PREFIX=~/Projects/neo4j-migrations/neo4j-migrations-core/target/test-classes/. Please adapt accordingly to your project and / or needs.

The example uses the info command to tell you which migrations have been applied and which not:

./bin/neo4j-migrations -uneo4j -psecret \
  --location file:$HOME/Desktop/foo \
  --package ac.simons.neo4j.migrations.core.test_migrations.changeset1 \
  --package ac.simons.neo4j.migrations.core.test_migrations.changeset2 \
  info

neo4j@localhost:7687 (Neo4j/4.4.0)
Database: neo4j

+---------+-----------------------------+--------+--------------+----+----------------+---------+--------------------------------------------------------------+
| Version | Description                 | Type   | Installed on | by | Execution time | State   | Source                                                       |
+---------+-----------------------------+--------+--------------+----+----------------+---------+--------------------------------------------------------------+
| 001     | FirstMigration              | JAVA   |              |    |                | PENDING | a.s.n.m.c.t.changeset1.V001__FirstMigration                  |
| 002     | AnotherMigration            | JAVA   |              |    |                | PENDING | a.s.n.m.c.t.changeset1.V002__AnotherMigration                |
| 023     | NichtsIstWieEsScheint       | JAVA   |              |    |                | PENDING | a.s.n.m.c.t.changeset2.V023__NichtsIstWieEsScheint           |
| 023.1   | NichtsIstWieEsScheintNeu    | JAVA   |              |    |                | PENDING | a.s.n.m.c.t.changeset2.V023_1__NichtsIstWieEsScheintNeu      |
| 023.1.1 | NichtsIstWieEsScheintNeuNeu | JAVA   |              |    |                | PENDING | a.s.n.m.c.t.changeset2.V023_1_1__NichtsIstWieEsScheintNeuNeu |
| 030     | Something based on a script | CYPHER |              |    |                | PENDING | V030__Something_based_on_a_script.cypher                     |
| 042     | The truth                   | CYPHER |              |    |                | PENDING | V042__The_truth.cypher                                       |
+---------+-----------------------------+--------+--------------+----+----------------+---------+--------------------------------------------------------------+

You can repeat both --package and --location parameter for fine-grained control. Use migrate to apply migrations:

./bin/neo4j-migrations -uneo4j -psecret \
  --location file:$HOME/Desktop/foo \
  --package ac.simons.neo4j.migrations.core.test_migrations.changeset1 \
  --package ac.simons.neo4j.migrations.core.test_migrations.changeset2 \
  migrate
[2022-05-31T11:25:29.894372000] Applied migration 001 ("FirstMigration").
[2022-05-31T11:25:29.985192000] Applied migration 002 ("AnotherMigration").
[2022-05-31T11:25:30.001006000] Applied migration 023 ("NichtsIstWieEsScheint").
[2022-05-31T11:25:30.016117000] Applied migration 023.1 ("NichtsIstWieEsScheintNeu").
[2022-05-31T11:25:30.032421000] Applied migration 023.1.1 ("NichtsIstWieEsScheintNeuNeu").
[2022-05-31T11:25:30.056182000] Applied migration 030 ("Something based on a script").
[2022-05-31T11:25:30.077719000] Applied migration 042 ("The truth").
Database migrated to version 042.

If we go back to the info example above and grab all migrations again, we find the following result:

./bin/neo4j-migrations -uneo4j -psecret \
  --location file:$HOME/Desktop/foo \
  --package ac.simons.neo4j.migrations.core.test_migrations.changeset1 \
  --package ac.simons.neo4j.migrations.core.test_migrations.changeset2 \
  info

Database: Neo4j/4.0.0@localhost:7687

+---------+-----------------------------+--------+-------------------------------+---------------+----------------+---------+--------------------------------------------------------------+
| Version | Description                 | Type   | Installed on                  | by            | Execution time | State   | Source                                                       |
+---------+-----------------------------+--------+-------------------------------+---------------+----------------+---------+--------------------------------------------------------------+
| 001     | FirstMigration              | JAVA   | 2021-12-14T12:16:43.577Z[UTC] | msimons/neo4j | PT0S           | APPLIED | a.s.n.m.c.t.changeset1.V001__FirstMigration                  |
| 002     | AnotherMigration            | JAVA   | 2021-12-14T12:16:43.876Z[UTC] | msimons/neo4j | PT0.032S       | APPLIED | a.s.n.m.c.t.changeset1.V002__AnotherMigration                |
| 023     | NichtsIstWieEsScheint       | JAVA   | 2021-12-14T12:16:43.993Z[UTC] | msimons/neo4j | PT0S           | APPLIED | a.s.n.m.c.t.changeset2.V023__NichtsIstWieEsScheint           |
| 023.1   | NichtsIstWieEsScheintNeu    | JAVA   | 2021-12-14T12:16:44.014Z[UTC] | msimons/neo4j | PT0S           | APPLIED | a.s.n.m.c.t.changeset2.V023_1__NichtsIstWieEsScheintNeu      |
| 023.1.1 | NichtsIstWieEsScheintNeuNeu | JAVA   | 2021-12-14T12:16:44.035Z[UTC] | msimons/neo4j | PT0S           | APPLIED | a.s.n.m.c.t.changeset2.V023_1_1__NichtsIstWieEsScheintNeuNeu |
| 030     | Something based on a script | CYPHER | 2021-12-14T12:16:44.093Z[UTC] | msimons/neo4j | PT0.033S       | APPLIED | V030__Something_based_on_a_script.cypher                     |
| 042     | The truth                   | CYPHER | 2021-12-14T12:16:44.127Z[UTC] | msimons/neo4j | PT0.011S       | APPLIED | V042__The truth.cypher                                       |
+---------+-----------------------------+--------+-------------------------------+---------------+----------------+---------+--------------------------------------------------------------+

Another migrate - this time with all packages - gives us the following output and result:

./bin/neo4j-migrations -uneo4j -psecret \
  --location file:$HOME/Desktop/foo \
  --package ac.simons.neo4j.migrations.core.test_migrations.changeset1 \
  --package ac.simons.neo4j.migrations.core.test_migrations.changeset2 \
  migrate
[2022-05-31T11:26:23.054169000] Skipping already applied migration 001 ("FirstMigration")
[2022-05-31T11:26:23.058779000] Skipping already applied migration 002 ("AnotherMigration")
[2022-05-31T11:26:23.059185000] Skipping already applied migration 023 ("NichtsIstWieEsScheint")
[2022-05-31T11:26:23.059504000] Skipping already applied migration 023.1 ("NichtsIstWieEsScheintNeu")
[2022-05-31T11:26:23.059793000] Skipping already applied migration 023.1.1 ("NichtsIstWieEsScheintNeuNeu")
[2022-05-31T11:26:23.060068000] Skipping already applied migration 030 ("Something based on a script")
[2022-05-31T11:26:23.060329000] Skipping already applied migration 042 ("The truth")
Database migrated to version 042.

The database will be now in a valid state:

./bin/neo4j-migrations -uneo4j -psecret \
  --location file:$HOME/Desktop/foo \
  --package ac.simons.neo4j.migrations.core.test_migrations.changeset1 \
  --package ac.simons.neo4j.migrations.core.test_migrations.changeset2 \
  validate
All resolved migrations have been applied to the default database.

Using the CLI as a script runner

The CLI can be used as a simple runner for migrations scripts as well. The only necessity is that all scripts have well-defined names according to the format described here:

./bin/neo4j-migrations -uneo4j -psecret \
  run \
  --migration file:`pwd`/../../../neo4j-migrations-core/src/test/resources/manual_resources/V000__Create_schema.cypher \
  --migration file:`pwd`/../../../neo4j-migrations-core/src/test/resources/manual_resources/V000__Create_graph.cypher \
  --migration file:`pwd`/../../../neo4j-migrations-core/src/test/resources/manual_resources/V000__Refactor_graph.xml
[2022-09-27T17:24:11.589274000] Applied 000 ("Create graph")
[2022-09-27T17:24:11.860457000] Applied 000 ("Refactor graph")
Applied 2 migration(s).
You can specify as many resources as you want. They will be applied in order. No checks will be done whether they have already been applied or not and no metadata will be recored.

A template for Java-based migrations

As stated above, this will work only with the JVM distribution. Follow those steps:

curl -LO https://github.com/michael-simons/neo4j-migrations/releases/download/2.0.3/neo4j-migrations-2.0.3.zip
unzip neo4j-migrations-2.0.3.zip
cd neo4j-migrations-2.0.3
mkdir -p my-migrations/some/migrations
cat <<EOT >> my-migrations/some/migrations/V001__MyFirstMigration.java
package some.migrations;

import ac.simons.neo4j.migrations.core.JavaBasedMigration;
import ac.simons.neo4j.migrations.core.MigrationContext;

import org.neo4j.driver.Driver;
import org.neo4j.driver.Session;

public class V001__MyFirstMigration implements JavaBasedMigration {

    @Override
    public void apply(MigrationContext context) {
        try (Session session = context.getSession()) {
        }
    }
}
EOT
javac -cp "lib/*" my-migrations/some/migrations/*
CLASSPATH_PREFIX=my-migrations ./bin/neo4j-migrations -v -uneo4j -psecret --package some.migrations info
We do add this here for completeness, but we do think that Java-based migrations makes most sense from inside your application, regardless whether it’s a Spring Boot, Quarkus or just a plain Java application. The CLI should be seen primarily as a script runner.

Core API

We publish the Java-API-Docs here: Neo4j Migrations (Core) 2.0.3 API. Follow the instructions for your favorite dependency management tool to get hold of the core API as described in download.

The classes you will be working with are ac.simons.neo4j.migrations.core.MigrationsConfig and its related builder and ac.simons.neo4j.migrations.core.Migrations and maybe ac.simons.neo4j.migrations.core.JavaBasedMigration in case you want to do programmatic refactorings.

Configuration and usage

Configuration is basically made up of two parts: Creating a driver instance that points to your database or cluster as described in the Connectivity section and an instance of MigrationsConfig. An instance of MigrationsConfig is created via a fluent-builder API. Putting everything together looks like this:

Listing 5. Creating an instance of Migrations based on a configuration object and the Java driver
Migrations migrations = new Migrations(
    MigrationsConfig.builder()
        .withPackagesToScan("some.migrations")
        .withLocationsToScan(
            "classpath:my/awesome/migrations",
            "file:/path/to/migration"
        )
        .build(),
    GraphDatabase.driver("bolt://localhost:7687", AuthTokens.basic("neo4j", "secret"))
);

migrations.apply(); (1)
1 Applies this migration object and migrates the database

In case anything goes wrong the API will throw a ac.simons.neo4j.migrations.core.MigrationsException. Of course your migrations will be recorded as a chain of applied migrations (as nodes with the label __Neo4jMigration) as well when you use the API directly.

The following operations are available:

info

Returns information about the context, the database, all applied and all pending applications

apply

Applies all discovered migrations

validate

Validates the database against the resolved migrations

clean

Cleans the selected schema database from every metadata created by this tool

The same operations are available in the CLI and Maven-Plugin. The corresponding starter for Spring Boot respectively the Quarkus extension will automatically run apply.

apply comes in a couple of overloads:

  • It will apply all discovered migrations when called without arguments or with a single boolean argument i

  • It will try to resolve URLS to supported migrations and apply them as is, without writing metadata when called with one or more URLs as argument. This method can also be ce called through the CLI (via the run command).

  • It will apply all refactorings in order when called with one or more instances of Refactoring. This method is only available in the Core API. Please read more about it here: [applying-refactorings-programmatically].

Running on the Java module-path

Neo4j-Migrations can be used on the Java module path. Make sure you require them in your module and export packages with Java-based migrations in case you’re using the latter. Resources on the classpath should be picked up automatically:

Listing 6. Using Neo4j-Migrations on the module path
module my.module {
    requires ac.simons.neo4j.migrations.core;

    exports my.module.java_based_migrations; (1)
}
1 Only needed when you actually have those

Spring-Boot-Starter

We provide a starter with automatic configuration for Spring Boot. Declare the following dependency in your Spring Boot application:

<dependency>
    <groupId>eu.michael-simons.neo4j</groupId>
    <artifactId>neo4j-migrations-spring-boot-starter</artifactId>
    <version>2.0.3</version>
</dependency>

Or follow the instructions for Gradle in download.

That starter itself depends on the Neo4j Java Driver. The driver is managed by Spring Boot since 2.4, and you can enjoy configuration support directly through Spring Boot. For Spring Boot versions prior to Spring Boot 2.4, please have a look at version 0.0.13 of this library.

Neo4j-Migrations will automatically look for migrations in classpath:neo4j/migrations and will fail if this location does not exist. It does not scan by default for Java-based migrations.

Here’s an example on how to configure the driver and the migrations:

Listing 7. Configure both the driver, disable the existence check for migration scripts and scan for Java-based migration
spring.neo4j.authentication.username=neo4j
spring.neo4j.authentication.password=secret
spring.neo4j.uri=bolt://localhost:7687

# Add configuration for your migrations, for example, additional packages to scan
org.neo4j.migrations.packages-to-scan=your.changesets, another.changeset

# Or disable the check if the location exists
org.neo4j.migrations.check-location=false

Have a look at Available configuration properties for all supported properties.

The starter will log some details about the product version and the database connected to. This can be disabled by setting the logger ac.simons.neo4j.migrations.core.Migrations.Startup to a level higher than INFO.

Usage with @DataNeo4jTest

If you want to use your migrations together with @DataNeo4jTest which is provided with Spring Boot out of the box, you have to manually import our autoconfiguration like this:

import ac.simons.neo4j.migrations.springframework.boot.autoconfigure.MigrationsAutoConfiguration;

import org.junit.jupiter.api.Test;
import org.neo4j.driver.Driver;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.autoconfigure.ImportAutoConfiguration;
import org.springframework.boot.test.autoconfigure.data.neo4j.DataNeo4jTest;

import org.springframework.test.context.DynamicPropertyRegistry;
import org.springframework.test.context.DynamicPropertySource;
import org.testcontainers.containers.Neo4jContainer;
import org.testcontainers.junit.jupiter.Container;
import org.testcontainers.junit.jupiter.Testcontainers;
import org.testcontainers.utility.TestcontainersConfiguration;

@Testcontainers(disabledWithoutDocker = true)
@DataNeo4jTest (1)
@ImportAutoConfiguration(MigrationsAutoConfiguration.class) (2)
public class UsingDataNeo4jTest {

    @Container
    private static Neo4jContainer<?> neo4j = new Neo4jContainer<>("neo4j:4.4")
        .withReuse(TestcontainersConfiguration.getInstance().environmentSupportsReuse()); (3)

    @DynamicPropertySource
    static void neo4jProperties(DynamicPropertyRegistry registry) { (4)

        registry.add("spring.neo4j.uri", neo4j::getBoltUrl);
        registry.add("spring.neo4j.authentication.username", () -> "neo4j");
        registry.add("spring.neo4j.authentication.password", neo4j::getAdminPassword);
    }

    @Test
    void yourTest(@Autowired Driver driver) {
        // Whatever is tested
    }
}
1 Use the dedicated Neo4j test slice
2 Import this auto-configuration (which is not part of Spring Boot)
3 Bring up a container to test against
4 Use DynamicPropertySource for configuring the test resources dynamically

Available configuration properties

The following configuration properties in the org.neo4j.migrations namespace are supported:

org.neo4j.migrations.check-location

Whether to check that migration scripts location exists.

  • Type: java.lang.Boolean

  • Default: true

org.neo4j.migrations.database

The database that should be migrated (Neo4j EE 4.0+ only). Leave null for using the default database.

  • Type: java.lang.String

  • Default: null

org.neo4j.migrations.schema-database

The database that should be used for storing information about migrations (Neo4j EE 4.0+ only). Leave null for using the default database.

  • Type: java.lang.String

  • Default: null

org.neo4j.migrations.impersonated-user

An alternative user to impersonate during migration. Might have higher privileges than the user connected, which will be dropped again after migration. Requires Neo4j EE 4.4+. Leave null for using the connected user.

  • Type: java.lang.String

  • Default: null

org.neo4j.migrations.enabled

Whether to enable Neo4j-Migrations or not.

  • Type: java.lang.Boolean

  • Default: true

org.neo4j.migrations.encoding

Encoding of Cypher migrations.

  • Type: java.nio.charset.Charset

  • Default: UTF-8

org.neo4j.migrations.installed-by

Username recorded as property by on the MIGRATED_TO relationship.

  • Type: java.lang.String

  • Default: System user

org.neo4j.migrations.locations-to-scan

Locations of migrations scripts.

  • Type: java.lang.String[]

  • Default: [classpath:neo4j/migrations]

org.neo4j.migrations.packages-to-scan

List of packages to scan for Java migrations.

  • Type: java.lang.String[]

  • Default: [] (an empty array)

org.neo4j.migrations.transaction-mode

The transaction mode in use (Defaults to "per migration", meaning one script is run in one transaction).

  • Type: TransactionMode

  • Default: PER_MIGRATION

org.neo4j.migrations.validate-on-migrate

Validating helps you verify that the migrations applied to the database match the ones available locally and is on by default.

  • Type: java.lang.Boolean

  • Default: true

org.neo4j.migrations.autocrlf

Automatically convert Windows line-endings (CRLF) to LF when reading resource based migrations, pretty much what the same Git option does during checkin.

  • Type: java.lang.Boolean

  • Default: false

Migrations can be disabled by setting org.neo4j.migrations.enabled to false.

Quarkus

We provide an extension with automatic configuration for Quarkus. Declare the following dependency in your Quarkus application:

<dependency>
    <groupId>eu.michael-simons.neo4j</groupId>
    <artifactId>neo4j-migrations-quarkus</artifactId>
    <version>2.0.3</version>
</dependency>

That extension itself depends on the Neo4j Java Driver and the corresponding Quarkus extension Quarkus-Neo4j and requires at least Quarkus 2.6. You don’t need to declare those dependencies, they are already transitive dependencies of this extension.

Neo4j-Migrations will automatically look for migrations in classpath:neo4j/migrations and will fail if this location does not exist. It does not scan by default for Java-based migrations.

Here’s an example on how to configure the driver and the migrations:

Listing 8. Configure both the driver and scan for Java-based migrations, too
quarkus.neo4j.uri=bolt://localhost:7687
quarkus.neo4j.authentication.username=neo4j
quarkus.neo4j.authentication.password=secret

org.neo4j.migrations.packages-to-scan=foo.bar

If you disable Neo4j-Migrations via org.neo4j.migrations.enabled we won’t apply Migrations at startup but the Migrations object will still be in the context to be used.

All other properties available for the Spring-Boot-Starter are available for the Quarkus extension, too. Their namespace is the same: org.neo4j.migrations. The module will also log some details about the product version and the database connected to. This can be disabled by setting the logger ac.simons.neo4j.migrations.core.Migrations.Startup to a level higher than INFO.

Build-time vs runtime config

org.neo4j.migrations.packages-to-scan and org.neo4j.migrations.locations-to-scan are build-time configuration options and cannot be changed during runtime. This allows for optimized images to be created: All migrations that are part of the classpath (both scripts and class based migrations) are discovered during image build-time already and are included in the image themselves (this applies to both native and JVM images).

While scripts in file system locations (all locations starting with file://) are still discovered during runtime and thus allows for scripts being added without recreating the application image, the location cannot be dynamically changed. If you need a dynamic, file:// based location, use org.neo4j.migrations.external-locations. This property is changeable during runtime and allows for one image being used in different deployments pointing to different external locations with scripts outside the classpath

An alternative approach to that is using the CLI in a sidecar container, pointing to the dynamic location and keep applying database migrations outside the application itself.

Dev Services integration

Neo4j-Migrations will appear as a tile in the Quarkus Dev UI under http://localhost:8080/q/dev/. It provides a list of migrations which can be used to clean the database or apply all migrations. The latter is handy when migrate at start is disabled or in case there are callbacks that might reset or recreate testdata.

Maven-Plugin

You can trigger Neo4j-Migrations from your build a Maven-Plugin. Please refer to the dedicated Maven-Plugin page for a detailed list of all goals and configuration option as well as the default lifecycle mapping of the plugin.

Configuration

Most of the time you will configure the following properties for the plugin:

Listing 9. Configuring the Maven-Plugin
<plugin>
    <groupId>eu.michael-simons.neo4j</groupId>
    <artifactId>neo4j-migrations-maven-plugin</artifactId>
    <version>2.0.3</version>
    <executions>
        <execution>
            <configuration>
                <user>neo4j</user>
                <password>secret</password>
                <address>bolt://localhost:${it-database-port}</address>
                <verbose>true</verbose>
            </configuration>
        </execution>
    </executions>
</plugin>

All goals provide those properties. By default, the plugin will look in neo4j/migrations for Cypher-based migrations. You can change that via locationsToScan inside the configuration element like this:

Listing 10. Changing the locations to scan for the Maven-Plugin
<locationsToScan>
    <locationToScan>file://${project.build.outputDirectory}/custom/path</locationToScan>
</locationsToScan>

Add multiple locationToScan elements for multiple locations to scan.

Goals

All goals as described in Common operations are supported.

The above list links to the corresponding Maven-Plugin page, please check those goals out for further details.

Defining and using catalogs

This chapter is more about conceptional usage or scenarios one can implement by using Catalog-based migrations. All scenarios can be executed with any of the previously explained APIS, being it the CLI, the Core API or within Spring Boot, Quarkus or Maven, except easily dumping a local or a remote catalog as XML or Cypher file.

Catalogs are a powerful mechanism to shape your database’s schema exactly the way you want it and this is only a small subset of possible scenarios that can be implemented.

For the rest of these steps we assume that you are using the CLI and used the init command to create a local directory structure holding connection data such as URL and credentials as well as your migrations:

Listing 11. Create migrations directory with credentials etc.
neo4j-migrations -a bolt://localhost:7687 -u neo4j -p secret init
tree -a

which will result in

.
├── .migrations.properties
└── neo4j
    └── migrations

2 directories, 1 file

All migrations we are going to work with will be stored in neo4j/migrations.

One sensible step before doing anything with the schema is to assert our local catalog meets the remote catalog as expected. In this example we assert toe remote catalog to be empty and we define our first migration like this:

Listing 12. V010__Assert_empty_schema.xml
<?xml version="1.0" encoding="UTF-8"?>
<migration xmlns="https://michael-simons.github.io/neo4j-migrations">
  <verify useCurrent="true"/> (1)
</migration>
1 useCurrent has been set to true to refer to the local catalog as defined in version 10, which is been empty, exactly what we expect

Applying this now via neo4j-migrations apply yields the following result:

[2022-06-01T15:13:39.997000000] Applied migration 010 ("Assert empty schema").
Database migrated to version 010.

Of course this step is only executed once, when this migration is applied. If we add another one too it, that verification does not happen again, as the 010 has been applied. Therefore, a verification step can be added to each catalog based migration:

Listing 13. V020__Create_person_name_unique.xml
<?xml version="1.0" encoding="UTF-8"?>
<migration xmlns="https://michael-simons.github.io/neo4j-migrations">
  <catalog>
    <constraints>
      <constraint name="person_name_unique" type="unique">
        <label>Person</label>
        <properties>
          <property>name</property>
        </properties>
      </constraint>
    </constraints>
  </catalog>

  <verify/>
  <create ref="person_name_unique"/>
</migration>

Note that we didn’t specify useCurrent here. This means verification should happen based on the local catalog prior to version 020. Applying this migration yields:

[2022-06-01T15:17:27.508000000] Skipping already applied migration 010 ("Assert empty schema")
[2022-06-01T15:17:27.771000000] Applied migration 020 ("Create person name unique").
Database migrated to version 020.

A day later you figure out that a unique constraint on a persons names isn’t the best of all ideas, and you decide to fix that. Assuming for sake of sanity that every person has a name, we replace that uniqueness with an existential constraint.

Existential constraints are a Neo4j enterprise feature, so we must cater for that as well and we define two different files for the next version:

Listing 14. V030__Fix_person_name_constraint_CE.xml
<?xml version="1.0" encoding="UTF-8"?>
<migration xmlns="https://michael-simons.github.io/neo4j-migrations">
  <?assume that edition is community ?>
  <drop item="person_name_unique"/>
</migration>

and for the enterprise edition, we can redefine the constraint like this:

Listing 15. V030__Fix_person_name_constraint_EE.xml
<?xml version="1.0" encoding="UTF-8"?>
<migration xmlns="https://michael-simons.github.io/neo4j-migrations">
  <?assume that edition is enterprise ?>
  <catalog>
    <constraints>
      <constraint name="person_name_unique" type="exists">
        <label>Person</label>
        <properties>
          <property>name</property>
        </properties>
      </constraint>
    </constraints>
  </catalog>

  <drop ref="person_name_unique"/>
  <create ref="person_name_unique"/>
</migration>

Note how we can refer to the constraint by ref in Listing 15 and how we must use item in Listing 14. The reason for that is that we refer to an older item only in the migration for community edition. We redefined the item in the script for the enterprise edition, so we might as well refer to it. In older Neo4j versions not supporting names for constraints Neo4j-Migrations will use the old definition to drop the item in question.

Applying the current state now yields

[2022-06-01T16:04:46.446188000] Skipping 030 ("Fix person name constraint CE") due to unmet preconditions:
// assume that edition is COMMUNITY
[2022-06-01T16:04:46.493400000] Skipping already applied migration 010 ("Assert empty schema")
[2022-06-01T16:04:46.496401000] Skipping already applied migration 020 ("Create person name unique")
[2022-06-01T16:04:46.659585000] Applied migration 030 ("Fix person name constraint EE").
Database migrated to version 030.

Assuming you some other enterprise stuff related items in the following listing:

Listing 16. V040__Additional_stuff.xml
<?xml version="1.0" encoding="UTF-8"?>
<migration xmlns="https://michael-simons.github.io/neo4j-migrations">
  <?assume that edition is enterprise ?>
  <catalog>
    <indexes/>
    <constraints>
      <constraint name="liked_day" type="exists">
        <type>LIKED</type>
        <properties>
          <property>day</property>
        </properties>
      </constraint>
      <constraint name="person_keys" type="key">
        <label>Person</label>
        <properties>
          <property>firstname</property>
          <property>surname</property>
        </properties>
      </constraint>
    </constraints>
  </catalog>

  <create ref="liked_day"/>
  <create ref="person_keys"/>
</migration>

To get some information about your database, you can inspect the remote catalog:

Listing 17. Inspecting the remote catalog
neo4j-migrations show-catalog

and it will print the catalog in XML:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<migration xmlns="https://michael-simons.github.io/neo4j-migrations">
    <catalog>
        <indexes/>
        <constraints>
            <constraint name="liked_day" type="unique">
                <label>LIKED</label>
                <properties>
                    <property>day</property>
                </properties>
            </constraint>
            <constraint name="person_keys" type="key">
                <label>Person</label>
                <properties>
                    <property>firstname</property>
                    <property>surname</property>
                </properties>
            </constraint>
            <constraint name="person_name_unique" type="exists">
                <label>Person</label>
                <properties>
                    <property>name</property>
                </properties>
            </constraint>
        </constraints>
    </catalog>
</migration>

You can also get the catalog as Cypher with

neo4j-migrations show-catalog format=CYPHER version=4.4

yielding

CREATE CONSTRAINT person_keys IF NOT EXISTS FOR (n:Person) REQUIRE (n.firstname, n.surname) IS NODE KEY;
CREATE CONSTRAINT liked_day IF NOT EXISTS FOR ()-[r:LIKED]-() REQUIRE r.day IS NOT NULL;
CREATE CONSTRAINT person_name_unique IF NOT EXISTS FOR (n:Person) REQUIRE n.name IS NOT NULL;

Changing the version number to an older version will give the correct syntax, too:

> neo4j-migrations show-catalog format=CYPHER version=3.5
CREATE CONSTRAINT ON (n:Person) ASSERT (n.firstname, n.surname) IS NODE KEY;
CREATE CONSTRAINT ON ()-[r:LIKED]-() ASSERT exists(r.day);
CREATE CONSTRAINT ON (n:Person) ASSERT exists(n.name);

After all, you decide it’s best not to stick with any constraint on the persons name and also drop your experiments. You could use <apply /> to make your database look exactly like your catalog. But that would include all previously defined items, too.

Therefore, you need to reset the catalog as shown in the following listing:

Listing 18. V050__A_new_start.xml
<?xml version="1.0" encoding="UTF-8"?>
<migration xmlns="https://michael-simons.github.io/neo4j-migrations">
  <catalog reset="true">
    <constraints>
      <constraint name="unique_person_id" type="unique">
        <label>Person</label>
        <properties>
          <property>id</property>
        </properties>
      </constraint>
    </constraints>
  </catalog>
  <apply/>
</migration>

followed by a final verification:

Listing 19. V060__Assert_final_state.xml
<?xml version="1.0" encoding="UTF-8"?>
<migration xmlns="https://michael-simons.github.io/neo4j-migrations">

  <verify allowEquivalent="false"/>
</migration>

Run the following commands to see the outcome:

neo4j-migrations apply

applies everything:

[2022-06-01T19:16:20.058218000] Skipping 030 ("Fix person name constraint CE") due to unmet preconditions:
// assume that edition is COMMUNITY
[2022-06-01T19:16:20.223937000] Skipping already applied migration 010 ("Assert empty schema")
[2022-06-01T19:16:20.225464000] Skipping already applied migration 020 ("Create person name unique")
[2022-06-01T19:16:20.225748000] Skipping already applied migration 030 ("Fix person name constraint EE")
[2022-06-01T19:16:20.226022000] Skipping already applied migration 040 ("Additional stuff")
[2022-06-01T19:16:20.501686000] Applied migration 050 ("A new start").
[2022-06-01T19:16:20.551983000] Applied migration 060 ("Assert final state").
Database migrated to version 060.
neo4j-migrations show-catalog format=CYPHER

presents the remote catalog as

CREATE CONSTRAINT unique_person_id IF NOT EXISTS FOR (n:Person) REQUIRE n.id IS UNIQUE;

and so does the local catalog

neo4j-migrations show-catalog format=CYPHER mode=LOCAL 2&>/dev/null

The redirect is included here so that log messages on stderr are skipped (the message about one migration skipped due to unmet preconditions).