Aggregate a database backup chain

Command

The aggregate command turns a a chain of backup artifacts into a single full backup artifact.

backup chain aggregation

The benefits of aggregating a backup chain are notably:

  • Reduces the size of backup artifacts in a given backup folder.

  • Keeps the recovery time objective (RTO) low by generating a single backup artifact ready to be restored. As part of the aggregation, transactions contained in the differential backups are applied to the store contained in the full backup artifact. This operation is called recovery and can be costly.

  • Reduces the risk of losing chain’s links.

  • Some commands, like consistency check, cannot run against a backup chain or a full backup that is not recovered. Aggregating produces an artifact they can work against.

Syntax

neo4j-admin backup aggregate [-h] [--expand-commands]
                                      [--verbose] [--keep-old-backup[=true|false]]
                                      [--parallel-recovery[=true|false]]
                                      [--additional-config=<file>] --from-path=<path>
                                      [--temp-path=<path>] [<database>]

In Neo4j 2025.01, the name neo4j-admin database aggregate-backup is deprecated. It is recommended to use the new command name, neo4j-admin backup aggregate.

Description

Aggregates a chain of backup artifacts into a single artifact.

Parameters

Table 1. neo4j-admin backup aggregate parameters
Parameter Description

[<database>]

Name of the database for which to aggregate the artifacts. Can contain * and ? for globbing.

Options

Table 2. neo4j-admin backup aggregate options
Option Description Default

--additional-config=<file>[1]

Configuration file with additional configuration.

--expand-commands

Allow command expansion in config value evaluation.

--from-path=<path>

Accepts either a path to a single artifact file or a folder containing backup artifacts.

When a file is supplied, the <database> parameter should be omitted. It is possible to aggregate backup artifacts from AWS S3 buckets, Google Cloud storage buckets, and Azure buckets using the appropriate URI as the path.

-h, --help

Show this help message and exit.

--keep-old-backup[=true|false]

If set to true, the old backup chain is not removed.

false

--parallel-recovery[=true|false]

Allow multiple threads to apply pulled transactions to a backup in parallel. For some databases and workloads, this may reduce aggregate times significantly. Note: this is an EXPERIMENTAL option. Consult Neo4j support before use.

false

--temp-path=<path>

Provide a path to a temporary empty directory for storing backup files until the command is completed. The files will be deleted once the command is finished.

--verbose

Enable verbose output.

The --from-path=<path> option can also load backup artifacts from AWS S3 buckets, Google Cloud storage buckets, and Azure buckets. For more information, see Aggregating a backup chain located in a cloud storage.

The --temp-path option addresses potential issues related to disk space when performing backup-related commands, especially when cloud storage is involved.

If --temp-path is not set, a temporary directory is created inside the directory specified by the --from-path option.

If you don’t provide the --from-path option or if your provided path points to a cloud storage bucket, a temporary folder is created inside the current working directory for Neo4j. This fallback option can cause issues because the local filesystem (or the partition where Neo4j is installed) may not have enough free disk to accommodate the intermediate computation.

Therefore, it is strongly recommended to provide a --temp-path option.

Examples

Aggregating a backup chain located in a given folder

The following is an example of how to perform aggregation of a set of backups located in a given folder for the neo4j database:

bin/neo4j-admin backup aggregate --from-path=/mnt/backups/ neo4j

The command first looks inside the /mnt/backups/ directory for a backup chain for the database neo4j. If found, it is then aggregated into a single backup artifact.

Aggregating a backup chain identified using a given backup file

The following is an example of how to perform aggregation of a set of backups identified using a given backup file for the neo4j database:

bin/neo4j-admin backup aggregate --from-path=/mnt/backups/neo4j-2022-10-18T13-00-07.backup

The command checks the /mnt/backups/ directory for a backup chain including the file neo4j-2022-10-18T13-00-07.backup, for the database neo4j. If found, it is then aggregated into a single backup artifact.

Aggregating a backup chain located in a cloud storage

In Neo4j 2025.03, new cloud integration settings are introduced to provide better support for deployment and management in cloud ecosystems. For details, refer to Configuration settings → Cloud storage integration settings.

The following examples show how to perform aggregation of a set of backups located in a cloud storage.

Neo4j uses the AWS SDK v2 to call the APIs on AWS using AWS URLs. Alternatively, you can override the endpoints so that the AWS SDK can communicate with alternative storage systems, such as Ceph, Minio, or LocalStack, using the system variables aws.endpointUrls3, aws.endpointUrlS3, or aws.endpointUrl, or the environments variables AWS_ENDPOINT_URL_S3 or AWS_ENDPOINT_URL.

  1. Install the AWS CLI by following the instructions in the AWS official documentation — Install the AWS CLI version 2.

  2. Create an S3 bucket and a directory to store the backup files using the AWS CLI:

    aws s3 mb --region=us-east-1 s3://myBucket
    aws s3api put-object --bucket myBucket --key myDirectory/

    For more information on how to create a bucket and use the AWS CLI, see the AWS official documentation — Use Amazon S3 with the AWS CLI and Use high-level (s3) commands with the AWS CLI.

  3. Verify that the ~/.aws/config file is correct by running the following command:

    cat ~/.aws/config

    The output should look like this:

    [default]
    region=us-east-1
  4. Configure the access to your AWS S3 bucket by setting the aws_access_key_id and aws_secret_access_key in the ~/.aws/credentials file and, if needed, using a bucket policy. For example:

    1. Use aws configure set aws_access_key_id aws_secret_access_key command to set your IAM credentials from AWS and verify that the ~/.aws/credentials is correct:

      cat ~/.aws/credentials

      The output should look like this:

      [default]
      aws_access_key_id=this.is.secret
      aws_secret_access_key=this.is.super.secret
    2. Additionally, you can use a resource-based policy to grant access permissions to your S3 bucket and the objects in it. Create a policy document with the following content and attach it to the bucket. Note that both resource entries are important to be able to download and upload files.

      {
          "Version": "2012-10-17",
          "Id": "Neo4jBackupAggregatePolicy",
          "Statement": [
              {
                  "Sid": "Neo4jBackupAggregateStatement",
                  "Effect": "Allow",
                  "Action": [
                      "s3:ListBucket",
                      "s3:GetObject",
                      "s3:PutObject",
                      "s3:DeleteObject"
                  ],
                  "Resource": [
                      "arn:aws:s3:::myBucket/*",
                      "arn:aws:s3:::myBucket"
                  ]
              }
          ]
      }
  5. Then, use the following command to aggregate the backup chain located in a given folder in your AWS S3 bucket. The example assumes that you have a backup chain located in the myBucket/myDirectory folder identifiable by the file myBackup.backup:

    bin/neo4j-admin backup aggregate --from-path=s3://myBucket/myDirectory/myBackup.backup mydatabase
  1. Ensure you have a Google account and a project created in the Google Cloud Platform (GCP).

    1. Install the gcloud CLI by following the instructions in the Google official documentation — Install the gcloud CLI.

    2. Create a service account and a service account key using Google official documentation — Create service accounts and Creating and managing service account keys.

    3. Download the JSON key file for the service account.

    4. Set the GOOGLE_APPLICATION_CREDENTIALS and GOOGLE_CLOUD_PROJECT environment variables to the path of the JSON key file and the project ID, respectively:

      export GOOGLE_APPLICATION_CREDENTIALS="/path/to/keyfile.json"
      export GOOGLE_CLOUD_PROJECT=YOUR_PROJECT_ID
    5. Authenticate the gcloud CLI with the e-mail address of the service account you have created, the path to the JSON key file, and the project ID:

      gcloud auth activate-service-account [email protected] --key-file=$GOOGLE_APPLICATION_CREDENTIALS --project=$GOOGLE_CLOUD_PROJECT

      For more information, see the Google official documentation — gcloud auth activate-service-account.

    6. Create a bucket in the Google Cloud Storage using Google official documentation — Create buckets.

    7. Verify that the bucket is created by running the following command:

      gcloud storage ls

      The output should list the created bucket.

  2. Then, use the following command to aggregate the backup chain located in a given folder in your Google storage bucket. The example assumes that you have a backup chain located in the myBucket/myDirectory folder identifiable by the file myBackup.backup:

    bin/neo4j-admin backup aggregate --from-path=gs://myBucket/myDirectory/myBackup.backup mydatabase
  1. Ensure you have an Azure account, an Azure storage account, and a blob container.

    1. You can create a storage account using the Azure portal.
      For more information, see the Azure official documentation on Create a storage account.

    2. Create a blob container in the Azure portal.
      For more information, see the Azure official documentation on Quickstart: Upload, download, and list blobs with the Azure portal.

  2. Install the Azure CLI by following the instructions in the Azure official documentation — Azure official documentation.

  3. Authenticate the neo4j or neo4j-admin process against Azure using the default Azure credentials.
    See the Azure official documentation on default Azure credentials for more information.

    az login

    Then you should be ready to use Azure URLs in either neo4j or neo4j-admin.

  4. To validate that you have access to the container with your login credentials, run the following commands:

    # Upload a file:
    az storage blob upload --file someLocalFile  --account-name accountName - --container someContainer --name remoteFileName  --auth-mode login
    
    # Download the file
    az storage blob download  --account-name accountName --container someContainer --name remoteFileName --file downloadedFile --auth-mode login
    
    # List container files
    az storage blob list  --account-name someContainer --container someContainer  --auth-mode login
  5. Then, use the following command to aggregate the backup chain located in a given folder in your Azure blob storage container. The example assumes that you have a backup chain located with a myStorageAccount/myContainer/myDirectory folder identifiable by the file myBackup.backup:

    bin/neo4j-admin backup aggregate --from-path=azb://myStorageAccount/myContainer/myDirectory/myBackup.backup mydatabase

Aggregating an unrecovered full backup

To check if a full backup is in a recovered state, run the neo4j-admin backup inspect command:

bin/neo4j-admin backup inspect /mnt/backups/neo4j-2025-09-10T08-14-56.backup

The output will include the RECOVERED column. If it shows false, the backup is not yet recovered:

|                                                 FILE | DATABASE |                          DATABASE ID |          TIME (UTC) | FULL | COMPRESSED | LOWEST TX | HIGHEST TX | STORE ID HASH | RECOVERED |
| file:///mnt/backups/neo4j-2025-09-10T08-14-56.backup |    neo4j | 4e98fab8-6ae6-46cc-84c0-61ee8a8db60a | 2025-09-10T08:14:56 | true |       true |         1 |          3 |    -116350002 |     false |

If your full backup is not recovered, aggregating it will perform recovery. Run the following command:

bin/neo4j-admin backup aggregate --from-path=/mnt/backups/neo4j-2025-09-10T08-14-56.backup --keep-old-backup=true

This command will:

  • Generate a new, recovered backup alongside the existing one.

  • Preserve the original backup file (due to --keep-old-backup=true).

After aggregation, you can run consistency checker against the recovered backup and/or restore it.

Glossary

allocator

A component in the cluster that allocates databases to servers according to the topology constraints specified and an allocation strategy.

asynchronous replication

Asynchronous replication is used by secondary copies to poll for new transactions, which means they cannot be guaranteed to have received the most recent transactions. This enables efficient scale-out of read-performance.

Aura instance

A fully-managed DBMS represented by a single instance ID, that is running in the Neo4j Aura cloud.

auto-commit transaction

An automatically committed transaction that contains a single query.

Bolt protocol

Bolt is a protocol used for interaction between Neo4j instances and drivers.

bookmark

A marker the client can request from the cluster to ensure that it is able to read its own writes so that the application’s state is consistent and only databases that have a copy of the bookmark are permitted to respond.

category (Bloom)

A category is based on a node label and is defined in a Perspective as a way of visually distinguishing nodes with the same label(s).

causal consistency

All servers in a cluster agree on the order in which transactions take place. The position of a server on the causal chain can be guaranteed using a bookmark.

cluster

A Neo4j DBMS that spans multiple servers working together to increase fault tolerance and/or read scalability. Databases on a cluster may be configured to replicate across servers in the cluster thus achieving read scalability or high availability.

client application

Software that interacts with a Neo4j server.

commit

A commit is the successful completion of a transaction, which ensures durability of any changes made. For more details, visit Operations Manual → Transaction management.

composite database

Composite databases are the means to access partitioned graph data with a single Cypher query.

constraint

Constraints are sets of data modeling rules that ensure the data is consistent and reliable.

Cypher®

Neo4j’s graph query language.

data model

A data model defines how information is organized in a database. A good data model will make querying and understanding your data easier. In Neo4j, the data models have a graph structure.

database

A database is a container used by the DBMS to manage and store graph data. The physical structure of data is controlled by the database.

database vs graph

Databases are the physical containers of graph data. Graphs are the logical structure of data in Neo4j.

Database Management System

Database Management System, or DBMS, capable of managing multiple databases. A DBMS may run on a single server, or span several servers configured as a cluster.

database schema

The prescribed property existence and datatypes for nodes and relationships.

deallocate

An act of removing a database from a server or a server from a cluster without loss of data or reduced fault tolerance.

degree (of a node)

The number of relationships of a specific node; loops are counted twice.

disaster recovery

A manual intervention to restore availability of a cluster, or databases within a cluster.

driver

A software library that provides access to Neo4j from a particular programming language.

election

In the event that the Raft leader becomes unresponsive, followers automatically trigger an election and vote for a new leader.

entity

A node or a relationship.

expression (Cypher)

A component of a Cypher query which produces values. It may be used in projections, as a predicate, or when setting properties on graph elements.

fabric

Fabric is the architectural design of a unified system that provides a single access point to local or distributed graph data.

fault tolerance

A guarantee that a cluster can maintain a database’s persistence and availability in the event of one or more servers failing.

follower

A primary copy of a database acting as a follower, receives and acknowledges synchronous writes from the leader.

Generative AI (GenAI)

A type of artificial intelligence (AI) system that generates text, images, or other media in response to prompts.

graph

A logical representation of a set of nodes where some pairs are connected by relationships.

index

Data structure that improves read performance of a database.

knowledge graph

A specific type of graph that has an organizing principle so that a user (or a computer system) can reason about the underlying data. The organizing principle provides an additional layer of structure that adds context to support knowledge discovery.

label

Marks a node as a member of a named and indexed subset. A node may be assigned zero or more labels.

leader

A single primary copy of a database is designated as the leader. It receives all write transactions from clients and replicates writes synchronously to followers and asynchronously to secondary copies of the database.

main database

In terms of Neo4j Enterprise Studio, the database(s) containing the user’s data. Can exist in the same Neo4j deployment as the tool asset database.

motif

A description of a specific pattern within a graph.

node

A node represents an entity or discrete object in your graph data model. Nodes can be connected by relationships, hold data in properties, and are classified by labels.

operator

A symbol representing a mathematical or logical operation.

parameter

Named value provided when running a Cypher statement.

path

A sequence of nodes and the relationships connecting them, that does not contain duplicate relationships. Several paths can match a pattern.

pattern

A specific arrangement of nodes and relationships that can be matched in a graph. A pattern follows a motif.

perspective (Bloom)

A Perspective defines a certain business view or domain that can be found in the target Neo4j graph. A single Neo4j graph can be viewed through different Perspectives, each tailored for a different business purpose.

primary

A copy of the database that is able to process write transactions and is eligible to be elected as a leader. It participates in fault tolerant writes as it is part of the majority required to acknowledge and commit write transactions.

primary vs secondary

In a cluster, databases can operate in either primary or secondary mode. Primary databases are able to process write and read transactions, ensuring fault tolerance. Secondary databases are replicated asynchronously from primaries, and their main purpose is to provide read scaling within the cluster.

project (Aura)

An isolated environment in the unified Aura console that contains its own database instances, configurations, and resources. Preceded by tenant in the classic Aura console.

property

Properties are key-value pairs that are used for storing data on nodes and relationships.

query (Cypher)

A statement that retrieves or writes information to a database.

Raft group

A group of servers that are participating in hosting a particular database in primary mode.

Raft group member

A server that is participating in a Raft group. A server can be a member of one or more groups.

Raft log

A shared log between all Raft group members that is guaranteed to be consistently updated and viewed by those members. The log contains both database data and operational state of the Raft group.

Raft protocol

The networking mechanism that enables a database to replicate its data across multiple servers to give high availability for accessing the data and high durability to the data stored.

read scaling

Distributing query load by creating additional database copies hosted in secondary mode (read-only).

relationship

A relationship represents a connection between nodes in your graph data model. Relationships connect a source node to a target node, hold data in properties, and are classified by type.

secondary

An asynchronously replicated copy of the database that provides read scaling within the cluster.

seed

A seed is a database dump or a full backup used to create a database on a cluster. This is sometimes called seeding.

server

A physical machine, a virtual machine, or a container running an instance of Neo4j. Servers can be standalone or part of a cluster.

session

A causally linked sequence of transactions.

session consistency

An alternative name for Neo4j’s causal consistency.

standalone

A single server running Neo4j and not part of a cluster.

synchronous replication

Synchronous replication requires the leader primary to replicate a transaction and block the commit until a quorum of the follower primaries acknowledges that the transaction is successfully replicated. Once the transaction is replicated, the commit is allowed to proceed. This ensures data durability and consistency within the cluster.

system database

A database used by Neo4j to store system information.

tenant (Aura)

An isolated environment in the classic Aura console that contains its own database instances, configurations, and resources. Replaced by project in the unified Aura console.

tool asset database

In terms of Neo4j Enterprise Studio, the database where tools' assets are stored. This can be in the same Neo4j deployment as the main database(s) or in a separate deployment.

topology

A configuration that describes how the copies of a database should be spread across the servers in a cluster, see primary mode and secondary mode.

transaction

A transaction comprises a unit of work performed against a database. It is treated in a coherent and reliable way, independent of other transactions. Transactions comply with the ACID consistency model (atomic, consistent, isolated, and durable).