Volume mounts and persistent volumes

Neo4j Helm chart uses volume mounts and persistent volumes to manage the storage of data and other Neo4j files.

Volume mounts

A volume mount is part of a Kubernetes Pod spec that describes how and where a volume is mounted within a container.

The Neo4j Helm chart creates the following volume mounts:

  • backups mounted at /backups

  • data mounted at /data

  • import mounted at /import

  • licenses mounted at /licenses

  • logs mounted at /logs

  • metrics mounted at /metrics (Neo4j Community Edition does not generate metrics.)

It is also possible to specify a plugins volume mount (mounted at /plugins), but this is not created by the default Helm chart. For more information, see Add plugins using a plugins volume.

Persistent volumes

PersistentVolume (PV) is a storage resource in the Kubernetes cluster that has a lifecycle independent of any individual pod that uses the PV.
PersistentVolumeClaim (PVC) is a request for a storage resource by a user. PVCs consume PV resources. For more information about what PVs are and how they work, see the Kubernetes official documentation.

The type of PV used and its configuration can have a significant effect on the performance of Neo4j. Some PV types are not suitable for use with Neo4j at all.

The volume type used for the data volume mount is particularly important. Neo4j supports the following PV types for the data volume mount:

  • persistentVolumeClaim

  • hostPath when using Docker Desktop [1].

Neo4j data volume mount does not support azureFile and nfs.

awsElasticBlockStore, azureDisk, gcePersistentDisk are now deprecated volume types in Kubernetes and their use is no longer supported by the Neo4j Helm chart. If you currently use one of these volume types, consult your Kubernetes vendor’s documentation on migrating to Container Storage Interface (CSI) driver-based storage.

For volume mounts other than the data volume mount, generally, all PV types are presumed to work.

hostPath, local, and emptyDir types are expected to perform well, provided suitable underlying storage, such as SSD, is used. However, these volume types have operational limitations and are not recommended.

It is also not recommended to use an HDD or cloud storage, such as AWS S3 mounted as a drive.

Mapping volume mounts to persistent volumes

By default, the Neo4j Helm chart uses a single PV, named data, to support volume mounts.

The volume used for each volume mount can be changed by modifying the volumes.<volume name> object in the Helm chart values.

The Neo4j Helm chart volumes object supports different modes, such as dynamic, share, defaultStorageClass, volume, selector, and volumeClaimTemplate. You can also set a label on creation for the volumes with mode dynamic, defaultStorageClass, selector, and volumeClaimTemplate, which can be used to filter the PVs that are used for the volume mount.

mode: share

Description

The volume mount shares the underlying volume from one of the other volume objects.

Example

The logs volume mount uses the data volume (this is the default behavior).

volumes:
  logs:
    mode: "share"
    share:
      name: "data"

mode: defaultStorageClass

Description

The volume mount is backed by a PV that Kubernetes dynamically provisions using the default StorageClass.

Example

A dynamically provisioned data volume with a size of 10Gi.

volumes:
  data:
    labels:
        data: "true"
    mode: "defaultStorageClass"
    defaultStorageClass:
      requests:
        storage: 10Gi

For the data volume, if requests.storage is not set, defaultStorageClass defaults to a 10Gi volume. For all other volumes, defaultStorageClass.requests.storage must be set explicitly when using defaultStorageClass mode.

mode: volume

Description

A complete Kubernetes volume object can be specified for the volume mount. Generally, volumes specified in this way have to be manually provisioned.

volume can be any valid Kubernetes volume type. This mode is typically used to mount a pre-existing Persistent Volume Claim (PVC).

For details on how to specify volume objects, see the Kubernetes documentation.

Set file permissions on mounted volumes

The Neo4j Helm chart supports an additional field not present in normal Kubernetes volume objects: setOwnerAndGroupWritableFilePermissions: true|false. If set to true, an initContainer will be run to modify the file permissions of the mounted volume, so that the contents can be written and read by the Neo4j process. This is to help with certain volume implementations that are not aware of the SecurityContext set on pods using them.

Example - reference an existing PersistentVolume

The backups volume mount is backed by the specified PVC. When this method is used, the persistentVolumeClaim object must already exist.

volumes:
  backups:
    mode: volume
    volume:
      persistentVolumeClaim:
        claimName: my-neo4j-pvc

mode: selector

Description

The volume to use is chosen from the existing PVs based on the provided selector object and a PVC that is dynamically generated.

If no matching PVs exist, the Neo4j pod will be unable to start. To match, a PV must have the specified StorageClass, match the label selectorTemplate, and have sufficient storage capacity to meet the requested storage amount.

Example

The data volume is chosen from the available volumes with the neo4j storage class and the label developer: alice.

volumes:
  import:
    labels:
        import: "true"
    mode: selector
    selector:
      storageClassName: "neo4j"
      requests:
        storage: 128Gi
      selectorTemplate:
        matchLabels:
          developer: "alice"

For the data volume, if requests.storage is not set, selector defaults to a 100Gi volume. For all other volumes, selector.requests.storage must be set explicitly when using selector mode.

mode: volumeClaimTemplate

Description

A complete Kubernetes volumeClaimTemplate object is specified for the volume mount. Volumes specified in this way are dynamically provisioned.

Example - provision Neo4j storage using a volume claim template

The data volume uses a dynamically provisioned PVC from the default storage class.

volumes:
  data:
    labels:
        data: "true"
    mode: volumeClaimTemplate
    volumeClaimTemplate:
      storageClassName: "default"
      accessModes:
        - ReadWriteOnce
      resources:
        requests:
          storage: 10Gi

In all cases, do not forget to set the mode field when customizing the volumes object. If not set, the default mode is used, regardless of the other properties set on the volume object.

Provision persistent volumes with Neo4j Helm chart

Provision persistent volumes dynamically

With the Neo4j Helm chart, you can provision a PV dynamically using the default or a custom StorageClass. To see a list of available storage classes in your Kubernetes cluster, run the following command:

kubectl get storageclass

Provision a PV using defaultStorageClass

Using the default StorageClass of the running Kubernetes cluster is the quickest way to spin up and run Neo4j for simple tests, handling small amounts of data. However, it is not recommended for large amounts of data, as it may lead to performance issues.

Example: Deploy Neo4j using defaultStorageClass

The following example shows how to deploy a Neo4j server with a dynamically provisioned PV that uses the default StorageClass.

  1. Create a file default-storage-class-values.yaml that configures the data volume to use the default StorageClass and a storage size 100Gi:

    storage-class-values.yaml
    volumes:
      data:
        mode: "defaultStorageClass"
        defaultStorageClass:
          requests:
            storage: 100Gi
  2. Install a single Neo4j server:

    helm install standalone-with-default-storage-class neo4j -f default-storage-class-values.yaml

Provision persistent volumes manually

Optionally, the Helm chart can use manually created disks for Neo4j storage. This installation option has more steps than using dynamic volumes, but it does provide more control over how disks are provisioned.

The instructions for the manual provisioning of PVs vary according to the type of PV being used and the underlying infrastructure. In general, there are two steps:

  1. Create the disk/volume to be used for storage in the underlying infrastructure. For example:

    • If using a csi volume — create the Persistent Disk using the cloud provider CLI or console.

    • If using a hostPath volume — on the host node, create the path (directory).

  2. Create a PV in Kubernetes that references the underlying resource created in step 1.

    1. Ensure that the created PV’s app label matches the name of the Neo4j Helm release.

    2. Ensure that the created PV’s capacity.storage matches the storage available on the underlying infrastructure.

If no suitable PV or PVC exists, the Neo4j pod will not start.

The Neo4j StatefulSet can select a persistent volume to use based on its labels. A Neo4j Helm release uses only manually provisioned PVs that have:

  • storageClassName that uses the provisioner kubernetes.io/no-provisioner.

  • An app label — set in their metadata, which matches the name of the neo4j.name value of the Helm installation.

  • Sufficient storage capacity — the PV capacity must be greater than or equal to the value of volumes.data.selector.requests.storage set for the Neo4j Helm release (default is 100Gi).

The neo4j/neo4j-persistent-volume Helm chart provides a convenient way to provision the persistent volume.

Example: Deploy Neo4j using a selector volume

The following example shows how to deploy Neo4j using a selector volume.

  1. Create a file persistent-volume-selector.yaml that configures the data volume to use a selector:

    storage-class-values.yaml
    neo4j:
      name: volume-selector
    volumes:
      data:
        mode: selector
        selector:
          storageClassName: "manual"
          accessModes:
            - ReadWriteOnce
          requests:
            storage: 10Gi
  2. Export environment variables to be used by the commands:

    export RELEASE_NAME=volume-selector
    export GCP_ZONE="$(gcloud config get compute/zone)"
    export GCP_PROJECT="$(gcloud config get project)"
  3. Create the disks to be used by the persistent volume:

    gcloud compute disks create --size 10Gi --type pd-ssd "${RELEASE_NAME}"
  4. Use the neo4j/neo4j-persistent-volume chart to configure the persistent volume. This command will create a persistent volume and a manual storage class that uses the kubernetes.io/no-provisioner provisioner.

    helm install "${RELEASE_NAME}"-disk neo4j/neo4j-persistent-volume \
           --set neo4j.name="${RELEASE_NAME}" \
           --set data.driver=pd.csi.storage.gke.io \
           --set data.storageClassName="manual" \
           --set data.reclaimPolicy="Delete" \
           --set data.createPvc=false \
           --set data.createStorageClass=true \
           --set data.volumeHandle="projects/${GCP_PROJECT}/zones/${GCP_ZONE}/disks/${RELEASE_NAME}" \
           --set data.capacity.storage=10Gi
  5. Now install Neo4j using the persistent-volume-selector.yaml created earlier:

    helm install "${RELEASE_NAME}" neo4j/neo4j -f persistent-volume-selector.yaml
  6. Clean up the helm installation and disks created for the example:

    helm uninstall ${RELEASE_NAME} ${RELEASE_NAME}-disk
    kubectl delete pvc data-${RELEASE_NAME}-0
    gcloud compute disks delete ${RELEASE_NAME} --quiet

The EBS CSI Driver addon is required to provision EBS disks in EKS clusters. You can run the command kubectl get daemonset ebs-csi-node -n kube-system to check if it is installed See the AWS Documentation for instructions on installing the driver.

  1. Create a file persistent-volume-selector.yaml that configures the data volume to use a selector:

    storage-class-values.yaml
    neo4j:
      name: volume-selector
    volumes:
      data:
        mode: selector
        selector:
          storageClassName: "manual"
          accessModes:
            - ReadWriteOnce
          requests:
            storage: 10Gi
  2. Export environment variables to be used by the commands:

    readonly RELEASE_NAME=volume-selector
    readonly AWS_ZONE={availability zone of EKS cluster}
  3. Create the disks to be used by the persistent volume:

    export volumeId=$(aws ec2 create-volume \
                        --availability-zone="${AWS_ZONE}" \
                        --size=10 \
                        --volume-type=gp3 \
                        --tag-specifications 'ResourceType=volume,Tags=[{Key=volume,Value='"${RELEASE_NAME}"'}]' \
                        --no-cli-pager \
                        --output text \
                        --query VolumeId)
  4. Use the neo4j/neo4j-persistent-volume chart to configure the persistent volume. This command will create a persistent volume and a manual storage class that uses the kubernetes.io/no-provisioner provisioner.

    helm install "${RELEASE_NAME}"-disk neo4j-persistent-volume \
        --set neo4j.name="${RELEASE_NAME}" \
        --set data.driver=ebs.csi.aws.com \
        --set data.reclaimPolicy="Delete" \
        --set data.createPvc=false \
        --set data.createStorageClass=true \
        --set data.volumeHandle="${volumeId}" \
        --set data.capacity.storage=10Gi
  5. Now install Neo4j using the persistent-volume-selector.yaml created earlier:

    helm install "${RELEASE_NAME}" neo4j/neo4j -f persistent-volume-selector.yaml
  6. Clean up the helm installation and disks created for the example:

    helm uninstall ${RELEASE_NAME} ${RELEASE_NAME}-disk
        kubectl delete pvc data-${RELEASE_NAME}-0
        aws ec2 delete-volume --volume-id ${volumeId}
  1. Create a file persistent-volume-selector.yaml that configures the data volume to use a selector:

    storage-class-values.yaml
    neo4j:
      name: volume-selector
    volumes:
      data:
        mode: selector
        selector:
          storageClassName: "manual"
          accessModes:
            - ReadWriteOnce
          requests:
            storage: 10Gi
  2. Export environment variables to be used by the commands:

    readonly AKS_CLUSTER_NAME={AKS Cluster name}
    readonly AZ_RESOURCE_GROUP={Resource group of cluster}
    readonly AZ_LOCATION={Location of cluster}
  3. Create the disks to be used by the persistent volume:

    export node_resource_group=$(az aks show --resource-group "${AZ_RESOURCE_GROUP}" --name "${AKS_CLUSTER_NAME}" --query nodeResourceGroup -o tsv)
    export disk_id=$(az disk create --name "${RELEASE_NAME}" --size-gb "10" --max-shares 1 --resource-group "${node_resource_group}" --location ${AZ_LOCATION} --output tsv --query id)
  4. Use the neo4j/neo4j-persistent-volume chart to configure the persistent volume. This command will create a persistent volume and a manual storage class that uses the kubernetes.io/no-provisioner provisioner.

    helm install "${RELEASE_NAME}"-disk neo4j-persistent-volume \
        --set neo4j.name="${RELEASE_NAME}" \
        --set data.driver=disk.csi.azure.com \
        --set data.storageClassName="manual" \
        --set data.reclaimPolicy="Delete" \
        --set data.createPvc=false \
        --set data.createStorageClass=true \
        --set data.volumeHandle="${disk_id}" \
        --set data.capacity.storage=10Gi
  5. Now install Neo4j using the persistent-volume-selector.yaml created earlier:

    helm install "${RELEASE_NAME}" neo4j/neo4j -f persistent-volume-selector.yaml
  6. Clean up the helm installation and disks created for the example:

    helm uninstall ${RELEASE_NAME} ${RELEASE_NAME}-disk
    kubectl delete pvc data-${RELEASE_NAME}-0
    az disk delete --name ${RELEASE_NAME} -y

Provision a PVC for Neo4j Storage

An alternative method for manual provisioning is to use a manually provisioned PVC. This is supported by the Neo4j Helm chart using the volume mode.

The neo4j/neo4j-persistent-volume Helm chart can be used to create a PV and PVC for a manually provisioned disk. A full example can be found in the Neo4j GitHub repository. For example, to use a pre-existing PVC called my-neo4j-pvc set these values:

volumes:
  data:
    mode: "volume"
    volume:
      persistentVolumeClaim:
        claimName: my-neo4j-pvc

Reuse a persistent volume

After uninstalling the Neo4j Helm chart, both the PVC and the PV remain and can be reused by a new install of the Helm chart. If you delete the PVC, the PV moves into a Released status and will not be reusable.

To be able to reuse the PV by a new install of the Neo4j Helm chart, remove its connection to the previous PVC:

  1. Edit the PV by running the following command:

    kubectl edit pv <pv-name>
  2. Remove the section spec.claimRef.
    The PV goes back to the Available status and can be reused by a new install of the Neo4j Helm chart.

The performance of Neo4j is very dependent on the latency, IOPS capacity, and throughput of the storage it is using. For the best performance of Neo4j, use the best available disks (e.g., SSD) and set IOPS throttling/quotas to high values. For some cloud providers, IOPS throttling is proportional to the size of the volume. In these cases, the best performance is achieved by setting the size of the volume based on the desired IOPS rather than the amount required for data storage.

Glossary

allocator

A component in the cluster that allocates databases to servers according to the topology constraints specified and an allocation strategy.

asynchronous replication

Asynchronous replication is used by secondary copies to poll for new transactions, which means they cannot be guaranteed to have received the most recent transactions. This enables efficient scale-out of read-performance.

Aura instance

A fully-managed DBMS represented by a single instance ID, that is running in the Neo4j Aura cloud.

auto-commit transaction

An automatically committed transaction that contains a single query.

Bolt protocol

Bolt is a protocol used for interaction between Neo4j instances and drivers.

bookmark

A marker the client can request from the cluster to ensure that it is able to read its own writes so that the application’s state is consistent and only databases that have a copy of the bookmark are permitted to respond.

category (Bloom)

A category is based on a node label and is defined in a Perspective as a way of visually distinguishing nodes with the same label(s).

causal consistency

All servers in a cluster agree on the order in which transactions take place. The position of a server on the causal chain can be guaranteed using a bookmark.

cluster

A Neo4j DBMS that spans multiple servers working together to increase fault tolerance and/or read scalability. Databases on a cluster may be configured to replicate across servers in the cluster thus achieving read scalability or high availability.

client application

Software that interacts with a Neo4j server.

commit

A commit is the successful completion of a transaction, which ensures durability of any changes made. For more details, visit Operations Manual → Transaction management.

composite database

Composite databases are the means to access partitioned graph data with a single Cypher query.

constraint

Constraints are sets of data modeling rules that ensure the data is consistent and reliable.

Cypher®

Neo4j’s graph query language.

data model

A data model defines how information is organized in a database. A good data model will make querying and understanding your data easier. In Neo4j, the data models have a graph structure.

database

A database is a container used by the DBMS to manage and store graph data. The physical structure of data is controlled by the database.

database vs graph

Databases are the physical containers of graph data. Graphs are the logical structure of data in Neo4j.

Database Management System

Database Management System, or DBMS, capable of managing multiple databases. A DBMS may run on a single server, or span several servers configured as a cluster.

database schema

The prescribed property existence and datatypes for nodes and relationships.

deallocate

An act of removing a database from a server or a server from a cluster without loss of data or reduced fault tolerance.

degree (of a node)

The number of relationships of a specific node; loops are counted twice.

disaster recovery

A manual intervention to restore availability of a cluster, or databases within a cluster.

driver

A software library that provides access to Neo4j from a particular programming language.

election

In the event that the Raft leader becomes unresponsive, followers automatically trigger an election and vote for a new leader.

entity

A node or a relationship.

expression (Cypher)

A component of a Cypher query which produces values. It may be used in projections, as a predicate, or when setting properties on graph elements.

fabric

Fabric is the architectural design of a unified system that provides a single access point to local or distributed graph data.

fault tolerance

A guarantee that a cluster can maintain a database’s persistence and availability in the event of one or more servers failing.

follower

A primary copy of a database acting as a follower, receives and acknowledges synchronous writes from the leader.

Generative AI (GenAI)

A type of artificial intelligence (AI) system that generates text, images, or other media in response to prompts.

graph

A logical representation of a set of nodes where some pairs are connected by relationships.

index

Data structure that improves read performance of a database.

knowledge graph

A specific type of graph that has an organizing principle so that a user (or a computer system) can reason about the underlying data. The organizing principle provides an additional layer of structure that adds context to support knowledge discovery.

label

Marks a node as a member of a named and indexed subset. A node may be assigned zero or more labels.

leader

A single primary copy of a database is designated as the leader. It receives all write transactions from clients and replicates writes synchronously to followers and asynchronously to secondary copies of the database.

main database

In terms of Neo4j Enterprise Studio, the database(s) containing the user’s data. Can exist in the same Neo4j deployment as the tool asset database.

motif

A description of a specific pattern within a graph.

node

A node represents an entity or discrete object in your graph data model. Nodes can be connected by relationships, hold data in properties, and are classified by labels.

operator

A symbol representing a mathematical or logical operation.

parameter

Named value provided when running a Cypher statement.

path

A sequence of nodes and the relationships connecting them, that does not contain duplicate relationships. Several paths can match a pattern.

pattern

A specific arrangement of nodes and relationships that can be matched in a graph. A pattern follows a motif.

perspective (Bloom)

A Perspective defines a certain business view or domain that can be found in the target Neo4j graph. A single Neo4j graph can be viewed through different Perspectives, each tailored for a different business purpose.

primary

A copy of the database that is able to process write transactions and is eligible to be elected as a leader. It participates in fault tolerant writes as it is part of the majority required to acknowledge and commit write transactions.

primary vs secondary

In a cluster, databases can operate in either primary or secondary mode. Primary databases are able to process write and read transactions, ensuring fault tolerance. Secondary databases are replicated asynchronously from primaries, and their main purpose is to provide read scaling within the cluster.

project (Aura)

An isolated environment in the unified Aura console that contains its own database instances, configurations, and resources. Preceded by tenant in the classic Aura console.

property

Properties are key-value pairs that are used for storing data on nodes and relationships.

query (Cypher)

A statement that retrieves or writes information to a database.

Raft group

A group of servers that are participating in hosting a particular database in primary mode.

Raft group member

A server that is participating in a Raft group. A server can be a member of one or more groups.

Raft log

A shared log between all Raft group members that is guaranteed to be consistently updated and viewed by those members. The log contains both database data and operational state of the Raft group.

Raft protocol

The networking mechanism that enables a database to replicate its data across multiple servers to give high availability for accessing the data and high durability to the data stored.

read scaling

Distributing query load by creating additional database copies hosted in secondary mode (read-only).

relationship

A relationship represents a connection between nodes in your graph data model. Relationships connect a source node to a target node, hold data in properties, and are classified by type.

secondary

An asynchronously replicated copy of the database that provides read scaling within the cluster.

seed

A seed is a database dump or a full backup used to create a database on a cluster. This is sometimes called seeding.

server

A physical machine, a virtual machine, or a container running an instance of Neo4j. Servers can be standalone or part of a cluster.

session

A causally linked sequence of transactions.

session consistency

An alternative name for Neo4j’s causal consistency.

standalone

A single server running Neo4j and not part of a cluster.

synchronous replication

Synchronous replication requires the leader primary to replicate a transaction and block the commit until a quorum of the follower primaries acknowledges that the transaction is successfully replicated. Once the transaction is replicated, the commit is allowed to proceed. This ensures data durability and consistency within the cluster.

system database

A database used by Neo4j to store system information.

tenant (Aura)

An isolated environment in the classic Aura console that contains its own database instances, configurations, and resources. Replaced by project in the unified Aura console.

tool asset database

In terms of Neo4j Enterprise Studio, the database where tools' assets are stored. This can be in the same Neo4j deployment as the main database(s) or in a separate deployment.

topology

A configuration that describes how the copies of a database should be spread across the servers in a cluster, see primary mode and secondary mode.

transaction

A transaction comprises a unit of work performed against a database. It is treated in a coherent and reliable way, independent of other transactions. Transactions comply with the ACID consistency model (atomic, consistent, isolated, and durable).


1. Not recommended because of inconsistencies in Docker Desktop handling of hostPath volumes.