Checkpointing and log pruning

Checkpointing is the process of flushing all pending updates from volatile memory to non-volatile data storage. This action is crucial to limit the number of transactions that need to be replayed during the recovery process, particularly to minimize the time required for recovery after an improper shutdown of the database or a crash.

Independent of the presence of checkpoints, database operations remain secure, as any transactions that have not been confirmed to have their modifications persisted to storage will be replayed upon the next database startup. However, this assurance is contingent upon the availability of the collection of changes comprising these transactions, which is maintained in the transaction logs.

Maintaining a long list of unapplied transactions (due to infrequent checkpoints) leads to the accumulation of transaction logs, as they are essential for recovery purposes. Checkpointing involves the inclusion of a special Checkpointing entry in the transaction log, marking the last transaction at which checkpointing occurred. This entry serves the purpose of identifying transaction logs that are no longer necessary, as all the transactions they contain have been securely stored in the storage files.

The process of eliminating transaction logs that are no longer required for recovery is known as pruning. Pruning is reliant on checkpointing. Checkpointing determines which logs can be pruned and determines the occurrence of pruning, as the absence of a checkpoint implies that the set of transaction log files available for pruning cannot have changed. Consequently, pruning is triggered whenever checkpointing occurs.

For information on checkpointing and log pruning in Neo4j 4.4, refer to Configuration settings → Checkpoint settings, Performance → Checkpoint IOPS limit, and Transaction log → Log pruning respectively.

Configure the checkpointing policy

The checkpointing policy, which is the driving event for log pruning is configured by db.checkpoint. Depending on your needs, the checkpoint can run on a periodic basis, which is the default, when a certain amount of data has been written to the transaction log, or continuously.

Table 1. Available checkpointing policies
Policy	Description
`PERIODIC`	Default This policy checks every 10 minutes whether there are changes pending flushing and if so, it performs a checkpoint and subsequently triggers a log prune. The periodic policy is specified by the `db.checkpoint.interval.tx` and `db.checkpoint.interval.time` settings and the checkpointing is triggered when either of them is reached. See Configure the checkpoint interval for more details.
`VOLUME`	This policy runs a checkpoint when the size of the transaction logs reaches the value specified by the `db.checkpoint.interval.volume` setting. By default, it is set to `250.00MiB`.
`CONTINUOUS`	Enterprise Edition This policy ignores `db.checkpoint.interval.tx` and `db.checkpoint.interval.time` settings and runs the checkpoint process all the time. The log pruning is triggered immediately after the checkpointing completes, just like in the periodic policy.
`VOLUMETRIC`	Enterprise Edition This policy checks every 10 seconds if there is enough volume of logs available for pruning and, if so, it triggers a checkpoint and subsequently, it prunes the logs. By default, the volume is set to 256MiB, but it can be configured using the setting `db.tx_log.rotation.retention_policy` and `db.tx_log.rotation.size`. For more information, see Configure transaction log rotation size.

Configure the checkpoint interval

Observing that you have more transaction log files than you expected is likely due to checkpoints either not happening frequently enough, or taking too long. This is a temporary condition and the gap between the expected and the observed number of log files will be closed on the next successful checkpoint. The interval between checkpoints can be configured using:

Table 2. Checkpoint interval configuration
Checkpoint configuration	Default value	Description
`db.checkpoint.interval.time`	`15m`	Configures the time interval between checkpoints.
`db.checkpoint.interval.tx`	`100000`	Configures the transaction interval between checkpoints.

Control transaction log pruning

Transaction log pruning refers to the safe and automatic removal of old, unnecessary transaction log files. Two things are necessary for a file to be removed:

The file must have been rotated.
At least one checkpoint must have happened in a more recent log file.

Transaction log pruning configuration primarily deals with specifying the number of transaction logs that should remain available. The primary reason for leaving more than the absolute minimum amount required for recovery comes from the requirements of clustered deployments and online backup. Since database updates are communicated between cluster members and backup clients through the transaction logs, keeping more than the minimum amount necessary allows for transferring just the incremental changes (in the form of transactions) instead of the whole store files, which can lead to substantial savings in time and network bandwidth.

The number of transaction logs left after a pruning operation is controlled by the setting db.tx_log.rotation.retention_policy.

The default value of db.tx_log.rotation.retention_policy is changed from 2 days to 2 days 2G, which means that Neo4j keeps logical logs that contain any transaction committed within two days and within the designated log space of 2G. For more information, see Configure transaction log retention policy.

Having the least amount of transaction log data speeds up the checkpoint process. To configure the number of IOs per second the checkpoint process is allowed to use, set the configuration parameter db.checkpoint.iops.limit.

Disabling the IOPS limit can cause transaction processing to slow down a bit. For more information, see Checkpoint IOPS limit and Transaction log settings.

Additionally, starting from 2025.07, you can also use db.checkpoint.throughput.limit to define checkpoint speed in terms of bytes per second. Compared to the IOPS limit, the throughput limit enforces a stricter control over flush speed, with the checkpoint process yielding more to stay within the configured throughput.

Starting from 2025.07, the checkpoint log messages also include the average flush speed:

Example of a log message with IOPS-limited checkpoint

Checkpoint triggered by "Call to db.checkpoint() procedure" @ txId: 92, append index: 92 checkpoint completed in 7s 464ms. Checkpoint flushed 251909 pages (9% of total available pages), in 249641 IOs. Checkpoint performed with IO limit: 600 IOPS, paused in total 70 times(6026 millis). Average checkpoint flush speed: 281.1MiB/s.

Example of a log message with throughput-limited checkpoint

Checkpoint triggered by "Call to db.checkpoint() procedure" @ txId: 88, append index: 88 checkpoint completed in 39s 457ms. Checkpoint flushed 314688 pages (12% of total available pages), in 311753 IOs. Checkpoint performed with IO limit: 64.00MiB/s, paused in total 77 times(38085 millis). Average checkpoint flush speed: 63.04MiB/s.

Checkpoint logging and metrics

The following details the expected messages to appear in the logs\debug.log upon a checkpoint event:

Checkpoint based upon db.checkpoint.interval.time:

2023-05-28 12:55:05.174+0000 INFO [o.n.k.i.t.l.c.CheckPointerImpl] Checkpoint triggered by "Scheduled checkpoint for time threshold" @ txId: 49 checkpoint started...
2023-05-28 12:55:05.253+0000 INFO [o.n.k.i.t.l.c.CheckPointerImpl] Checkpoint triggered by "Scheduled checkpoint for time threshold" @ txId: 49 checkpoint completed in 79ms. Checkpoint flushed 74 pages (7% of total available pages), in 58 IOs. Checkpoint performed with IO limit: 789 IOPS, paused in total 0 times(0 millis). Average checkpoint flush speed: 592.0KiB/s.

Checkpoint based upon db.checkpoint.interval.tx:

2023-05-28 13:08:51.603+0000 INFO [o.n.k.i.t.l.c.CheckPointerImpl] Checkpoint triggered by "Scheduled checkpoint for tx count threshold" @ txId: 118 checkpoint started...
2023-05-28 13:08:51.669+0000 INFO [o.n.k.i.t.l.c.CheckPointerImpl] Checkpoint triggered by "Scheduled checkpoint for tx count threshold" @ txId: 118 checkpoint completed in 66ms. Checkpoint flushed 74 pages (7% of total available pages), in 58 IOs. Checkpoint performed with IO limit: 789 IOPS, paused in total 0 times(0 millis). Average checkpoint flush speed: 592.0KiB/s.

Checkpoint when db.checkpoint=continuous:

2023-05-28 13:17:21.927+0000 INFO [o.n.k.i.t.l.c.CheckPointerImpl] Checkpoint triggered by "Scheduled checkpoint for continuous threshold" @ txId: 171 checkpoint started...
2023-05-28 13:17:21.941+0000 INFO [o.n.k.i.t.l.c.CheckPointerImpl] Checkpoint triggered by "Scheduled checkpoint for continuous threshold" @ txId: 171 checkpoint completed in 13ms. Checkpoint flushed 74 pages (7% of total available pages), in 58 IOs. Checkpoint performed with IO limit: 789 IOPS, paused in total 0 times(0 millis). Average checkpoint flush speed: 592.0KiB/s.

Checkpoint as a result of database shutdown:

2023-05-28 12:35:56.272+0000 INFO [o.n.k.i.t.l.c.CheckPointerImpl] Checkpoint triggered by "Database shutdown" @ txId: 47 checkpoint started...
2023-05-28 12:35:56.306+0000 INFO [o.n.k.i.t.l.c.CheckPointerImpl] Checkpoint triggered by "Database shutdown" @ txId: 47 checkpoint completed in 34ms. Checkpoint flushed 74 pages (7% of total available pages), in 58 IOs. Checkpoint performed with IO limit: 789 IOPS, paused in total 0 times(0 millis). Average checkpoint flush speed: 592.0KiB/s.

Checkpoint as a result of CALL db.checkpoint():

2023-05-28 12:31:56.463+0000 INFO [o.n.k.i.t.l.c.CheckPointerImpl] Checkpoint triggered by "Call to db.checkpoint() procedure" @ txId: 47 checkpoint started...
2023-05-28 12:31:56.490+0000 INFO [o.n.k.i.t.l.c.CheckPointerImpl] Checkpoint triggered by "Call to db.checkpoint() procedure" @ txId: 47 checkpoint completed in 27ms. Checkpoint flushed 74 pages (7% of total available pages), in 58 IOs. Checkpoint performed with IO limit: 789 IOPS, paused in total 0 times(0 millis). Average checkpoint flush speed: 592.0KiB/s.

Checkpoint as a result of a backup run:

2023-05-28 12:33:30.489+0000 INFO [o.n.k.i.t.l.c.CheckPointerImpl] Checkpoint triggered by "Full backup" @ txId: 47 checkpoint started...
2023-05-28 12:33:30.509+0000 INFO [o.n.k.i.t.l.c.CheckPointerImpl] Checkpoint triggered by "Full backup" @ txId: 47 checkpoint completed in 20ms. Checkpoint flushed 74 pages (7% of total available pages), in 58 IOs. Checkpoint performed with IO limit: 789 IOPS, paused in total 0 times(0 millis). Average checkpoint flush speed: 592.0KiB/s.

Checkpoint Metrics are also available and are detailed in the following files, in the metrics/ directory:

neo4j.check_point.duration.csv
neo4j.check_point.total_time.csv
neo4j.check_point.events.csv

Glossary

allocator: A component in the cluster that allocates databases to servers according to the topology constraints specified and an allocation strategy.
asynchronous replication: Asynchronous replication is used by secondary copies to poll for new transactions, which means they cannot be guaranteed to have received the most recent transactions. This enables efficient scale-out of read-performance.
Aura instance: A fully-managed DBMS represented by a single instance ID, that is running in the Neo4j Aura cloud.
auto-commit transaction: An automatically committed transaction that contains a single query.
Bolt protocol: Bolt is a protocol used for interaction between Neo4j instances and drivers.
bookmark: A marker the client can request from the cluster to ensure that it is able to read its own writes so that the application’s state is consistent and only databases that have a copy of the bookmark are permitted to respond.
category (Bloom): A category is based on a node label and is defined in a Perspective as a way of visually distinguishing nodes with the same label(s).
causal consistency: All servers in a cluster agree on the order in which transactions take place. The position of a server on the causal chain can be guaranteed using a bookmark.
cluster: A Neo4j DBMS that spans multiple servers working together to increase fault tolerance and/or read scalability. Databases on a cluster may be configured to replicate across servers in the cluster thus achieving read scalability or high availability.
client application: Software that interacts with a Neo4j server.
commit: A commit is the successful completion of a transaction, which ensures durability of any changes made. For more details, visit Operations Manual → Transaction management.
composite database: Composite databases are the means to access partitioned graph data with a single Cypher query.
constraint: Constraints are sets of data modeling rules that ensure the data is consistent and reliable.
Cypher®: Neo4j’s graph query language.
data model: A data model defines how information is organized in a database. A good data model will make querying and understanding your data easier. In Neo4j, the data models have a graph structure.
database: A database is a container used by the DBMS to manage and store graph data. The physical structure of data is controlled by the database.
database vs graph: Databases are the physical containers of graph data. Graphs are the logical structure of data in Neo4j.
Database Management System: Database Management System, or DBMS, capable of managing multiple databases. A DBMS may run on a single server, or span several servers configured as a cluster.
database schema: The prescribed property existence and datatypes for nodes and relationships.
deallocate: An act of removing a database from a server or a server from a cluster without loss of data or reduced fault tolerance.
degree (of a node): The number of relationships of a specific node; loops are counted twice.
disaster recovery: A manual intervention to restore availability of a cluster, or databases within a cluster.
driver: A software library that provides access to Neo4j from a particular programming language.
election: In the event that the Raft leader becomes unresponsive, followers automatically trigger an election and vote for a new leader.
entity: A node or a relationship.
expression (Cypher): A component of a Cypher query which produces values. It may be used in projections, as a predicate, or when setting properties on graph elements.
fabric: Fabric is the architectural design of a unified system that provides a single access point to local or distributed graph data.
fault tolerance: A guarantee that a cluster can maintain a database’s persistence and availability in the event of one or more servers failing.
follower: A primary copy of a database acting as a follower, receives and acknowledges synchronous writes from the leader.
Generative AI (GenAI): A type of artificial intelligence (AI) system that generates text, images, or other media in response to prompts.
graph: A logical representation of a set of nodes where some pairs are connected by relationships.
index: Data structure that improves read performance of a database.
knowledge graph: A specific type of graph that has an organizing principle so that a user (or a computer system) can reason about the underlying data. The organizing principle provides an additional layer of structure that adds context to support knowledge discovery.
label: Marks a node as a member of a named and indexed subset. A node may be assigned zero or more labels.
leader: A single primary copy of a database is designated as the leader. It receives all write transactions from clients and replicates writes synchronously to followers and asynchronously to secondary copies of the database.
main database: In terms of Neo4j Enterprise Studio, the database(s) containing the user’s data. Can exist in the same Neo4j deployment as the tool asset database.
motif: A description of a specific pattern within a graph.
node: A node represents an entity or discrete object in your graph data model. Nodes can be connected by relationships, hold data in properties, and are classified by labels.
operator: A symbol representing a mathematical or logical operation.
parameter: Named value provided when running a Cypher statement.
path: A sequence of nodes and the relationships connecting them, that does not contain duplicate relationships. Several paths can match a pattern.
pattern: A specific arrangement of nodes and relationships that can be matched in a graph. A pattern follows a motif.
perspective (Bloom): A Perspective defines a certain business view or domain that can be found in the target Neo4j graph. A single Neo4j graph can be viewed through different Perspectives, each tailored for a different business purpose.
primary: A copy of the database that is able to process write transactions and is eligible to be elected as a leader. It participates in fault tolerant writes as it is part of the majority required to acknowledge and commit write transactions.
primary vs secondary: In a cluster, databases can operate in either primary or secondary mode. Primary databases are able to process write and read transactions, ensuring fault tolerance. Secondary databases are replicated asynchronously from primaries, and their main purpose is to provide read scaling within the cluster.
project (Aura): An isolated environment in the unified Aura console that contains its own database instances, configurations, and resources. Preceded by tenant in the classic Aura console.
property: Properties are key-value pairs that are used for storing data on nodes and relationships.
query (Cypher): A statement that retrieves or writes information to a database.
Raft group: A group of servers that are participating in hosting a particular database in primary mode.
Raft group member: A server that is participating in a Raft group. A server can be a member of one or more groups.
Raft log: A shared log between all Raft group members that is guaranteed to be consistently updated and viewed by those members. The log contains both database data and operational state of the Raft group.
Raft protocol: The networking mechanism that enables a database to replicate its data across multiple servers to give high availability for accessing the data and high durability to the data stored.
read scaling: Distributing query load by creating additional database copies hosted in secondary mode (read-only).
relationship: A relationship represents a connection between nodes in your graph data model. Relationships connect a source node to a target node, hold data in properties, and are classified by type.
secondary: An asynchronously replicated copy of the database that provides read scaling within the cluster.
seed: A seed is a database dump or a full backup used to create a database on a cluster. This is sometimes called seeding.
server: A physical machine, a virtual machine, or a container running an instance of Neo4j. Servers can be standalone or part of a cluster.
session: A causally linked sequence of transactions.
session consistency: An alternative name for Neo4j’s causal consistency.
standalone: A single server running Neo4j and not part of a cluster.
synchronous replication: Synchronous replication requires the leader primary to replicate a transaction and block the commit until a quorum of the follower primaries acknowledges that the transaction is successfully replicated. Once the transaction is replicated, the commit is allowed to proceed. This ensures data durability and consistency within the cluster.
system database: A database used by Neo4j to store system information.
tenant (Aura): An isolated environment in the classic Aura console that contains its own database instances, configurations, and resources. Replaced by project in the unified Aura console.
tool asset database: In terms of Neo4j Enterprise Studio, the database where tools' assets are stored. This can be in the same Neo4j deployment as the main database(s) or in a separate deployment.
topology: A configuration that describes how the copies of a database should be spread across the servers in a cluster, see primary mode and secondary mode.
transaction: A transaction comprises a unit of work performed against a database. It is treated in a coherent and reliable way, independent of other transactions. Transactions comply with the ACID consistency model (atomic, consistent, isolated, and durable).