Metrics reference

This section includes metrics that are primarily to allow Neo4j experts to fine-tune specific issues.

You should use caution when interpreting unfamiliar metrics. Reading the Performance section is recommended to better understand the metrics.

1. Types of metrics

Neo4j has the following types of metrics:

  • Global — covers the whole Neo4j DBMS.

  • Per database — covers an individual database.

1.1. Global metrics

Global metrics cover the whole database management system, and represents the status of the system as a whole.

Global metrics have the following name format: <user-configured-prefix>.<metric-name> if metrics.namespaces.enabled is false, or <user-configured-prefix>.dbms.<metric-name> if the setting is true.

Metrics of this type are reported as soon as the database management system is available. For example, all JVM related metrics are global. In particular, the neo4j.vm.thread.count metric has a default user-configured-prefix neo4j and the metric name is vm.thread.count.

By default, global metrics include:

  • Bolt metrics

  • Page cache metrics

  • GC metrics

  • Thread metrics

  • Database operation metrics

  • Web Server metrics

  • JVM metrics

1.2. Database metrics

Each database metric is reported for a particular database only. Database metrics are only available during the lifetime of the database. When a database becomes unavailable, all of its metrics become unavailable also.

Database metrics have the following name format: <user-configured-prefix>.<database-name>.<metric-name> if metrics.namespaces.enabled is false, or <user-configured-prefix>.database.<database-name>.<metric-name> if the setting is true.

For example, any transaction metric is a database metric. In particular, the neo4j.mydb.transaction.started metric has a default user-configured-prefix neo4j and is a metric for the mydb database.

By default, database metrics include:

  • Transaction metrics

  • Checkpoint metrics

  • Log rotation metrics

  • Database data metrics

  • Cypher metrics

  • Causal clustering metrics

2. General-purpose metrics

Table 1. Bolt metrics
Name Description

<prefix>.bolt.sessions_started

The total number of Bolt sessions started since this instance started. This includes both succeeded and failed sessions (deprecated, use connections_opened instead). (counter)

<prefix>.bolt.connections_opened

The total number of Bolt connections opened since this instance started. This includes both succeeded and failed connections. (counter)

<prefix>.bolt.connections_closed

The total number of Bolt connections closed since this instance started. This includes both properly and abnormally ended connections. (counter)

<prefix>.bolt.connections_running

The total number of Bolt connections currently being executed. (gauge)

<prefix>.bolt.connections_idle

The total number of Bolt connections sitting idle. (gauge)

<prefix>.bolt.messages_received

The total number of messages received via Bolt since this instance started. (counter)

<prefix>.bolt.messages_started

The total number of messages that began processing since this instance started. This is different from messages received in that this counter tracks how many of the received messages havebeen taken on by a worker thread. (counter)

<prefix>.bolt.messages_done

The total number of messages that completed processing since this instance started. This includes successful, failed and ignored Bolt messages. (counter)

<prefix>.bolt.messages_failed

The total number of messages that failed processing since this instance started. (counter)

<prefix>.bolt.accumulated_queue_time

The accumulated time messages have spent waiting for a worker thread. (counter)

<prefix>.bolt.accumulated_processing_time

The accumulated time worker threads have spent processing messages. (counter)

Table 2. Database checkpointing metrics
Name Description

<prefix>.check_point.events

The total number of check point events executed so far. (counter)

<prefix>.check_point.total_time

The total time, in milliseconds, spent in check pointing so far. (counter)

<prefix>.check_point.duration

The duration, in milliseconds, of the last check point event. (gauge)

Table 3. Cypher metrics
Name Description

<prefix>.cypher.replan_events

The total number of times Cypher has decided to re-plan a query. (counter)

<prefix>.cypher.replan_wait_time

The total number of seconds waited between query replans. (counter)

Table 4. Database data count metrics
Name Description

<prefix>.neo4j.count.relationship

The total number of relationships in the database. (gauge)

<prefix>.neo4j.count.node

The total number of nodes in the database. (gauge)

Table 5. Database neo4j pools metrics
Name Description

<prefix>.pool.<pool>.<database>.used_heap

Used or reserved heap memory in bytes. (gauge)

<prefix>.pool.<pool>.<database>.used_native

Used or reserved native memory in bytes. (gauge)

<prefix>.pool.<pool>.<database>.total_used

Sum total used heap and native memory in bytes. (gauge)

<prefix>.pool.<pool>.<database>.total_size

Sum total size of capacity of the heap and/or native memory pool. (gauge)

<prefix>.pool.<pool>.<database>.free

Available unused memory in the pool, in bytes. (gauge)

Table 6. Database operation count metrics
Name Description

<prefix>.db.operation.count.create

Count of successful database create operations. (counter)

<prefix>.db.operation.count.start

Count of successful database start operations. (counter)

<prefix>.db.operation.count.stop

Count of successful database stop operations. (counter)

<prefix>.db.operation.count.drop

Count of successful database drop operations. (counter)

<prefix>.db.operation.count.failed

Count of failed database operations. (counter)

<prefix>.db.operation.count.recovered

Count of database operations which failed previously but have recovered. (counter)

Table 7. Database data metrics
Name Description

<prefix>.ids_in_use.relationship_type

The total number of different relationship types stored in the database. (gauge)

<prefix>.ids_in_use.property

The total number of different property names used in the database. (gauge)

<prefix>.ids_in_use.relationship

The total number of relationships stored in the database. (gauge)

<prefix>.ids_in_use.node

The total number of nodes stored in the database. (gauge)

Table 8. Global neo4j pools metrics
Name Description

<prefix>.dbms.pool.<pool>.used_heap

Used or reserved heap memory in bytes. (gauge)

<prefix>.dbms.pool.<pool>.used_native

Used or reserved native memory in bytes. (gauge)

<prefix>.dbms.pool.<pool>.total_used

Sum total used heap and native memory in bytes. (gauge)

<prefix>.dbms.pool.<pool>.total_size

Sum total size of capacity of the heap and/or native memory pool. (gauge)

<prefix>.dbms.pool.<pool>.free

Available unused memory in the pool, in bytes. (gauge)

Table 9. Database page cache metrics
Name Description

<prefix>.page_cache.eviction_exceptions

The total number of exceptions seen during the eviction process in the page cache. (counter)

<prefix>.page_cache.flushes

The total number of page flushes executed by the page cache. (counter)

<prefix>.page_cache.merges

The total number of page merges executed by the page cache. (counter)

<prefix>.page_cache.unpins

The total number of page unpins executed by the page cache. (counter)

<prefix>.page_cache.pins

The total number of page pins executed by the page cache. (counter)

<prefix>.page_cache.evictions

The total number of page evictions executed by the page cache. (counter)

<prefix>.page_cache.evictions.cooperative

The total number of cooperative page evictions executed by the page cache due to low available pages. (counter)

<prefix>.page_cache.page_faults

The total number of page faults happened in the page cache. (counter)

<prefix>.page_cache.hits

The total number of page hits happened in the page cache. (counter)

<prefix>.page_cache.hit_ratio

The ratio of hits to the total number of lookups in the page cache. (gauge)

<prefix>.page_cache.usage_ratio

The ratio of number of used pages to total number of available pages. (gauge)

<prefix>.page_cache.bytes_read

The total number of bytes read by the page cache. (counter)

<prefix>.page_cache.bytes_written

The total number of bytes written by the page cache. (counter)

<prefix>.page_cache.iops

The total number of IO operations performed by page cache.

<prefix>.page_cache.throttled.times

The total number of times page cache flush IO limiter was throttled during ongoing IO operations.

<prefix>.page_cache.throttled.millis

The total number of millis page cache flush IO limiter was throttled during ongoing IO operations.

Table 10. Query execution metrics
Name Description

<prefix>.db.query.execution.success

Count of successful queries executed. (meter)

<prefix>.db.query.execution.failure

Count of failed queries executed. (meter)

<prefix>.db.query.execution.latency.millis

Execution time in milliseconds of queries executed successfully. (histogram)

Table 11. Database store size metrics
Name Description

<prefix>.store.size.total

The total size of the database and transaction logs, in bytes. (gauge)

<prefix>.store.size.database

The size of the database, in bytes. (gauge)

Table 12. Database transaction log metrics
Name Description

<prefix>.log.rotation_events

The total number of transaction log rotations executed so far. (counter)

<prefix>.log.rotation_total_time

The total time, in milliseconds, spent in rotating transaction logs so far. (counter)

<prefix>.log.rotation_duration

The duration, in milliseconds, of the last log rotation event. (gauge)

<prefix>.log.appended_bytes

The total number of bytes appended to transaction log. (counter)

<prefix>.log.flushes

The total number of transaction log flushes. (counter)

<prefix>.log.append_batch_size

The size of the last transaction append batch. (gauge)

Table 13. Database transaction metrics
Name Description

<prefix>.transaction.started

The total number of started transactions. (counter)

<prefix>.transaction.peak_concurrent

The highest peak of concurrent transactions. (counter)

<prefix>.transaction.active

The number of currently active transactions. (gauge)

<prefix>.transaction.active_read

The number of currently active read transactions. (gauge)

<prefix>.transaction.active_write

The number of currently active write transactions. (gauge)

<prefix>.transaction.committed

The total number of committed transactions. (counter)

<prefix>.transaction.committed_read

The total number of committed read transactions. (counter)

<prefix>.transaction.committed_write

The total number of committed write transactions. (counter)

<prefix>.transaction.rollbacks

The total number of rolled back transactions. (counter)

<prefix>.transaction.rollbacks_read

The total number of rolled back read transactions. (counter)

<prefix>.transaction.rollbacks_write

The total number of rolled back write transactions. (counter)

<prefix>.transaction.terminated

The total number of terminated transactions. (counter)

<prefix>.transaction.terminated_read

The total number of terminated read transactions. (counter)

<prefix>.transaction.terminated_write

The total number of terminated write transactions. (counter)

<prefix>.transaction.last_committed_tx_id

The ID of the last committed transaction. (counter)

<prefix>.transaction.last_closed_tx_id

The ID of the last closed transaction. (counter)

<prefix>.transaction.tx_size_heap

The transactions' size on heap in bytes. (histogram)

<prefix>.transaction.tx_size_native

The transactions' size in native memory in bytes. (histogram)

Table 14. Server metrics
Name Description

<prefix>.server.threads.jetty.idle

The total number of idle threads in the jetty pool. (gauge)

<prefix>.server.threads.jetty.all

The total number of threads (both idle and busy) in the jetty pool. (gauge)

3. Metrics specific to Causal Clustering

Table 15. CatchUp Metrics
Name Description

<prefix>.causal_clustering.catchup.tx_pull_requests_received

TX pull requests received from read replicas. (counter)

Table 16. Discovery core metrics
Name Description

<prefix>.causal_clustering.core.discovery.replicated_data

Size of replicated data structures. (gauge)

<prefix>.causal_clustering.core.discovery.cluster.members

Discovery cluster member size. (gauge)

<prefix>.causal_clustering.core.discovery.cluster.unreachable

Discovery cluster unreachable size. (gauge)

<prefix>.causal_clustering.core.discovery.cluster.converged

Discovery cluster convergence. (gauge)

Table 17. Raft core metrics
Name Description

<prefix>.causal_clustering.core.append_index

The append index of the Raft log. Each index represents a write transaction (possibly internal) proposed for commitment. The values mostly increase, but sometimes they can decrease as a consequence of leader changes. The append index should always be less than or equal to the commit index. (gauge)

<prefix>.causal_clustering.core.commit_index

The commit index of the Raft log. Represents the commitment of previously appended entries. Its value increases monotonically if you do not unbind the cluster state. The commit index should always be bigger than or equal to the append index. (gauge)

<prefix>.causal_clustering.core.applied_index

The applied index of the Raft log. Represents the application of the committed Raft log entries to the database and internal state. The applied index should always be bigger than or equal to the commit index. The difference between this and the commit index can be used to monitor how up-to-date the follower database is. (gauge)

<prefix>.causal_clustering.core.term

The Raft Term of this server. It increases monotonically if you do not unbind the cluster state. (gauge)

<prefix>.causal_clustering.core.tx_retries

Transaction retries. (counter)

<prefix>.causal_clustering.core.is_leader

Is this server the leader? (gauge)

<prefix>.causal_clustering.core.in_flight_cache.total_bytes

In-flight cache total bytes. (gauge)

<prefix>.causal_clustering.core.in_flight_cache.max_bytes

In-flight cache max bytes. (gauge)

<prefix>.causal_clustering.core.in_flight_cache.element_count

In-flight cache element count. (gauge)

<prefix>.causal_clustering.core.in_flight_cache.max_elements

In-flight cache maximum elements. (gauge)

<prefix>.causal_clustering.core.in_flight_cache.hits

In-flight cache hits. (counter)

<prefix>.causal_clustering.core.in_flight_cache.misses

In-flight cache misses. (counter)

<prefix>.causal_clustering.core.raft_log_entry_prefetch_buffer.lag

Raft Log Entry Prefetch Lag. (gauge)

<prefix>.causal_clustering.core.raft_log_entry_prefetch_buffer.bytes

Raft Log Entry Prefetch total bytes. (gauge)

<prefix>.causal_clustering.core.raft_log_entry_prefetch_buffer.size

Raft Log Entry Prefetch buffer size. (gauge)

<prefix>.causal_clustering.core.raft_log_entry_prefetch_buffer.async_put

Raft Log Entry Prefetch buffer async puts. (gauge)

<prefix>.causal_clustering.core.raft_log_entry_prefetch_buffer.sync_put

Raft Log Entry Prefetch buffer sync puts. (gauge)

<prefix>.causal_clustering.core.message_processing_delay

Delay between Raft message receive and process. (gauge)

<prefix>.causal_clustering.core.message_processing_timer

Timer for Raft message processing. (counter, histogram)

<prefix>.causal_clustering.core.replication_new

The total number of Raft replication requests. It increases with write transactions (possibly internal) activity. (counter)

<prefix>.causal_clustering.core.replication_attempt

The total number of Raft replication requests attempts. It is bigger or equal than the replication requests. (counter)

<prefix>.causal_clustering.core.replication_fail

The total number of Raft replication attempts that have failed. (counter)

<prefix>.causal_clustering.core.replication_maybe

Raft Replication maybe count. (counter)

<prefix>.causal_clustering.core.replication_success

The total number of Raft replication requests that have succeeded. (counter)

<prefix>.causal_clustering.core.last_leader_message

The time elapsed since the last message from a leader in milliseconds. Should reset periodically. (gauge)

Table 18. Read Replica Metrics
Name Description

<prefix>.causal_clustering.read_replica.pull_updates

The total number of pull requests made by this instance. (counter)

<prefix>.causal_clustering.read_replica.pull_update_highest_tx_id_requested

The highest transaction id requested in a pull update by this instance. (counter)

<prefix>.causal_clustering.read_replica.pull_update_highest_tx_id_received

The highest transaction id that has been pulled in the last pull updates by this instance. (counter)

4. Java Virtual Machine Metrics

The JVM metrics show information about garbage collections (for example, the number of events and time spent collecting), memory pools and buffers, and the number of active threads running. They are environment dependent and therefore, may vary on different hardware and with different JVM configurations. The metrics about the JVM’s memory usage expose values that are provided by the MemoryPoolMXBeans and BufferPoolMXBeans. The memory pools are memory managed by the JVM, for example, neo4j.vm.memory.pool.g1_survivor_space. Therefore, if necessary, you can tune them using the JVM settings. The buffer pools are space outside of the memory managed by the garbage collector. Neo4j allocates buffers in those pools as it needs them. You can limit this memory using JVM settings, but there is never any good reason for you to set them.

Table 19. JVM file descriptor metrics.
Name Description

<prefix>.vm.file.descriptors.count

The current number of open file descriptors. (gauge)

<prefix>.vm.file.descriptors.maximum

The maximum number of open file descriptors. (gauge)

Table 20. GC metrics.
Name Description

<prefix>.vm.gc.time.<gc>

Accumulated garbage collection time in milliseconds. (counter)

<prefix>.vm.gc.count.<gc>

Total number of garbage collections. (counter)

Table 21. JVM Heap metrics.
Name Description

<prefix>.vm.heap.committed

Amount of memory (in bytes) guaranteed to be available for use by the JVM. (gauge)

<prefix>.vm.heap.used

Amount of memory (in bytes) currently used. (gauge)

<prefix>.vm.heap.max

Maximum amount of heap memory (in bytes) that can be used. (gauge)

Table 22. JVM memory buffers metrics.
Name Description

<prefix>.vm.memory.buffer.<bufferpool>.count

Estimated number of buffers in the pool. (gauge)

<prefix>.vm.memory.buffer.<bufferpool>.used

Estimated amount of memory used by the pool. (gauge)

<prefix>.vm.memory.buffer.<bufferpool>.capacity

Estimated total capacity of buffers in the pool. (gauge)

Table 23. JVM memory pools metrics.
Name Description

<prefix>.vm.memory.pool.<pool>

Estimated amount of memory in bytes used by the pool. (gauge)

Table 24. JVM pause time metrics.
Name Description

<prefix>.vm.pause_time

Accumulated detected VM pause time. (gauge)

Table 25. JVM threads metrics.
Name Description

<prefix>.vm.thread.count

Estimated number of active threads in the current thread group. (gauge)

<prefix>.vm.thread.total

The total number of live threads including daemon and non-daemon threads. (gauge)