Monitor a Neo4j instance in standalone Docker container

A Neo4j instance running in a Docker container can be monitored by a NOM agent in two ways:

The NOM agent can be run as an additional (daemon) process in the Neo4j container
The NOM agent can be run on the Docker host to externally monitor the Neo4j container

Running additional processes that are not tied to the lifecycle of the container process is not a best practice for containerized applications.

In-container Monitoring

The following steps describe running a NOM agent in Neo4j container as an additional (daemon) process to monitor the instance:

Run the Neo4j docker container.
- For more information, see Operations Manual → Docker.
- A custom entrypoint script can be mounted to the Neo4j container to run the NOM agent at DBMS startup which is described further on.

Install the NOM agent to make the agent command available to run.

Bundled
- NOM agent binaries are bundled in the Neo4j package and reside in the products folder of the Neo4j installation directory.
- It can be extracted from the package and installed into a system binaries directory for access to the agent command.
  tar -xzf $NEO4J_HOME/products/neo4j-ops-manager-agent-<NOM_VERSION>-linux-amd64.tar.gz --strip-components 1 && mv bin/agent-<NOM_VERSION>-linux-amd64 /bin/agent

Mounted

NOM agent binaries can be mounted into the container as a volume mount and installed into a system binaries directory for access to agent command.

docker run -e NEO4J_ACCEPT_LICENSE_AGREEMENT=yes -v <path/to/agent/binary>:/agent/bin neo4j/neo4j:latest --name <neo4j container name>

Wait for the container to be ready and then run :

docker exec -it <neo4j container name> -- /bin/sh -c "mv /agent/bin/agent-*-linux-amd64 /bin/agent"

Downloaded

NOM agent package can be directly downloaded on to the Neo4j container and then extracted and installed.

wget -O /tmp/neo4j-ops-manager-agent-<NOM_VERSION>-linux-amd64.tar.gz https://dist.neo4j.org/ops-manager/<NOM_VERSION>/neo4j-ops-manager-agent-<NOM_VERSION>-linux-amd64.tar.gz
tar -xzf /tmp/neo4j-ops-manager-agent-*-linux-amd64.tar.gz --strip-components 1 && mv bin/agent-*-linux-amd64 /bin/agent

Run the NOM agent, which involves executing the agent binary with appropriate agent configuration.

Agent configuration can be provided as environment variables set during docker exec command or as part of container startup command.

Run agent in console mode

docker exec -dit <neo4j container name> -- /bin/sh 'CONFIG_AGENT_NAME="test_local" \
    CONFIG_AGENT_DESCRIPTION="local self-registered agent" \
    CONFIG_SERVER_GRPC_ADDRESS="server:9090" \
    CONFIG_SERVER_HTTP_ADDRESS="https://server:8080" \
    CONFIG_TOKEN_URL=https://server:8080/api/login \
    CONFIG_TOKEN_CLIENT_ID=rj34hg6456ghrhjthet45hg6 \
    CONFIG_TOKEN_CLIENT_SECRET=5h346jhrjkthkjthjerhtkrt \
    CONFIG_TLS_TRUSTED_CERTS=/path/to/trusted/certs/pem/file \
    CONFIG_LOG_FILE=/path/to/nom-agent/log.txt \
    CONFIG_LOG_LEVEL="debug" \
    CONFIG_INSTANCE_1_NAME="test_local" \
    CONFIG_INSTANCE_1_BOLT_URI="bolt://localhost:7687" \
    CONFIG_INSTANCE_1_BOLT_USERNAME="neo4j" \
    CONFIG_INSTANCE_1_BOLT_PASSWORD="passw0rd" \
    CONFIG_INSTANCE_1_QUERY_LOG_PORT="9500" \
    CONFIG_INSTANCE_1_LOG_CONFIG_PATH="/var/lib/neo4j/conf/server-logs.xml" \
    CONFIG_INSTANCE_1_QUERY_LOG_MIN_DURATION="100" \
    agent console -s'

Start a Neo4j container with NOM agent using a custom entry point

Start the Neo4j container by running the entrypoint script /startup/docker-entrypoint.sh with the command neo4j. To run the NOM agent on the container startup, this entrypoint needs to be overridden with a custom entrypoint script as shown:

In console mode:

entrypoint.sh

#!/bin/bash -eu

/startup/docker-entrypoint.sh neo4j &
CONFIG_AGENT_NAME="test_local" \
CONFIG_AGENT_DESCRIPTION="local self-registered agent" \
CONFIG_SERVER_GRPC_ADDRESS="server:9090" \
CONFIG_SERVER_HTTP_ADDRESS="https://server:8080" \
CONFIG_TOKEN_URL=https://server:8080/api/login \
CONFIG_TOKEN_CLIENT_ID=rj34hg6456ghrhjthet45hg6 \
CONFIG_TOKEN_CLIENT_SECRET=5h346jhrjkthkjthjerhtkrt \
CONFIG_TLS_TRUSTED_CERTS=/path/to/trusted/certs/pem/file \
CONFIG_LOG_FILE=/path/to/nom-agent/log.txt \
CONFIG_LOG_LEVEL="debug" \
CONFIG_INSTANCE_1_NAME="test_local" \
CONFIG_INSTANCE_1_BOLT_URI="bolt://localhost:7687" \
CONFIG_INSTANCE_1_BOLT_USERNAME="neo4j" \
CONFIG_INSTANCE_1_BOLT_PASSWORD="passw0rd" \
CONFIG_INSTANCE_1_QUERY_LOG_PORT="9500" \
CONFIG_INSTANCE_1_LOG_CONFIG_PATH="/var/lib/neo4j/conf/server-logs.xml" \
CONFIG_INSTANCE_1_QUERY_LOG_MIN_DURATION="100" \
agent console -s

 docker run --entrypoint /custom/entrypoint.sh -e NEO4J_ACCEPT_LICENSE_AGREEMENT=yes -v <path/to/custom/entrypoint>:/custom $NEO4J_IMAGE

In service mode:

entrypoint.sh

#!/bin/bash -eu

/startup/docker-entrypoint.sh neo4j &
agent service install
cat > neo4j-ops-manager-agent.service <<EOF
[Service]
Environment="CONFIG_SERVER_GRPC_ADDRESS=<server grpc address>"
Environment="CONFIG_SERVER_HTTP_ADDRESS=<server http address>"
Environment="CONFIG_TOKEN_URL=<server http login url>"
Environment="CONFIG_TOKEN_CLIENT_ID=<client id>"
Environment="CONFIG_TOKEN_CLIENT_SECRET=<client secret>"
Environment="CONFIG_TLS_TRUSTED_CERTS=</path/to/trusted/certs/pem/file>"
Environment="CONFIG_LOG_FILE=</path/to/nom-agent/log.txt>"
Environment="CONFIG_INSTANCE_1_NAME=<instance name>"
Environment="CONFIG_INSTANCE_1_BOLT_URI=<bolt uri of the local instance>"
Environment="CONFIG_INSTANCE_1_BOLT_USERNAME=<local instance user name>"
Environment="CONFIG_INSTANCE_1_BOLT_PASSWORD=<local instance password>"
Environment="CONFIG_INSTANCE_1_QUERY_LOG_PORT=<an available port>"
Environment="CONFIG_INSTANCE_1_LOG_CONFIG_PATH=<path to server-logs.xml>"
EOF
systemctl start neo4j-ops-manager-agent.service

 docker run --entrypoint /custom/entrypoint.sh -e NEO4J_ACCEPT_LICENSE_AGREEMENT=yes -v <path/to/custom/entrypoint>:/custom $NEO4J_IMAGE

External monitoring

A NOM agent can be run in either the console or service mode on the Docker host. It can be configured to have access to Neo4j container resources. Apply the following additional configurations to the Neo4j container run config to enable an external NOM agent to monitor the instance correctly:

docker run --entrypoint /custom/entrypoint.sh -e NEO4J_ACCEPT_LICENSE_AGREEMENT=yes \
           -v <path/to/custom/entrypoint>:/custom \
           -v </path/to/local/neo4j/home>:/var/lib/neo4j \ (1)
           -p "8884:2004" \ (2)
           -p "9500:9500" \ (3)
       neo4j/neo4j:latest

1	Neo4j home directory needs to be mounted back on the Docker host to enable access to agent.
2	Neo4j prometheus endpoint port (2004) needs to be exposed via port mapping.
3	Query log port (9500) needs to be mapped for log appender to forward query logs.