Log forwarding
AuraDB Business CriticalAuraDB Virtual Dedicated CloudAuraDS Enterprise

With log forwarding, you can stream logs directly to a cloud project owned by your organization, in real time. The following types of logs are available:

  • Security logs

  • Query logs

To access log forwarding, you need to have the Project Admin role.

To access Log forwarding:

  1. Navigate to Settings under the Project section in the sidebar.

  2. Select Log forwarding.

This will display a list of currently configured log forwarding processes for the active project. Each configuration will show its scope (region or instance) and status (forwarding, setting up or paused).

For actions related to existing configurations, use the …​ button on the right side of the row to open a menu from which the following actions can be taken:

  • View configuration - Displays the complete details of the configuration.

  • Edit - Allows you to change the configuration.

  • Delete - Removes the configuration.

If no log forwarding process is set up, a button to do so is displayed in the center of the page.

Set up log forwarding

Security is of paramount importance, and therefore the security logs are initially available at no cost. For query logs, a volume-based egress charge will apply. Refer to your contract for details.

The complete steps for setting up log forwarding depend on the chosen cloud provider.

Exhaustive instructions are provided in the wizard which appears by following the steps below.

  1. Navigate to the Log forwarding page as described above.

  2. Use Configure log forwarding and select the scope for log forwarding.

    • AuraDB Business Critical A specific instance will have its logs forwarded.

    • AuraDB Virtual Dedicated Cloud All instances in the selected region will have their logs forwarded.

    • AuraDS Enterprise All instances in the selected region will have their logs forwarded.

  3. Select the type of logs to forward: Security logs or Query logs.

    • For query logs, it is possible to apply certain filters to decrease the volume of forwarded data.

  4. Follow the instructions specific to your cloud provider.

Only one log forwarding configuration of each type is permitted for each unique scope.

Query log filters

To apply filters to query logs, expand the Filter section in the log forwarding configuration wizard.

Neo4j queries produce two log entries—a start entry indicating that the query was started, and an end entry containing the result of the query execution.

The option Remove start entries allows you to only forward the end entries, which is sufficient for most analytical use cases and cuts the egress volume by almost half.

The Include dropdown selects whether to forward all queries, only successful queries, or only failed queries.

Output destination

Log forwarding can forward logs to the log service of the same cloud provider as the monitored instance is located in.

Cross-region log forwarding is supported.

If your instance is in:

  • Google Cloud Platform - Forward logs to Google Cloud Logging in your own GCP project.

  • Amazon Web Services - Forward logs to CloudWatch in your own AWS account.

  • Azure - Forward logs to a Log Analytics workspace in your own Azure subscription.

Logs can be further forwarded into third party systems using the log routing capabilities provided by your cloud provider.

Traffic flows and data privacy

Log forwarding is designed to maintain complete privacy of your data. By forwarding logs within the same cloud provider your instance is running on, the backplane of that cloud provider can be used to ensure your logs never cross the public internet.

All data transmission is handled by the industry standard fluent-bit component.

Table 1. Fluent-bit output plugin by cloud service provider
Cloud server provider Output plugin used

Google Cloud Platform

Stackdriver

Amazon Web Services

Amazon CloudWatch

Azure

Azure Log Analytics

For details on the implementation for specific cloud providers, see the sections below.

After the log data has been securely handed off from the Neo4j cluster to your cloud provider, Neo4j cannot control and does not take any responsibility for how the data is handled. The descriptions below are provided for information only. For the latest information, consult your cloud provider and/or any third party solution provider you choose to employ.

Google Cloud Platform

When using log forwarding with GCP, all logs are transmitted to the Cloud Logging API of the source data center via the internal endpoint logging.googleapis.com. At this point, the logs become available to you via Log Router without ever leaving the data center. Authentication uses Workload Identity Federation.

If you set up a log sink that sends logs to a Google service in another region, traffic is generally kept secure through encryption and by using an internal load balancer which keeps the traffic on the internal backplane. For more information, see load balancing overview.

Amazon Web Services

When using log forwarding with AWS, all logs are transmitted to the local CloudWatch destination endpoint. Data is always encrypted both at rest and in transit. Authentication uses Identity federation.

If you choose to forward logs to a different AWS region than the one your Aura instance resides in, the AWS Global Network backplane is used to transfer data without crossing the public internet.

Azure

When using log forwarding with Azure, all logs are transmitted to Azure Monitor of the source data center via the internal endpoint opinsights.azure.com. The traffic is TLS encrypted and authenticated with HMAC-SHA256.

If you choose to forward logs to a different Azure region than the one your Aura instance resides in, the Azure global network backplane is used to transfer data without crossing the public internet.