3.4.3. Neo4j deployments automation on Azure

This chapter describes how to automate Neo4j deployments on Azure.

Automate Neo4j deployment when you want to integrate Neo4j into your CI/CD pipeline to be able to create/destroy instances temporarily, or to spin up a sample instance.

3.4.3.1. Prerequisites

  • You have installed the Azure command-line interface.
  • You have installed jq tool for working with JSON responses. See the Download jq page.
  • You have authenticated your az CLI to be able to interact with your resource groups and use the right subscription by default. For more information on how to change the tool subscription, see the Azure CLI documentation.

3.4.3.2. Azure Resource Manager

Neo4j provides Azure Resource Manager (ARM) templates for Neo4j Enterprise standalone and Neo4j Causal Cluster (highly-available clusters).

ARM templates are a recipe that tells Azure how to deploy a whole set of interrelated resources. By deploying all of this as a stack you can keep all your resources together, and manage the entire instance by managing this resource group.

3.4.3.3. Creating an ARM deployment job

Depending on what Neo4j edition you want to deploy, you create ARM Deployment job by running a shell script.

Each script contains the following configurations:

  • The URL of the Neo4j stack template that tells Azure what to deploy.
  • Various parameters that control how much hardware you want to use.
  • VM_SIZE – the Azure VM type you want to launch, which controls how much hardware you will be using.
  • DISK_SIZE and DISK_TYPE – controls whether Neo4j uses standard spinning magnetic platters (pd-standard) or SSD disks (pd-ssd), and how many GB of storage you want to allocate.
  • LOCATION - specifies where to deploy Neo4j.
  • Authentication details - the administrative username and password for access to the VMs.

3.4.3.4. Deploying Neo4j Enterprise Causal Cluster

To deploy Neo4j Enterprise Causal Cluster, use the Causal Cluster template.

You indicate how many core servers and read replicas you want in your cluster by configuring the CORE_NODES and READ_REPLICAS parameters.

Take note of the TEMPLATE_BASE parameter, which contains the Neo4j version you want to launch. This can be adjusted to any version of Neo4j where there are published ARM templates. Create a simple JSON file with your deployment configurations and pass it to ARM. Based on your inputs, ARM produces a set of infrastructure as an output.

#!/bin/bash
export CORE_NODES=3
export READ_REPLICAS=0
export NEO4J_PASSWORD=s00pers3cR3T:
export ADMIN_AUTH_TYPE=password
export USERNAME=graph-hacker
export ADMIN_PASSWORD=s00pers3cR3T:
export VM_SIZE=Standard_B2ms
export DISK_TYPE=StandardSSD_LRS
export DISK_SIZE=256
export IP_ALLOCATION=Dynamic
export SEED=$(head -c 3 /dev/urandom | base64 | sed 's/[^a-zA-Z0-9]/X/g')
export RESOURCE_GROUP="neo4j-RG-${SEED}"
export CLUSTERNAME="neo4j-${SEED}"
export DEPLOYMENT=neo4j-bmdeploy
export LOCATION="East US"
# The ARM template to deploy.
export TEMPLATE_BASE=http://neo4j-arm.s3.amazonaws.com/3.5.16/causal-cluster/
export TEMPLATE_URL=${TEMPLATE_BASE}mainTemplate.json
echo $(cat <<JSON
{
"ClusterName": { "value": "${CLUSTERNAME}" },
"CoreNodes": { "value": ${CORE_NODES} },
"ReadReplicas": { "value": ${READ_REPLICAS} },
"VmSize": { "value": "${VM_SIZE}" },
"DataDiskType": { "value": "${DISK_TYPE}" },
"DataDiskSizeGB": { "value": ${DISK_SIZE} },
"AdminUserName": { "value": "${USERNAME}" },
"AdminAuthType": { "value": "${ADMIN_AUTH_TYPE}" },
"AdminCredential": { "value": "${ADMIN_PASSWORD}" },
"PublicIPAllocationMethod": { "value": "${IP_ALLOCATION}" },
"Neo4jPassword": { "value": "${NEO4J_PASSWORD}" },
"_artifactsLocation": { "value": "${TEMPLATE_BASE}" }
}
JSON
) > "${RESOURCE_GROUP}.json"
echo "Creating resource group named ${RESOURCE_GROUP}"
if ! az group create --name "${RESOURCE_GROUP}" --location "${LOCATION}"; then
   echo STACK_NAME=$RESOURCE_GROUP
   echo "Failed to create necessary resource group ${RESOURCE_GROUP}"
   exit 1
fi
echo "Creating deployment"
az group deployment create \
  --template-uri "$TEMPLATE_URL" \
  --parameters @./${RESOURCE_GROUP}.json \
  --resource-group "${RESOURCE_GROUP}" \
  --name "${DEPLOYMENT}"
if [ $? -ne 0 ] ; then
  echo STACK_NAME=$RESOURCE_GROUP
  echo "Stack deploy failed"
  exit 1
fi
# JSON Path to server response where the IP address is.
ADDR_FIELD=".[].virtualMachine.network.publicIpAddresses[0].ipAddress"
IP_ADDRESS=$(az vm list-ip-addresses --resource-group "${RESOURCE_GROUP}" | jq -r "$ADDR_FIELD" | head -n 1)
echo STACK_NAME=$RESOURCE_GROUP
echo NEO4J_URI=bolt+routing://$IP_ADDRESS:7687

As a result, a new resource group is created with all the assets, and you get a URI of a bolt endpoint you can use. Alternatively, go to https://<IP address>:7473/ to access Neo4j Browser for your new clustered instance.

3.4.3.5. Deploying Neo4j Enterprise Standalone

To deploy Neo4j Enterprise Standalone, create a simple VM and configure its firewall/security rules. It will not have high-availability failover capabilities, but it is a very fast way to get started.

Neo4j provides the VM through an Azure marketplace offer. To refer to the right VM image, you need to know the publisher (that’s Neo4j), the “offer” (Neo4j version series), and SKU (the particular Neo4j version). Because you are not using ARM for this deployment, the script polls and waits until the VM service comes up, and then changes the Neo4j default password. At the top, you can choose a different password for the neo4j user as for a system administrator. Make sure to customize the SUBSCRIPTION variable to make this work.

#!/bin/bash
export LOCATION=eastus
export SUBSCRIPTION=My-Subscription-Name
export RG=neo4j-standalone-RG
export NAME=neo4j-standalone
export ADMIN_USERNAME=graph-hacker
export ADMIN_PASSWORD=ch00se:A@PASSw0rd
export NEO4J_PASSWORD=ch00se:A@PASSw0rd
export NETWORK_SECURITY_GROUP=neo4j-nsg
# Options: https://azure.microsoft.com/en-us/pricing/details/virtual-machines/
export VM_SIZE=Standard_D2_v3
# Can change this to static if desired
export ADDRESS_ALLOCATION=dynamic
# Configuration bits of what you're launching
# Publisher:Offer:Sku:Version
export PUBLISHER=neo4j
export OFFER=neo4j-enterprise-3_5
export SKU=neo4j_3_5_5_apoc
export VERSION=latest
export IMAGE=$PUBLISHER:$OFFER:$SKU:$VERSION
echo "Creating resource group named $RG"
az group create --location $LOCATION \
   --name $RG \
   --subscription $SUBSCRIPTION
echo "Creating Network Security Group named $NETWORK_SECURITY_GROUP"
az network nsg create \
   --resource-group $RG \
   --location $LOCATION \
   --name $NETWORK_SECURITY_GROUP
echo "Assigning NSG rules to allow inbound traffic on Neo4j ports..."
prio=1000
for port in 7473 7474 7687; do
  az network nsg rule create \
    --resource-group $RG \
    --nsg-name "$NETWORK_SECURITY_GROUP" \
    --name neo4j-allow-$port \
    --protocol tcp \
    --priority $prio \
    --destination-port-range $port
  prio=$(($prio+1))
done
echo "Creating Neo4j VM named $NAME"
az vm create --name $NAME \
  --resource-group $RG \
  --image $IMAGE \
  --vnet-name $NAME-vnet \
  --subnet $NAME-subnet \
  --admin-username "$ADMIN_USERNAME" \
  --admin-password "$ADMIN_PASSWORD" \
  --public-ip-address-allocation $ADDRESS_ALLOCATION \
  --size $VM_SIZE
if [ $? -ne 0 ] ; then
  echo "VM creation failed"
  exit 1
fi
echo "Updating NIC to have your NSG"
# Uses default assigned NIC name
az network nic update \
  --resource-group "$RG" \
  --name "${NAME}VMNic" \
  --network-security-group "$NETWORK_SECURITY_GROUP"
# Get the IP address of our instance
IP_ADDRESS=$(az vm list-ip-addresses -g "$RG" -n "$NAME" | jq -r '.[0].virtualMachine.network.publicIpAddresses[0].ipAddress')
export NEO4J_URI=bolt://$IP_ADDRESS
# Change password
echo "Checking if Neo4j is up and changing password...."
while true; do
   if curl -s -I http://$IP_ADDRESS:7474 | grep "200 OK"; then
     echo "Neo4j is up; changing default password" 2>&1
     curl -v -H "Content-Type: application/json" \
       -XPOST -d '{"password":"'$NEO4J_PASSWORD'"}' \
       -u neo4j:neo4j \
       http://$IP_ADDRESS:7474/user/neo4j/password 2>&1
     echo "Password reset, signaling success" 2>&1
     break
   fi
   echo "Waiting for neo4j to come up" 2>&1
   sleep 1
done
echo NEO4J_URI=$NEO4J_URI
exit 0

3.4.3.6. Cleaning up and removing your deployment

When you are done with your deployment, you can delete the entire resource group by using the following script:

#!/bin/bash
if [ -z $1 ] ; then
  echo "Usage: call me with deployment name"
  exit 1
fi
STACK_NAME=$1
if [ -f "$STACK_NAME.json" ] ; then
   rm -f "$STACK_NAME.json"
fi
az group delete -n "$STACK_NAME" --no-wait --yes
exit $?