Aura Graph Analytics for non-Neo4j data sources

Open In Colab

This Jupyter notebook is hosted here in the Neo4j Graph Data Science Client Github repository.

The notebook shows how to use the graphdatascience Python library to create, manage, and use an Aura Graph Analytics (AGA) Session.

We consider a graph of people and fruits, which we’re using as a simple example to show how to load data from Pandas DataFrame to an AGA Session, run algorithms, and inspect the results. We will cover all management operations: creation, listing, and deletion.

If you are using AuraDB, follow this example. If you are using a self-managed Neo4j instance, follow this example.

1. Prerequisites

This notebook requires having the Aura Graph Analytics feature enabled for your Neo4j Aura project.

You also need to have the graphdatascience Python library installed, version 1.15 or later.

%pip install "graphdatascience>=1.15" python-dotenv "neo4j_viz[gds]"
from dotenv import load_dotenv

# This allows to load required secrets from `.env` file in local directory
# This can include Aura API Credentials. If file does not exist this is a noop.
load_dotenv(".env")

2. Aura API credentials

The entry point for managing GDS Sessions is the GdsSessions object, which requires creating Aura API credentials.

import os

from graphdatascience.session import AuraAPICredentials, GdsSessions

# you can also use AuraAPICredentials.from_env() to load credentials from environment variables
api_credentials = AuraAPICredentials(
    client_id=os.environ["CLIENT_ID"],
    client_secret=os.environ["CLIENT_SECRET"],
    # If your account is a member of several project, you must also specify the project ID to use
    project_id=os.environ.get("PROJECT_ID", None),
)

sessions = GdsSessions(api_credentials=api_credentials)

3. Creating a new session

A new session is created by calling sessions.get_or_create() with the following parameters:

  • A session name, which lets you reconnect to an existing session by calling get_or_create again.

  • The session memory.

  • The cloud location.

  • A time-to-live (TTL), which ensures that the session is automatically deleted after being unused for the set time, to avoid incurring costs.

See the API reference documentation or the manual for more details on the parameters.

from graphdatascience.session import AlgorithmCategory, CloudLocation, SessionMemory

# Estimate the memory needed for the GDS session
memory = sessions.estimate(
    node_count=20,
    relationship_count=50,
    algorithm_categories=[AlgorithmCategory.CENTRALITY, AlgorithmCategory.NODE_EMBEDDING],
)

print(f"Estimated memory: {memory}")

# Explicitly define the size of the session
memory = SessionMemory.m_2GB

# Specify your cloud location
cloud_location = CloudLocation("gcp", "europe-west1")

# You can find available cloud locations by calling
cloud_locations = sessions.available_cloud_locations()
print(f"Available locations: {cloud_locations}")
from datetime import timedelta

# Create a GDS session!
gds = sessions.get_or_create(
    # we give it a representative name
    session_name="people-and-fruits-standalone",
    memory=memory,
    ttl=timedelta(minutes=30),
    cloud_location=cloud_location,
)

4. Listing sessions

You can use sessions.list() to see the details for each created session.

from pandas import DataFrame

gds_sessions = sessions.list()

# for better visualization
DataFrame(gds_sessions)

5. Projecting a dataset

AGA sessions always start empty, with no data. So our first step will be to project data into the session. In this example, we will illustrate how to do this using Pandas DataFrames.

Many systems offer ways to read data into Pandas DataFrames, enabling these systems to be used as data sources for AGA. For simplicity, we will define the DataFrames used in this notebook by hand.

import pandas as pd

people_df = pd.DataFrame(
    [
        {"nodeId": 0, "name": "Dan", "age": 18, "experience": 63, "hipster": 0},
        {"nodeId": 1, "name": "Annie", "age": 12, "experience": 5, "hipster": 0},
        {"nodeId": 2, "name": "Matt", "age": 22, "experience": 42, "hipster": 0},
        {"nodeId": 3, "name": "Jeff", "age": 51, "experience": 12, "hipster": 0},
        {"nodeId": 4, "name": "Brie", "age": 31, "experience": 6, "hipster": 0},
        {"nodeId": 5, "name": "Elsa", "age": 65, "experience": 23, "hipster": 0},
        {"nodeId": 6, "name": "Bobby", "age": 38, "experience": 4, "hipster": 1},
        {"nodeId": 7, "name": "John", "age": 4, "experience": 100, "hipster": 0},
    ]
)
people_df["labels"] = "Person"

fruits_df = pd.DataFrame(
    [
        {"nodeId": 8, "name": "Apple", "tropical": 0, "sourness": 0.3, "sweetness": 0.6},
        {"nodeId": 9, "name": "Banana", "tropical": 1, "sourness": 0.1, "sweetness": 0.9},
        {"nodeId": 10, "name": "Mango", "tropical": 1, "sourness": 0.3, "sweetness": 1.0},
        {"nodeId": 11, "name": "Plum", "tropical": 0, "sourness": 0.5, "sweetness": 0.8},
    ]
)
fruits_df["labels"] = "Fruit"

like_relationships = [(0, 8), (1, 9), (2, 10), (3, 10), (4, 9), (5, 11), (7, 11)]
likes_df = pd.DataFrame([{"sourceNodeId": src, "targetNodeId": trg} for (src, trg) in like_relationships])
likes_df["relationshipType"] = "LIKES"

knows_relationship = [(0, 1), (0, 2), (1, 2), (1, 3), (1, 4), (2, 5), (7, 3)]
knows_df = pd.DataFrame([{"sourceNodeId": src, "targetNodeId": trg} for (src, trg) in knows_relationship])
knows_df["relationshipType"] = "KNOWS"

6. Construct Graph from DataFrames

With DataFrames in hand, the next step is to build a graph from them. We do that by using the gds.graph.construct() function.

After calling this function, we will get a Graph Object back, representing the graph that now exists within the AGA session. We will use it as input to the various algorithms that we will run on the graph.

# Dropping `name` column as GDS does not support string properties
nodes = [people_df.drop(columns="name"), fruits_df.drop(columns="name")]
relationships = [likes_df, knows_df]

G = gds.graph.construct("people-fruits", nodes, relationships)
str(G)
# Let us visualize the projected graph
from neo4j_viz.gds import from_gds

VG = from_gds(gds, G)

# Concatenate the dataframes with name and nodeId columns
names = pd.concat([people_df[["nodeId", "name"]], fruits_df[["nodeId", "name"]]])
# Create a dictionary mapping nodeId to name
names_mapping = names.set_index("nodeId")["name"]

for node in VG.nodes:
    node.caption = names_mapping[node.id]

VG.render(initial_zoom=1.2)

7. Running Algorithms

You can run algorithms on the constructed graph using the standard GDS Python Client API. See the other tutorials for more examples.

print("Running PageRank ...")
pr_result = gds.pageRank.mutate(G, mutateProperty="pagerank")
print(f"Compute millis: {pr_result['computeMillis']}")
print(f"Node properties written: {pr_result['nodePropertiesWritten']}")
print(f"Centrality distribution: {pr_result['centralityDistribution']}")

print("Running FastRP ...")
frp_result = gds.fastRP.mutate(
    G,
    mutateProperty="fastRP",
    embeddingDimension=8,
    featureProperties=["pagerank"],
    propertyRatio=0.2,
    nodeSelfInfluence=0.2,
)
print(f"Compute millis: {frp_result['computeMillis']}")
# stream back the results
result = gds.graph.nodeProperties.stream(G, ["pagerank", "fastRP"], separate_property_columns=True)

result

To resolve each nodeId to name, we can merge it back with the source data frames.

names = pd.concat([people_df, fruits_df])[["nodeId", "name"]]
result.merge(names, how="left")

8. Deleting the session

After the analysis is done, you can delete the session. As this example is not connected to a Neo4j DB, you need to make sure the algorithm results are persisted on your own.

Deleting the session will release all resources associated with it, and stop incurring costs.

# or gds.delete()
sessions.delete(session_name="people-and-fruits-standalone")
# let's also make sure the deleted session is truly gone:
sessions.list()