2.4. Memory Requirements

This section describes memory requirements for the projected graph model used by the Neo4j Graph Algorithms Library.

The graph algorithms library operates completely on the heap, which means we’ll need to configure our Neo4j Server with a much larger heap size than we would for transactional workloads. The diagram belows shows how memory is used by the projected graph model:

graph model memory

The model contains three types of data:

Memory configuration depends on the graph projection that we’re using.

This section includes:

2.4.1. algo.memrec procedure

The algo.memrec procedure returns an estimate of the amount of memory required to run graph algorithms.

Syntax. 

CALL algo.memrec(label, relationship, algorithm, config)

Table 2.1. Parameters
Name Type Default Optional Description

label

string

null

yes

The label to load from the graph. If null, load all nodes.

relationship

string

null

yes

The relationship type to load from the graph. If null, load all relationships.

algorithm

string

null

yes

The name of a graph algorithm or graph.load to return the memory requirements for the in memory graph only.

Config

       

direction

string

'OUTGOING'

yes

The relationship-direction to use in the algorithm.

graph

string

'huge'

yes

Use 'huge' when describing the subset of the graph with label and relationship-type parameter. Use 'cypher' for describing the subset with cypher node statement and relationship statement.

concurrency

int

available CPUs

yes

The number of concurrent threads.

Algorithm specific parameters can also be provided as config.

Table 2.2. Results
Name Type Description

requiredMemory

int

The required memory in human readable format.

bytesMin

int

The minimum number of bytes required.

bytesMax

int

The maximum number of bytes required.

mapView

Map

A breakdown of the memory used by different components involved in the execution of a graph algorithm.

algo.memrec(label, relationship, algorithm, config) maps to algo.<algorithm>.memrec(label, relationship, config), which means that CALL algo.memrec(null, null, "pageRank") is equivalent to CALL algo.pageRank.memrec(null, null)

2.4.2. algo.memrec sample

This sample will explain the algo.memrec procedure, using a Game of Thrones graphs.

The following creates a graph containing Game of Thrones characters and the interactions between them. 

CREATE CONSTRAINT ON (c:Character) ASSERT c.id IS UNIQUE;

UNWIND range(1,7) AS season
LOAD CSV WITH HEADERS FROM "https://github.com/neo4j-apps/neuler/raw/master/sample-data/got/got-s" + season + "-nodes.csv" AS row
MERGE (c:Character {id: row.Id})
ON CREATE SET c.name = row.Label;

UNWIND range(1,7) AS season
LOAD CSV WITH HEADERS FROM "https://github.com/neo4j-apps/neuler/raw/master/sample-data/got/got-s" + season + "-edges.csv" AS row
MATCH (source:Character {id: row.Source})
MATCH (target:Character {id: row.Target})
CALL apoc.merge.relationship(source, "INTERACTS_SEASON" + season, {}, {}, target) YIELD rel
SET rel.weight = toInteger(row.Weight);

2.4.2.1. Huge graph projection

This is the default graph projection, and supports graphs up to 35 trillion nodes and relationships.

The following computes the amount of memory required by the huge in memory graph for any node label and any relationship type: 

CALL algo.memrec(null, null, "graph.load", {graph: "huge"})
YIELD nodes, relationships, requiredMemory, bytesMin, bytesMax
RETURN nodes, relationships, requiredMemory, bytesMin, bytesMax

Table 2.3. Results
Nodes Relationships Required Memory Bytes Min Bytes Max

400

3551

299 KiB

306688

306688

The following computes the amount of memory required by the huge in memory graph for node label Character and relationship type INTERACTS_SEASON1

CALL algo.memrec("Character", "INTERACTS_SEASON1", "graph.load", {graph: "huge"})
YIELD nodes, relationships, requiredMemory, bytesMin, bytesMax
RETURN nodes, relationships, requiredMemory, bytesMin, bytesMax

Table 2.4. Results
Nodes Relationships Required Memory Bytes Min Bytes Max

400

550

299 KiB

306680

306680

The following computes the amount of memory required by the huge in memory graph for node label Character, relationship type INTERACTS_SEASON1, and direction BOTH

CALL algo.memrec("Character", "INTERACTS_SEASON1", "graph.load", {direction: "BOTH", graph: "huge"})
YIELD nodes, relationships, requiredMemory, bytesMin, bytesMax
RETURN nodes, relationships, requiredMemory, bytesMin, bytesMax

Table 2.5. Results
Nodes Relationships Required Memory Bytes Min Bytes Max

400

550

563 KiB

577112

577112

2.4.2.2. Cypher graph projection

This graph projection supports graphs up to 2 billion nodes and 2 billion relationships, specfied by Cypher queries that return node ids.

algo.memrec executes the provided Cypher queries to compute the number of nodes and relationships in the projected graph. The results from running the procedure for Cypher graph projections will therefore not be immediate, as they will be with other graph projections.

The following computes the amount of memory required by the in memory graph for all nodes and relationships: 

CALL algo.memrec(
  "MATCH (n) RETURN id(n) AS id",
  "MATCH (a)-->(b) RETURN id(a) AS source, id(b) as target",
  "graph.load",
  {graph: "cypher"}
)
YIELD nodes, relationships, requiredMemory, bytesMin, bytesMax
RETURN nodes, relationships, requiredMemory, bytesMin, bytesMax

Table 2.6. Results
Nodes Relationships Required Memory Bytes Min Bytes Max

400

3551

40 KiB

41456

41456

The following computes the amount of memory required by the huge in memory graph for node label Character and relationship type INTERACTS_SEASON1

CALL algo.memrec(
  "MATCH (n:Character) RETURN id(n) AS id",
  "MATCH (a:Character)-[:INTERACTS_SEASON1]->(b) RETURN id(a) AS source, id(b) as target",
  "graph.load",
  {graph: "cypher"}
)
YIELD nodes, relationships, requiredMemory, bytesMin, bytesMax
RETURN nodes, relationships, requiredMemory, bytesMin, bytesMax

Table 2.7. Results
Nodes Relationships Required Memory Bytes Min Bytes Max

400

3551

40 KiB

41456

41456

2.4.2.3. Named Graph

The following creates a huge in memory graph named my-graph for the node label Character and relationship type INTERACTS_SEASON3

CALL algo.graph.load("my-graph6", "Character", "INTERACTS_SEASON3")
YIELD name, graph, direction, nodes, relationships
RETURN name, graph, direction, nodes, relationships

Table 2.8. Results
Name Graph Direction Nodes Relationships

my-graph

huge

OUTGOING

400

504

The following computes the amount of memory being used by the graph named my-graph

CALL algo.memrec(null, null, "graph.load", {graph: "my-graph"})
YIELD nodes, relationships, requiredMemory, bytesMin, bytesMax
RETURN nodes, relationships, requiredMemory, bytesMin, bytesMax

Table 2.9. Results
Nodes Relationships Required Memory Bytes Min Bytes Max

400

504

27 KiB

28656

28656

2.4.3. Memory Requirements on Sample Graphs

Below are the memory requirements for various sample datasets. These numbers are computed by running the following queries:

Huge Graph. 

CALL algo.memrec(null, null, "graph.load", {graph: "huge"})

2.4.3.1. Pokec

This dataset contains people and friends relationships from Pokec, the most popular online social network in Slovakia.

Table 2.10. Memory Usage
Graph Type Nodes Relationships Required Memory Bytes Min Bytes Max

Huge

1,632,803

30,622,564

[74 MiB…​130 MiB]

78560176

137284912

2.4.3.2. Dbpedia

This dataset contains Wikipedia pages and the links between them. Instructions for importing this dataset are available from jexp/graphipedia.

Table 2.11. Memory Usage
Graph Type Nodes Relationships Required Memory Bytes Min Bytes Max

Huge

11,474,730

116,601,029

[438 MiB…​657 MiB]

459722704

689148984

2.4.3.3. Twitter 2010

This dataset contains users and followers from a crawl of Twitter presented by Haewoon Kwak, Changhyun Lee, Hosung Park, and Sue Moon in “What is Twitter, a Social Network or a News Media?”

Table 2.12. Memory Usage
Graph Type Nodes Relationships Required Memory Bytes Min Bytes Max

Huge

41,652,230

1,468,365,182

[2582 MiB…​5363 MiB]

2708130496

5623918504

2.4.3.4. Friendster

This dataset contains people and friends relationships from Friendster, the online gaming network.

Table 2.13. Memory Usage
Graph Type Nodes Relationships Required Memory Bytes Min Bytes Max

Huge

65,608,366

1,806,067,135

[3567 MiB…​8635 MiB]

3740896712

9055485320