Schema

This is the public preview documentation of Neo4j Virtual Graph. To provide feedback, please use the feedback form you were granted access to.

During the public preview, we advise you not to use sensitive or production data with Virtual Graph.

The file schema.json defines how the relational tables, their columns and datatypes, and their primary keys are mapped to Neo4j entities (nodes and relationships). The schema is loaded during boot time and cannot be refreshed without a restart of the Neo4j server.

An empty schema looks like this:

{
  "catalog" : "name_of_the_relational_catalog",
  "schema" : "name_of_the_relational_schema",
  "entities" : {
    "nodes" : [],
    "relationships" : []
  }
}

The entities field is not required for defining an empty schema. It has been included for illustration purposes: the mapping clearly distinguishes between tables and entities, which are associated with relational and graph databases respectively.

The following example assumes :People and :Movie nodes and the :ACTED_IN relationship in the movie graph. Consider this relational schema:

CREATE TABLE IF NOT EXISTS people
(
    id   INTEGER PRIMARY KEY,
    name VARCHAR(32) NOT NULL,
    born SMALLINT
);

CREATE TABLE IF NOT EXISTS movies
(
    id       INTEGER PRIMARY KEY,
    title    VARCHAR(64) NOT NULL,
    tagline  VARCHAR(256),
    released SMALLINT
);

CREATE TABLE IF NOT EXISTS movie_actors
(
    person_id INTEGER NOT NULL REFERENCES people (id),
    movie_id  INTEGER     NOT NULL REFERENCES movies (id),
    role      VARCHAR(64) NOT NULL,
    PRIMARY KEY (person_id, movie_id, role)
);

The original graph contains multiple roles for several :ACTED_IN entities. In the relational model this is amounts to two 1:n relationships: The same person or movie can appear multiple times in the movie_actors table:

erDiagram movie_actors 1+--1 people : has_role movie_actors 1+--1 movies : in movie_actors { string role PK }

The foreign keys of people and movies in movie_actors as well as the role property constitute the primary key of the table.

These are mapped to two nodes, :People and :Movie, and the relationship :ACTED_IN, with some renaming of relational columns:

{
  "catalog": "name_of_the_relational_catalog",
  "schema": "name_of_the_relational_schema",
  "entities": {
    "nodes": [
      {
        "label": "Movie",
        "table": "MOVIES",
        "properties": [
          {
            "name": "release_year",
            "column": "RELEASED",
            "type": "INTEGER"
          },
          {
            "name": "tagline",
            "column": "TAGLINE",
            "type": "STRING"
          },
          {
            "name": "title",
            "column": "TITLE",
            "type": "STRING"
          }
        ],
        "key": [
          {
            "column": "ID"
          }
        ]
      },
      {
        "label": "Person",
        "table": "PEOPLE",
        "properties": [
          {
            "name": "name",
            "column": "NAME",
            "type": "STRING"
          },
          {
            "name": "born",
            "column": "BORN",
            "type": "INTEGER"
          }
        ],
        "key": [
          {
            "column": "ID"
          }
        ]
      }
    ],
    "relationships": [
      {
        "label": "ACTED_IN",
        "table": "MOVIE_ACTORS",
        "start": {
          "targetEntity": "Person",
          "keys": [
            {
              "nodeColumn": "ID",
              "relationshipColumn": "PERSON_ID"
            }
          ]
        },
        "end": {
          "targetEntity": "Movie",
          "keys": [
            {
              "nodeColumn": "ID",
              "relationshipColumn": "MOVIE_ID"
            }
          ]
        },
        "properties": [
          {
            "name": "rolle",
            "column": "ROLE",
            "type": "String"
          }
        ],
        "key": [
          {
            "column": "PERSON_ID"
          },
          {
            "column": "MOVIE_ID"
          },
          {
            "column": "ROLE"
          }
        ]
      }
    ]
  }
}

  • entities are a map of two arrays, nodes and relationships

  • both types expose:

    • label1: The Neo4j entity label (often called type for relationships)

    • table1: The backing relational table

    • properties: An array of properties, can be absent or empty. In both cases the resulting entity acts as node as if created with CREATE (n:Label) RETURN n. A property object requires the presence of three keys:

      • column: The name of the column in the backing table

      • name: The name of the property as it will appear in the graph

      • type: A suggested graph type

    • key1: An array of column objects that constitutes the identifier for the entity. This may or may not be the primary key, simple or composite, of the backing table, but it must be unique (that is, it might be enforced on the relational table with an additional unique key constrained)

    • The column object has one entry named column with the value of the backing column for the key in the backing table.

  • The relationship type additionally provides start1 and end1 objects. They are themselves entities (node objects), which is why it makes sense to point to entities instead of tables, as those are defined by the target entities. The start and end objects consists of:

    • targetEntity1: A string value containing the label of a node entity

    • keys1: An array of reference key objects:

      • nodeColumn1: The column inside the table backing the target node (must be part of the nodes key)

      • relationshipColumn1: The column inside the table backing this relationship (must be part of the relationships key)

1 indicates required fields, which may neither be absent nor empty.

During startup the key property is validated against the primary key which can be retrieved from JDBC metadata for the backing table. If the primary key of the table is build from a different set of columns, and you cannot verify that all key columns are at least part of a unique key, Virtual Graph issues a warning.

The foreign key constructs in the relationships is also validated.

A full JSON schema for the definition is available in the Appendix.

You can get an instance of com.neo4j.graphengine.schema.spi.SchemaRepository via Neo4j services or directly via Java’s unwrapped service loader:

var repository = ServiceLoader.load(SchemaRepository.class).findFirst().orElseThrow();

You can find several overloads of fromJson in the graph-engine repository. They can read this structure into a Java object of type Schema.

The same object can be created empty and used as a mutable builder:

var repository = ServiceLoader.load(SchemaRepository.class).findFirst().orElseThrow();
var schema = repository.empty("hive", "various").unwrap(MutableSchema.class);
// define your entities
schema.withEntity(entity);

This is helpful for creating test data outside data importer modeller.