Quickstart: BigQuery

This is the public preview documentation of Neo4j Virtual Graph. To provide feedback, please use the feedback form you were granted access to.

During the public preview, we advise you not to use sensitive or production data with Virtual Graph.

This page presents the general workflow of virtualizing an example dataset from BigQuery into Neo4j Aura with Virtual Graph.

Requirements

Preparing BigQuery

Create a service account key

BigQuery service-account authentication uses a JSON key file.

  1. In the Google Cloud console, select the project that contains your BigQuery dataset.

  2. Go to IAM & Admin → Service Accounts and create a service account for Virtual Graph, or select an existing service account.

  3. Grant the service account the following roles:

    • bigquery.dataViewer

    • bigquery.readSessionUser

    • bigquery.jobUser

    • bigquery.user

  4. Open the service account, select Keys → Add key → Create new key, choose JSON, and select Create.

  5. Store the downloaded JSON key securely. It contains a private key; do not share it or commit it to source control.

For details, see the Google Cloud documentation on creating service accounts and creating service account keys.

Virtual Graph only ever requires read access. You can and should restrict the service account to read privileges. It is worthwhile to consider additional access limitations, for example, limit the service account to certain database tables. Virtual Graph does not account for that - any user with access to Virtual Graph has access to everything the graph has access to.

Creating a Virtual Graph

In the Aura console:

  1. Select Instances from the left-side navigation.

  2. Select the Virtual Graphs tab and proceed with Create virtual graph.

  3. In the Configure Virtual Graph step, select a name, a cloud provider and a memory volume for your Virtual Graph.

Connect BigQuery as the data source

  1. Use Add new data source and select BigQuery.

  2. Complete the form.

    1. Assign your data source a name.

    2. Enter the Google Cloud Project ID and BigQuery dataset name that contain your Movies data.

    3. Paste the full contents of the service account key JSON file created in Create a service account key into Service Account Key JSON.

  3. Proceed with Next and wait for your connection to be verified, then Confirm.

Select a graph model

  1. Under Select graph model, proceed with Create new graph model.

  2. When the model has been prepared, select Generate from schema.

Inspect and complete your graph model

The inferred schema likely doesn’t fully reflect the tables from your BigQuery data source.

For instance, A model generated from schema with errors shows a generated model that is missing two relationships (FOLLOWS and ACTED_IN). This is also indicated by the missing dots for the fields under FOLLOWERS and MOVIE_ACTORS in the Data source panel. The existing node and relationship labels may not be suitable either: the example model uses PEOPLE and MOVIES instead of Person and Movie as well as the same type for all relationships. The relationship directions are reversed and should go from Person nodes to Movie nodes instead. Finally, selecting the nodes and relationships may reveal that IDs or fields are missing.

The following steps are required to fix the model:

  1. Select the nodes and relationships and change the Label or Relationship type accordingly.

  2. Make sure that all fields are set for nodes and relationships. Map from table → Select all infers them when you have selected a table for the node or relationship.

  3. Select relationships with wrong direction and reverse it with the Reverse relationship direction button in the tool bar on the bottom of the model panel.

  4. For missing relationships:

    1. Select the node or nodes you want to connect.

    2. Add a new relationship with the Add new relationship button (two circles, a connected line and a plus).

    3. Assign a Relationship type.

    4. Select the appropriate table under Properties and add missing properties.

    5. Add From and To references under Node ID mapping.

    6. Assign IDs as necessary.

  5. Field names can be edited, and data types assigned.

Relationship types and node labels must be unique, see Entity type uniqueness.

Your model should look like A corrected model for MOVIES.

Finish with Create Virtual Graph and download the credential data.

Inspect your graph

Select Query from the left-side navigation and query your graph, for example with the following query:

MATCH (p:Person)-[r:ACTED_IN]->(m:Movie)
RETURN p, r, m LIMIT 10
Cypher query against a virtual graph
Figure 3. Cypher query against a virtual graph

You have successfully created a virtual graph.

Keep learning