Known issues and limitations

This page is the canonical place to check for the current limitations of the Neo4j Connector for Apache Spark. Before opening a support ticket or a GitHub issue, check whether the behavior you are seeing is listed here.

Each entry describes the limitation and the recommended workaround, if any.

Unless stated otherwise, entries apply to all actively supported versions of the connector. For questions that are not about limitations, see FAQ and troubleshooting.

Writing

ErrorIfExists is not supported for relationship source and target nodes

The relationship.source.save.mode and relationship.target.save.mode options do not accept ErrorIfExists. The write fails during validation with the following message:

This connector does not support save mode 'ErrorIfExists'. Use save mode 'Append' instead.

Workaround: use Append, Overwrite, or Match for the source and target save modes. See Node save modes for details.

Schema optimization options do not work with the query option

The schema.optimization.* options are ignored when writing with a custom Cypher® query.

Workaround: create indexes and constraints manually with the script options. See Schema optimization for details.

Reading

Cast errors when a property has different types across nodes

Reading nodes whose property has different types (for example, a String on some nodes and a Long on others) can fail with an error such as the following:

java.lang.ClassCastException: org.apache.spark.unsafe.types.UTF8String cannot be cast to java.lang.Long

Cause: Spark is not schema-free and requires each column to have a single type, while Neo4j allows the same property to hold different types on different nodes.

Workaround: install APOC on the Neo4j instance. This removes the error, but all values of that property are cast to String. Alternatively, define the schema explicitly. See Schema inference and Define a schema.

A graph with name [name] already exists when creating a GDS projection

Creating a GDS graph projection through the connector can fail with the following error:

Failed to invoke procedure gds.graph.create.cypher:
Caused by: java.lang.IllegalArgumentException: A graph with name [name] already exists

Cause: the connector runs the query twice — once to infer the DataFrame schema and once to fetch the data — so the projection is created twice.

Workaround: provide a user-defined schema, which skips the schema inference run.

Schema inference falls back to String columns on empty results

Schema inference samples the data to determine column types. If the query returns no rows, sampling is not possible and the connector builds the schema from the RETURN statement with every column typed as String.

This is expected behavior and causes no issues, because the result set is empty. See Schema inference for details.

Graph Data Science

mutate and write procedure modes are not supported

GDS procedures in mutate or write mode are rejected with the following message:

You cannot execute GDS mutate or write procedure in a read query

Cause: these modes do not return data that can be mapped to a DataFrame.

Workaround: use the stream mode, join the resulting DataFrames in Spark, and write the results back to Neo4j with the connector. See Data science with Neo4j GDS for examples.

GDS queries support a single partition only

GDS reads cannot be partitioned. Setting partitions to a value greater than 1 fails with the following message:

For GDS queries we support only one partition

Workaround: leave partitions at its default value of 1 for GDS queries.

Databricks

This page includes instructions on the usage of a third-party platform, which may be subject to changes beyond our control. In case of doubt, refer to the third-party platform documentation.

Shared access modes are not supported

The connector only works on Databricks compute clusters created with the Single user access mode and the Unrestricted policy. Shared and no-isolation-shared access modes are not supported.

Workaround: create a dedicated Single user cluster for jobs that use the connector. See Set up a compute cluster.

Unity Catalog is supported in Single user access mode only

Unity Catalog support follows the same restriction as the cluster access mode: Neo4j supports the Unity Catalog in Single user access mode only.

Refer to the Databricks access mode limitations for further information.

Report a new issue

If you hit a limitation that is not listed on this page: