Arrow Client

class graphdatascience.arrow_client.v2.gds_arrow_client.GdsArrowClient
__init__(flight_client: AuthenticatedArrowClient)

Creates a new GdsArrowClient instance.

Parameters:

flight_client (AuthenticatedArrowClient) – The authenticated flight client to use for communication with the GDS server. Ownership of the client is transferred to this GdsArrowClient.

abort_job(job_id: str) → None

Aborts the specified process

Parameters:

job_id (str) – The job id of the process

Return type:

None

advertised_connection_info() → ConnectionInfo

Returns the host and port of the GDS Arrow server.

Returns:

the host and port of the GDS Arrow server

Return type:

ConnectionInfo

create_graph(graph_name: str, *, undirected_relationship_types: list[str] | None = None, inverse_indexed_relationship_types: list[str] | None = None, concurrency: int | None = None, log_progress: bool = True, job_id: str | None = None) → str

Starts a new graph import process on the GDS server.

The import process accepts separate node and relationship stream uploads.

Parameters:

  • graph_name (str) – The name used to identify the graph in the catalog

  • undirected_relationship_types (list[str] | None) – A list of relationship types that should be treated as undirected

  • inverse_indexed_relationship_types (list[str] | None) – A list of relationship types that should be indexed in reverse direction as well

  • concurrency (int | None) – Number of concurrent threads to use.

  • log_progress (bool) – Display progress logging.

  • job_id (str | None) – Identifier for the computation.

Returns:

The JobId identifying the import process

Return type:

str

create_graph_from_triplets(graph_name: str, *, undirected_relationship_types: list[str] | None = None, inverse_indexed_relationship_types: list[str] | None = None, concurrency: int | None = None, log_progress: bool = True, job_id: str | None = None) → str

Starts a new graph import process on the GDS server.

The import process accepts triplets as input data.

Parameters:

  • graph_name (str) – The name used to identify the graph in the catalog

  • undirected_relationship_types (list[str] | None) – A list of relationship types that should be treated as undirected

  • inverse_indexed_relationship_types (list[str] | None) – A list of relationship types that should be indexed in reverse direction as well

  • concurrency (int | None) – Number of concurrent threads to use.

  • log_progress (bool) – Display progress logging.

  • job_id (str | None) – Identifier for the computation.

Returns:

The JobId identifying the import process

Return type:

str

flight_client() → AuthenticatedArrowClient

Returns the underlying flight client.

Return type:

AuthenticatedArrowClient

get_node_properties(graph_name: str, node_properties: str | list[str], node_labels: list[str] | None = None, list_node_labels: bool = False, concurrency: int | None = None, log_progress: bool = True, job_id: str | None = None) → str

Start a node property export process.

Parameters:

  • graph_name (str) – Name of the graph to be created

  • node_properties (str | list[str]) – Names of the node properties to retrieve

  • node_labels (list[str] | None) – A list of node labels to filter the nodes

  • list_node_labels (bool) – A flag that indicates whether the node labels should be included in the result

  • concurrency (int | None) – Number of threads used on the server side when serving the data

  • log_progress (bool) – Display progress logging.

  • job_id (str | None) – Identifier for the computation.

Returns:

The JobId identifying the export process

Return type:

str

get_nodes(graph_name: str, *, node_filter: str | None = None, log_progress: bool = True, concurrency: int | None = None, job_id: str | None = None) → str

Start a new export process to stream the nodes that match the filter from the graph.

Parameters:

  • graph_name (str) – Name of the graph to be created

  • node_filter (str | None) – A Cypher predicate for filtering nodes in the input graph.

  • log_progress (bool) – Display progress logging.

  • concurrency (int | None) – Number of concurrent threads to use.

  • job_id (str | None) – Identifier for the computation.

Returns:

The JobId identifying the export process

Return type:

str

get_relationships(graph_name: str, relationship_types: list[str] = ['*'], relationship_properties: list[str] | None = None, *, concurrency: int | None = None, log_progress: bool = True, job_id: str | None = None) → str

Start a new export process to stream all relationships of the specified types with the specified properties.

Parameters:

  • graph_name (str) – The graph name.

  • relationship_types (list[str]) – Filter the graph using the given relationship types. Relationships with any of the given types will be included.

  • relationship_properties (list[str] | None) – The relationship properties to stream. If not specified, no properties will be streamed.

  • concurrency (int | None) – Number of concurrent threads to use.

  • log_progress (bool) – Display progress logging.

  • job_id (str | None) – Identifier for the computation.

Returns:

The JobId identifying the export process

Return type:

str

job_status(job_id: str) → JobStatus

Retrieves the status of a job.

Parameters:

job_id (str) – The job id of the process

Returns:

The status of the job

Return type:

JobStatus

job_summary(job_id: str) → dict[str, Any]

Retrieves the summary of a job.

Parameters:

job_id (str) – The job id of the process

Returns:

The summary of the job

Return type:

dict[str, Any]

node_load_done(job_id: str) → None

Notifies the server that all node data has been sent.

Parameters:

job_id (str) – The job id of the import process

Return type:

None

relationship_load_done(job_id: str) → None

Notifies the server that all relationship data has been sent.

This will trigger the finalization of the import process and make the graph or database available.

Parameters:

job_id (str) – The job id of the import process

Return type:

None

request_token() → str | None

Requests a token from the server and returns it.

Returns:

a token from the server and returns it.

Return type:

str | None

stream_job(job_id: str) → DataFrame

Streams the results of a previously started job.

Parameters:

job_id (str) – Identifier for the computation.

Returns:

A pandas DataFrame containing the results of the job.

Return type:

pandas.DataFrame

triplet_load_done(job_id: str) → None

Notifies the server that all triplet data has been sent.

This will trigger the finalization of the import process and make the graph available in the graph catalog.

Parameters:

job_id (str) – The job id of the import process

Return type:

None

upload_nodes(job_id: str, data: ~pyarrow.lib.Table | list[~pyarrow.lib.RecordBatch] | ~pandas.DataFrame | list[~pandas.DataFrame], batch_size: int = 10000, progress_callback: ~graphdatascience.arrow_client.progress_callback.ProgressCallback = <function GdsArrowClient.<lambda>>, termination_flag: ~graphdatascience.query_runner.termination_flag.TerminationFlag | None = None) → None

Uploads node data to the server for a given job.

Parameters:

  • job_id (str) – The job id of the import process

  • data (Table | list[RecordBatch] | DataFrame | list[DataFrame]) – The data to upload

  • batch_size (int) – The number of rows per batch

  • progress_callback (ProgressCallback) – A callback function that is called with the number of rows uploaded after each batch

  • termination_flag (TerminationFlag | None) – A termination flag to cancel the upload if requested

Return type:

None

upload_relationships(job_id: str, data: ~pyarrow.lib.Table | list[~pyarrow.lib.RecordBatch] | ~pandas.DataFrame | list[~pandas.DataFrame], batch_size: int = 10000, progress_callback: ~graphdatascience.arrow_client.progress_callback.ProgressCallback = <function GdsArrowClient.<lambda>>, termination_flag: ~graphdatascience.query_runner.termination_flag.TerminationFlag | None = None) → None

Uploads relationship data to the server for a given job.

Parameters:

  • job_id (str) – The job id of the import process

  • data (Table | list[RecordBatch] | DataFrame | list[DataFrame]) – The data to upload

  • batch_size (int) – The number of rows per batch

  • progress_callback (ProgressCallback) – A callback function that is called with the number of rows uploaded after each batch

  • termination_flag (TerminationFlag | None) – A termination flag to cancel the upload if requested

Return type:

None

upload_triplets(job_id: str, data: ~pyarrow.lib.Table | list[~pyarrow.lib.RecordBatch] | ~pandas.DataFrame, batch_size: int = 10000, progress_callback: ~graphdatascience.arrow_client.progress_callback.ProgressCallback = <function GdsArrowClient.<lambda>>, termination_flag: ~graphdatascience.query_runner.termination_flag.TerminationFlag | None = None) → None

Uploads triplet data to the server for a given job.

Parameters:

  • job_id (str) – The job id of the import process

  • data (Table | list[RecordBatch] | DataFrame) – The data to upload

  • batch_size (int) – The number of rows per batch

  • progress_callback (ProgressCallback) – A callback function that is called with the number of rows uploaded after each batch

  • termination_flag (TerminationFlag | None) – A termination flag to cancel the upload if requested

Return type:

None

class graphdatascience.arrow_client.authenticated_flight_client.AuthenticatedArrowClient

Arrow Flight client used to communicate with the GDS Arrow server.

__init__(connection_info: str | tuple[str, int], auth: ArrowAuthentication | None = None, encrypted: bool = False, arrow_client_options: dict[str, Any] | None = None, user_agent: str | None = None, advertised_listen_address: tuple[str, int] | None = None, retry_config: RetryConfigV2 | None = None, health_check: ServerHealthCheck | None = None)

Creates a new AuthenticatedArrowClient instance.

Parameters:

  • connection_info (str | tuple[str, int]) – The host address and port of the GDS Arrow server

  • auth (ArrowAuthentication | None) – An implementation of ArrowAuthentication providing a pair to be used for basic authentication

  • encrypted (bool) – A flag that indicates whether the connection should be encrypted (default is False)

  • arrow_client_options (dict[str, Any] | None) – Additional options for the Arrow Flight client. The key call_timeout sets the per-call RPC timeout in seconds (default 30s); all other keys are forwarded to pyarrow.flight.FlightClient.

  • user_agent (str | None) – The user agent string to use for the connection. (default is neo4j-graphdatascience-v[VERSION] pyarrow-v[PYARROW_VERSION])

  • retry_config (RetryConfigV2 | None) – The retry configuration to use for the Arrow requests send by the client. (default: up to 5 attempts with a total delay budget derived from the per-call call_timeout so that timed-out calls are actually retried)

  • advertised_listen_address (tuple[str, int] | None) – The advertised listen address of the GDS Arrow server. This will be used by remote projection and writeback operations.

  • health_check (ServerHealthCheck | None) – Optional health check consulted when the server could not be reached, after all retries were exhausted. It is expected to raise a more descriptive error if it can explain the failure (such as out-of-memory error) and to return without raising otherwise.

advertised_connection_info() → ConnectionInfo

Returns the advertised host and port of the GDS Arrow server.

Returns:

the host and port of the GDS Arrow server

Return type:

ConnectionInfo

connection_info() → ConnectionInfo

Returns the host and port of the GDS Arrow server.

Returns:

the host and port of the GDS Arrow server

Return type:

ConnectionInfo

request_token() → str | None

Requests a token from the server and returns it.

Returns:

a token from the server and returns it.

Return type:

str | None

class graphdatascience.arrow_client.v2.api_types.JobStatus

Status and progress information for a GDS Arrow job.

class graphdatascience.arrow_client.authenticated_flight_client.ConnectionInfo

Host, port and encryption details for an Arrow server connection.

__init__(host: str, port: int, encrypted: bool) → None
Parameters:
Return type:

None

class graphdatascience.arrow_client.arrow_authentication.ArrowAuthentication

Abstract authentication strategy for the GDS Arrow server.

abstract auth_pair() → tuple[str, str]

Returns the auth pair used for authentication.

Return type:

tuple[str, str]

class graphdatascience.arrow_client.server_health_check.ServerHealthCheck

Protocol for checking GDS Arrow server health before use.

__init__(*args, **kwargs)
raise_if_unhealthy() → None

Raises a descriptive error if the server is known to be in an unhealthy state, and returns without raising if the state could not be determined.

Return type:

None

class graphdatascience.arrow_client.progress_callback.ProgressCallback
__init__(*args, **kwargs)