GenAI integrations
Neo4j provides integrations with various generative AI services using a native GenAI plugin. This includes support for transforming text data into vector embeddings using VertexAI, OpenAI, Azure OpenAI, or Amazon Bedrock. The resulting vector embeddings can be used together with Neo4j’s vector search indexes.
Prerequisites
To use the GenAI plugin, you need to have the following:
-
A Neo4j DBMS running Neo4j 5.17 or later.
-
A GenAI provider account, such as VertexAI, OpenAI, Azure OpenAI, or Amazon Bedrock.
-
API credentials for the provider you want to use.
-
(On-prem only) the Neo4j GenAI plugin installed in your Neo4j deployment. The plugin JAR is located in the <NEO4J_HOME>/products directory of the Neo4j installation. For more information on how to install and configure the plugin, see Operations Manual → Configure plugins. The plugin is enabled by default in Neo4j Aura.
Vector embeddings
A vector embedding is a numerical representation of some object, typically unstructured data, such as natural language text. For more information and examples of vector embeddings in general, see Vector search indexes.
Text can be transformed into vector embeddings using some model of natural language. Providers of LLM (Large Language Model) and Generative AI services, such as VertexAI, OpenAI, Azure OpenAI, and Amazon Bedrock, expose this functionality via their APIs, which can be invoked directly from within Neo4j using this plugin.
Generate a single embedding
You can use the genai.vector.encode()
function to generate a vector embedding for a single value, for example, when applying the transformation inline in an expression when querying for or writing to a specific property.
This function sends one API request every time it is called, which may result in a lot of overhead in terms of both network traffic and latency.
If you want to generate many embeddings at once, use |
genai.vector.encode()
Functiongenai.vector.encode(resource :: STRING, provider :: STRING, configuration :: MAP = {}) :: LIST<FLOAT>
-
The
resource
(aSTRING
) is the object to transform into an embedding, such as a chunk of natural language text. -
The
provider
(aSTRING
) is the case-insensitive identifier of the provider to use. See identifiers under GenAI providers for supported options. -
The
configuration
(aMAP
) contains provider-specific settings, such as which model to invoke, as well as any required API credentials. See GenAI providers for details of each supported provider.Note that because this argument may contain sensitive data, it is obfuscated in the query.log. However, if the function call is misspelled or the query is otherwise malformed, it may be logged without being obfuscated.
The returned list of floats is the vector representing the resource that was passed in.
The genai.vector.encode
function is useful for generating vectors that can be used to query a vector index.
For example, a vector index can be queried for entries that are semantically near some query string by generating a vector embedding for that query string and then using that vector as the query vector in a vector index search.
Here, you query a vector index called my_index
for the 10 nearest neighbors:
WITH "embeddings are cool" AS queryString
WITH genai.vector.encode(queryString, "VertexAI", { token: $token, projectId: $project }) AS queryVector
CALL db.index.vector.queryNodes("my_index", 10, queryVector) YIELD node, score RETURN node, score
Assuming nodes with the Tweet
label have an id
property and a text
property, you can generate and return the text embedding for the tweet with ID 1234:
MATCH (n:Tweet { id: 1234 })
RETURN genai.vector.encode(n.text, "VertexAI", { token: $token, projectId: $project }) AS embedding
Generating a batch of embeddings
You can use the genai.vector.encodeBatch()
procedure to generate many vector embeddings with a single API request.
This procedure takes a list of resources as an input, and returns the same number of result rows, instead of a single one.
Using this procedure is recommended in cases where a single large resource is split up into multiple chunks (like the pages of a book), or when generating embeddings for a large number of resources.
This procedure attempts to generate embeddings for all supplied resources in a single API request. Therefore, it is recommended to see the respective provider’s documentation for details on, for example, the maximum number of embeddings that can be generated per request. |
genai.vector.encodeBatch()
Proceduregenai.vector.encodeBatch(resources :: LIST<STRING>, provider :: STRING, configuration :: MAP = {}) :: (index :: INTEGER, resource :: STRING, vector :: LIST<FLOAT>)
-
The
resources
(aLIST<STRING>
) parameter is the list of objects to transform into embeddings, such as chunks of natural language text. -
The
provider
(aSTRING
) is the case-insensitive identifier of the provider to use. See GenAI providers for supported options. -
The
configuration
(aMAP
) specifies provider-specific settings such as which model to invoke, as well as any required API credentials. See GenAI providers for details of each supported provider.Because this argument may contain sensitive data, it is obfuscated in the query.log. However, if the function call is misspelled or the query is otherwise malformed, it may be logged without being obfuscated.
Each returned row contains the following columns:
-
The
index
(anINTEGER
) is the index of the corresponding element in the input list, to aid in correlating results back to inputs. -
The
resource
(aSTRING
) is the name of the input resource. -
The
vector
(aLIST<FLOAT>
) is the generated vector embedding for this resource.
Given a list of page texts, you can generate an embedding for each of the pages with a single procedure call and create a corresponding graph structure:
CREATE (book:Book { title: $bookTitle })
WITH book
CALL genai.vector.encodeBatch($pageTexts, "VertexAI", { token: $token, projectId: $project }) YIELD index, resource, vector
WITH book, index, resource, vector
CREATE (:Page { index: index, text: resource, vector: vector })-[:OF]->(book)
If you want to generate embeddings for the text content of all nodes with the label Tweet
, you can divide the nodes up into batches, and issue one API request per batch.
Assuming nodes with the Tweet
label have a text
property, you can generate vector embeddings for each one and write them to their embedding
property in batches of, for example, a thousand at a time.
You can use this in combination with CALL … IN TRANSACTIONS
to commit each batch separately to manage transaction memory consumption:
MATCH (n:Tweet)
WHERE size(n.text) <> 0 AND n.embedding IS NULL
WITH collect(n) AS nodes,
count(*) AS total,
1000 AS batchSize
UNWIND range(0, total, batchSize) AS batchStart
CALL {
WITH nodes, batchStart, batchSize
WITH nodes, batchStart, [node IN nodes[batchStart .. batchStart + batchSize] | node.text] AS batch
CALL genai.vector.encodeBatch(batch, "OpenAI", { token: $token }) YIELD index, vector
CALL db.create.setNodeVectorProperty(nodes[batchStart + index], "embedding", vector)
} IN TRANSACTIONS OF 1 ROW
You can control how many batches are committed by each inner transaction by modifying the OF 1 ROW
clause.
For example, OF 10 ROWS
will only commit once per 10 batches. Because vector embeddings can be very large, this may require significantly more memory.
GenAI providers
The following GenAI providers are supported for generating vector embeddings.
Each provider has its own configuration map that can be passed to the genai.vector.encode()
or genai.vector.encodeBatch()
functions.
Vertex AI
-
Identifier (
provider
argument):"VertexAI"
Key | Type | Description | Default |
---|---|---|---|
|
|
API access token. |
Required |
|
|
GCP project ID. |
Required |
|
|
The name of the model you want to invoke.
|
|
|
|
GCP region where to send the API requests.
|
|
|
|
The intended downstream application (see provider documentation). The specified |
|
|
|
The title of the document that is being encoded (see provider documentation). The specified |
OpenAI
-
Identifier (
provider
argument):"OpenAI"
Key | Type | Description | Default |
---|---|---|---|
|
|
API access token. |
Required |
|
|
The name of the model you want to invoke. |
|
|
|
The number of dimensions you want to reduce the vector to. Only supported for certain models. |
Model-dependent. |
Azure OpenAI
-
Identifier (
provider
argument):"AzureOpenAI"
Note that, unlike the other providers, the model is configured when creating the deployment on Azure, and is thus not part of the configuration map.
Key | Type | Description | Default |
---|---|---|---|
|
|
API access token. |
Required |
|
|
The name of the resource to which the model has been deployed. |
Required |
|
|
The name of the model deployment. |
Required |
|
|
The number of dimensions you want to reduce the vector to. Only supported for certain models. |
Model-dependent. |
Amazon Bedrock
-
Identifier (
provider
argument):"Bedrock"
Key | Type | Description | Default |
---|---|---|---|
|
|
AWS access key ID. |
Required |
|
|
AWS secret key. |
Required |
|
|
The name of the model you want to invoke.
|
|
|
|
AWS region where to send the API requests.
|
|