AWS Bedrock procedures

These procedures leverage the Amazon Bedrock API.

Here is a list of all available Aws Bedrock procedures:

name description

apoc.ml.bedrock.custom(body, $config)

To create a customizable Bedrock API call

apoc.ml.bedrock.list($config)

To get the list of foundation or custom models

apoc.ml.bedrock.embedding(texts, $config)

To create an API call to generate embedding

apoc.ml.bedrock.chat(messages, $config)

To create a Chat Completion API call

apoc.ml.bedrock.completion(prompt, $config)

To create a Text Completion API call

apoc.ml.bedrock.image(body, $config)

To create an API call to get an image

All the procedures, leverage the apoc.ml.bedrock.custom procedures, and support the same config parameter, but unlike the custom one, they have some different default parameters and model id.

Moreover, the return data is consistent with the called API, instead of returning a generic Object as a result

Config

Table 1. Config parameters
name type default description

keyId

String

null

The AWS key ID. We can also evaluate it via apoc.conf, with the key apoc.aws.key.id. As an alternative to the pair keyId-secretKey, we can directly pass the aws V4 signature via the headers config

secretKey

String

null

The AWS secret access key. We can also evaluate it via apoc.conf, with the key apoc.aws.secret.id. As an alternative to the pair keyId-secretKey, we can directly pass the aws V4 signature via the headers config

region

String

us-east-1

The AWS region

endpoint

String

see below

The AWS endpoint.

method

String

"POST" (or "GET" with the apoc.ml.bedrock.list procedure)

The HTTP Method

headers

Map<String, Object>

{Content-Type: application/json', Accept, '*/*'}

The HTTP Header

model

String

see below

(This config is ignored with the bedrock.list proc.) The Bedrock Model

path

String

"foundation-models"

(Valid only with the bedrock.list) The endpoint path. It will create an endpoint of the type https://bedrock.<regionConfigValue>.amazonaws.com/<path>;, i.e. with default https://bedrock.us-east-1.amazonaws.com/foundation-models

openAICompatible

String

false

To pass a body request compatible with OpenAI Chat Completions API, using the apoc.ml.bedrock.chat, for example: {role:"system", content:"Only answer with a single word"} ,{role:"user", content:"What planet do humans live on?"}

The endpoint config takes precedence over the model one. In case of all procedures, except the bedrock.list, the default endpoint is "https://bedrock-runtime.<regionConfigValue>.amazonaws.com/model/<modelConfigValue>/invoke". So, with the default region config, i.e. "us-east-1", the default endpoint is "https://bedrock-runtime.us-east-1.amazonaws.com/model/<modelConfigValue>/invoke".

The <modelConfigValue> part must be configured if we use the ml.bedrock.custom procedure, while with the bedrock.chat, bedrock.completion, bedrock.embedding, bedrock.image ones, has a default value of "anthropic.claude-v2", "ai21.j2-ultra-v1", "anthropic.claude-v2" and "stability.stable-diffusion-xl-v0" respectively.

Authentication settings

To authenticate to bedrock services, we can set in the apoc.conf file the following entries.

apoc.conf
apoc.aws.key.id=<AWS Key ID>
apoc.aws.secret.key=<AWS Secret Access Key>

Alternatively we can set them as $config parameters, i.e.: {keyId: '<AWS Key ID>', secretKey:'<AWS Secret Access Key>'}.

Or also, we can put an Authorization header, by using the header parameter, i.e. {header: {Authorization: 'AWS4-HMAC-SHA256 <CredentialAndSignature..>', …​other entries…​} }.

Note that the default Content-Type: application/json and the Accept: */* header entries, are always passed to the http request, unless overridden via the config header.

In the following examples, we assume that we set Key id and Secret Access Key via apoc.conf.

Usage Examples

Chat Completion API

This procedure apoc.ml.bedrock.chat takes a list of maps of chat exchanges between assistant and user (with optional system context), and will return the next message in the flow.

Additional configuration is passed to the API, the default model used is anthropic.claude-v2.

apoc.ml.bedrock.chat
CALL apoc.ml.bedrock.chat([
    {
        prompt: "\n\nHuman: Hello world\n\nAssistant:",
        max_tokens_to_sample: 50,
        top_k: 250,
        top_p: 1,
        stop_sequences: ["\\n\\nHuman:"]
    }
])
Table 2. Results
value

{"stop_reason": "stop_sequence","completion": " Hello!"}

We can use the config openAICompatible: true, to use a message body request consistent with the apoc.ml.openai.chat procedure. With this config, the prompt request will be placed in an entry {content: '<promts>'} and will have a default "\n\nHuman:"` prefix and \n\nAssistant: suffix, if not present.

For example, instead of:

apoc.ml.bedrock.chat (with openAICompatible: false)
CALL apoc.ml.bedrock.chat(
    [ {prompt: "\n\nHuman: Hello world\n\nAssistant:",max_tokens_to_sample: 200} ]
)

we can execute this query (note that the role:"system" entry is optional, it is just to be consistent with the OpenAI body):

apoc.ml.bedrock.chat (with openAICompatible: true)
CALL apoc.ml.bedrock.chat([
    {role:"system", content:"Hello world"}
])

Text Completion API

This procedure apoc.ml.bedrock.completion can continue/complete a given text. Additional configuration is passed to the API, the default model used is ai21.j2-ultra-v1.

apoc.ml.bedrock.completion
CALL apoc.ml.bedrock.completion('What color is the sky? Answer in one word: ')
Table 3. Results
value 
{
  "id": 1234,
  "completions": [
    {
      "data": {
        "text": "\nBlue",
        "tokens": ["....."]
      },
      "finishReason": {
        "reason": "endoftext"
      }
    }
  ],
  "prompt": {}
}

Image API

This procedure apoc.ml.bedrock.completion can get a base64 image. Additional configuration is passed to the API, the default model used is stability.stable-diffusion-xl-v0.

apoc.ml.bedrock.image
CALL apoc.ml.bedrock.image({
    text_prompts: [{text: "picture of a bird", weight: 1.0}],
    cfg_scale: 5,
    seed: 123,
    steps: 70,
    style_preset: "photographic"
})
Table 4. Results
base64Image

"iVBORw0KGgoAAAANSUhEUgAAAgAAAAIACAIAAAB7GkOtAAABjmVYSWZNTQAqAAAACAAGAQAABAAAAAEAAAIAAQEABAAA…​."

List of models

CALL apoc.ml.bedrock.list()
Table 5. Results
modelId modelArn modelName providerName responseStreamingSupported customizationsSupported inferenceTypesSupported inputModalities outputModalities

"amazon.titan-tg1-large"

"arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-tg1-large"

"Titan Text Large"

"Amazon"

true

["FINE_TUNING"]

["ON_DEMAND"]

["TEXT"]

["TEXT"]

"amazon.titan-e1t-medium"

"arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-e1t-medium"

"Titan Text Embeddings"

"Amazon"

null

[]

["ON_DEMAND"]

["TEXT"]

["EMBEDDING"]

…​

…​

…​

…​

null

[]

…​

…​

…​

Custom AWS API Call

Via the apoc.ml.bedrock.custom we can create a customizable Bedrock API Request, by choosing the HTTP Method, the endpoint, the region and the additional headers. Useful both for invoke a model, in the case the response is incompatible with the previous procedures, and to use any other Bedrock API.

For example, we can call the GetModelInvocationLoggingConfiguration API by executing the following query (note that the body parameter is null, since the API does not have a request body.):

CALL apoc.ml.bedrock.custom(null,{
    endpoint: "https://bedrock.us-east-1.amazonaws.com/logging/modelinvocations",
    method: "GET"
})
Table 6. Results
value

{ "loggingConfig": {"cloudWatchConfig": { …​ }}}