LLM provider API reference

Reference for the neo4j_agent_memory.llm Protocols, types, factory, and exception hierarchy.

These protocols describe client capabilities; their TCK tiers are not NAMS service versions or access tiers.

Every adapter in neo4j_agent_memory.llm.adapters implements one or more of the Protocols below. Adapter constructors are documented separately at Adapters. The factory is documented at Factory.

Overview

The module exposes three Protocols (LLMProvider, StructuredExtractor, EmbeddingProvider), the Pydantic types they use, an exception hierarchy, the from_provider factory, and the schema_aligned_extract helper. The sections below cover each in the order they appear in the public exports of neo4j_agent_memory.llm.

Module

neo4j_agent_memory.llm

Public exports

from neo4j_agent_memory.llm import (
    # Protocols
    LLMProvider,
    StructuredExtractor,
    EmbeddingProvider,
    # Types
    ChatMessage,
    Completion,
    Usage,
    # Errors
    ProviderError,
    ProviderAuthError,
    ProviderRateLimitError,
    ProviderTimeoutError,
    ProviderInvalidRequestError,
    ProviderServiceError,
    StructuredExtractionError,
    EmbeddingDimensionMismatchError,
    # Factory
    from_provider,
    # Helpers
    schema_aligned_extract,
)

Protocols

LLMProvider

Chat completions. Bronze TCK tier.

class LLMProvider(Protocol):
    model: str

    async def complete(
        self,
        messages: Sequence[ChatMessage],
        *,
        temperature: float = 0.0,
        max_tokens: int | None = None,
        stop: Sequence[str] | None = None,
        timeout: float | None = None,
    ) -> Completion: ...

Implementations MUST:

  • Be safe to call concurrently — no shared mutable state.

  • Translate SDK-specific errors to ProviderError subclasses.

  • Honor temperature=0.0 as deterministic where the provider supports it.

The protocol requires the canonical "provider/model" identifier in the model attribute. Adapters that accept bare names normalise to include the prefix.

StructuredExtractor

Validated Pydantic outputs. Silver TCK tier.

class StructuredExtractor(Protocol):
    async def complete_structured(
        self,
        messages: Sequence[ChatMessage],
        response_model: type[T],
        *,
        temperature: float = 0.0,
        max_retries: int = 2,
        timeout: float | None = None,
    ) -> T: ...

Implementations MUST:

  • Use the most reliable structured-output mode the underlying provider supports — strict JSON schema (OpenAI), forced tool use (Anthropic), response_format (LiteLLM), etc.

  • Retry on ValidationError up to max_retries times, with feedback.

  • Raise StructuredExtractionError after exhausting retries.

Adapters without a native structured mode delegate to schema_aligned_extract.

EmbeddingProvider

Text embeddings. Bronze TCK tier.

class EmbeddingProvider(Protocol):
    model: str
    dimensions: int

    async def embed(self, texts: Sequence[str]) -> list[list[float]]: ...
    async def embed_one(self, text: str) -> list[float]: ...

Contract:

  • dimensions must be available at construction time (not lazily).

  • embed([]) returns [].

  • Every returned vector has length dimensions.

MemoryClient.connect() reads dimensions to size vector indexes and validates against existing indexes — see the migration runbook.

Types

ChatMessage

Pydantic, frozen.

class ChatMessage(BaseModel):
    role: Literal["system", "user", "assistant", "tool"]
    content: str
    name: str | None = None
    tool_call_id: str | None = None

content is str only — multimodal is a future concern.

Completion

Pydantic.

class Completion(BaseModel):
    content: str
    model: str
    usage: Usage | None = None
    finish_reason: str | None = None
    raw: dict[str, Any] | None = None    # only set when adapter return_raw=True

Usage

Pydantic.

class Usage(BaseModel):
    prompt_tokens: int = 0
    completion_tokens: int = 0
    total_tokens: int = 0
    cached_tokens: int = 0    # Anthropic prompt cache, OpenAI cached inputs
    cost_usd: float | None = None

Exception hierarchy

All inherit from ProviderError, which inherits from Exception. None inherit from MemoryError — provider errors are intentionally separate from storage errors.

Class Raised when

ProviderError

Base class.

ProviderAuthError

API key invalid / missing / expired.

ProviderRateLimitError

429 / quota exceeded. Carries retry_after: float | None.

ProviderTimeoutError

Request exceeded configured timeout.

ProviderInvalidRequestError

Malformed request (unknown model, bad params).

ProviderServiceError

Retriable 5xx server error.

StructuredExtractionError

SAP exhausted retries. Carries last_attempts: list[str], validation_errors.

EmbeddingDimensionMismatchError

Existing vector index disagrees with embedder. Carries expected_dimensions, actual_dimensions, index_name.

Factory

from_provider(model: str, *, kind="llm", prefer_litellm=False, **kwargs) — string-shorthand factory. Returns an LLMProvider or EmbeddingProvider depending on kind. See Provider factory reference.

Helper functions

schema_aligned_extract

async def schema_aligned_extract(
    provider: LLMProvider,
    messages: Sequence[ChatMessage],
    response_model: type[T],
    *,
    temperature: float = 0.0,
    max_retries: int = 2,
    timeout: float | None = None,
) -> T: ...

Generic structured-output path. Builds a system message containing the schema, parses tolerantly, validates against response_model, retries with feedback on failure. Use directly when your provider does not implement StructuredExtractor.

Tolerant JSON parser

schema_aligned_extract strips markdown fences, smart quotes, trailing commas, and finds the largest balanced {...} block. Truncated JSON (mid-string cutoff) surfaces as json.JSONDecodeError — the correct outcome, since it must trigger a retry.

Default model dimensions

neo4j_agent_memory.llm.defaults exposes:

  • EMBEDDING_DIMENSIONS: dict[str, int] — known model → dimensions.

  • lookup_embedding_dimensions(model: str) → int | None — tolerant lookup with and without provider prefix.

For models not in the table, pass an explicit dimensions=N constructor argument.

See also