Databricks Agent Bricks + Neo4j Integration

Overview

Databricks Agent Bricks (Mosaic AI Agent Framework) launched June 2025 with auto-optimization, evaluations, and governance. It features an MCP Catalog, Unity Catalog integration, and support for multiple models (GPT-5, Gemini, Claude, Llama).

Key Features:

  • MCP Catalog for managed MCP servers

  • Unity Catalog integration for governance

  • Auto-generates evaluations and tunes agents

  • Multi-model support

  • Document Intelligence

  • MLflow for observability

Official Resources:

Samples

This directory contains four end-to-end samples, each demonstrating a different integration pattern between Databricks and Neo4j via custom and official Neo4j MCP server. Samples can be deployed using Databricks SDK/CLI or via Databricks UI and use the public Neo4j companies demo database: neo4j+s://demo.neo4jlabs.com:7687 by default.

# Sample Pattern Auth Model Deployment

1

MCP over Unity AI Gateway

MCP managed connection as Tools

Unity AI Gateway-managed, per-user OAuth flow

Databricks Notebook

2

UC Functions MCP Tools

UC Functions as Tools

In code, protected by UC Governance

Databricks Notebook

3

Custom MCP Server — Using Databricks App

Python Custom MCP Server as Databricks App

Databricks Secrets

Databricks CLI

4

Official MCP Server — Using Databricks App

Official MCP Server as Databricks App

Databricks Secrets

Databricks CLI

Sample 1: MCP over Unity AI Gateway

Uses Unity AI Gateway to manage the MCP connection (including credentials and/or authentication). The MCP server can be self managed (like in samples 3 and 4) but here we are using Neo4j Aura MCP server instances, with OAuth authentication.

→ Full documentation

Sample 2: UC Functions MCP Tools

Uses custom UC Functions Tools that perform requests on Neo4j. Neo4j credentials are kept in the function, not intended for production but for fast-prototyping.

→ Full documentation

Sample 3: Custom MCP Server — Using Databricks App

Uses Databricks App to deploy a Custom MCP Server written in Python who defines Neo4j driver queries as Tools. Neo4j Basic Auth credentials retrieved from Databricks Secrets.

→ Full documentation

Sample 4: Official MCP Server — Using Databricks App

Uses Databricks App to deploy the Neo4j Official MCP Server. Neo4j Basic Auth credentials retrieved from Databricks Secrets.

→ A quick guide walks through the deployment the MCP server in minutes using automated scripts. → Step-by-step documentation covers in details all the aspects of the Databricks App implementation.

Extension Points

1. MCP Catalog (Primary)

Install Neo4j MCP server from Databricks Marketplace or manually:

from databricks_mcp import DatabricksMCPClient
from databricks.sdk import WorkspaceClient

workspace_client = WorkspaceClient(profile="DEFAULT")
host = workspace_client.config.host

client = DatabricksMCPClient(
    servers=[
        f"{host}/api/2.0/mcp/neo4j/prod/company_research"
    ],
    workspace_client=workspace_client
)

# Use tools
tools = client.as_tools()

2. Unity Catalog Connection

Create Unity Catalog connection for Neo4j:

CREATE CONNECTION neo4j_prod
  TYPE http
  URL 'https://your-neo4j-mcp-server.com/mcp'
  WITH (
    CREDENTIAL bearer_token SECRET 'your-token'
  );

3. Direct Integration in Notebooks

from neo4j import GraphDatabase

driver = GraphDatabase.driver(
    "neo4j+s://demo.neo4jlabs.com:7687",
    auth=("companies", "companies")
)

MCP Authentication

✅ Personal Access Tokens (PAT) - For user access

✅ Service Principals (Primary for M2M)

  • OAuth M2M required for Agent Bricks Multi-Agent Supervisor

  • OAuth application registration in Databricks account

✅ M2M OIDC

  • OAuth 2.0 with Databricks as Identity Provider

  • Dynamic Client Registration support

  • Azure AD integration for Azure Databricks

Other Mechanisms:

  • Unity Catalog permissions

  • Managed MCP Proxies (token refresh handled by Databricks)

  • On-Behalf-Of-User (OBO) authentication

  • Automatic authentication passthrough

Industry Research Agent Example

import mlflow
from databricks_mcp import DatabricksMCPClient
from databricks.sdk import WorkspaceClient

# Setup
workspace_client = WorkspaceClient()
mcp_client = DatabricksMCPClient(
    servers=["https://your-workspace/api/2.0/mcp/neo4j/prod"],
    workspace_client=workspace_client
)

# Define agent
class ResearchAgent:
    def __init__(self, tools):
        self.tools = tools

    def research_company(self, company_name: str) -> str:
        # Query company data
        company_data = self.tools["query_company"](company_name)
        # Search news
        news = self.tools["search_news"](company_name)
        # Generate report
        return self.synthesize_report(company_data, news)

# Log agent to MLflow
tools = mcp_client.as_tools()
agent = ResearchAgent(tools)

with mlflow.start_run():
    mlflow.log_param("model", "claude-3-5-sonnet")
    mlflow.pyfunc.log_model("research_agent", python_model=agent)

# Deploy
deployment = mlflow.deployments.create_deployment(
    name="research-agent",
    model_uri=f"runs:/{run.info.run_id}/research_agent",
    endpoint="agents"
)

Additional Integration Opportunities

  • Neo4j as episodic memory backend

  • Unity Catalog governance for graph queries

  • MLflow tracking for agent performance

  • Auto-evaluation of graph query accuracy

Additional Resources