Database Agent for Semantic Kernel

Overview

The Database Agent for Semantic Kernel is a service that provides a database management system (DBMS) for the Semantic Kernel (NL2SQL). The Agent is responsible for managing the storage and retrieval of data from the Semantic Kernel. This built on top of the Microsoft's Semantic Kernel and Semantic Kernel Memory connectors to memorize database schema and relationships to provide a more efficient and accurate database management system.

<img alt="image" src="https://github.com/user-attachments/assets/adff6bac-440b-46d6-a0b3-a4fa84679c17" />

Models Tested

| Model Family | Model Name | NL 2 SQL | Quality Insurance | Score | Speed (avg time/op.) | |--------------|---------------------|:---------:|:----------------:|:-----------:|:----------------------:| | OpenAI | gpt-4.1-mini | ✅ | ✅ | 100% | Fast (~3sec) | | OpenAI | gpt-4o-mini | ✅ | ✅ | 90% | Fast (~3sec) | | Phi | phi4:14b | ✅ | ✅ | 70% | Medium (~10sec) | | Meta | llama4:scout | ✅ | ✅ | 70% | Medium (~10sec) | | MistralAI | magistral:24b | ✅ | ✅ | 90% | Medium (~10sec) | | MistralAI | devstral:24b | ✅ | ✅ | 70% | Medium (~10sec) | | Qwen | qwen3:30b-a3b | ✅ | ✅ | 80% | Medium (~10sec) | | Qwen | qwen3:14b | ⚠️ (WIP) | ⚠️ (WIP) | 50% | Medium (~10sec) | | Qwen | qwen3:8b | ⚠️ (WIP) | ⚠️ (WIP) | 50% | Medium (~10sec) | | Qwen | qwen2.5-coder:7b | ⚠️ (WIP) | ⚠️ (WIP) | 30% | Fast (~3sec) |

Note: current score is a personal evaluation regarding the test cases with Northwind database and a set of queries. development is firstly focused on the gpt-4o-mini model, which is the most performant and accurate model for NL2SQL tasks. for the evaluation, the TopP and Temperature parameters are set to 0.1, which is the recommended setting.

DICLAIMER

Even if the model is marked as tested, it does not mean that it will work for all queries.

Furthermore, using LLM agents might lead to risks such as unintended data exposure, security vulnerabilities, and inefficient query execution, potentially compromising system integrity and compliance requirements.

Getting Started

Prerequisites

• .NET 8.0 SDK

Installation

To use the Database Agent for Semantic Kernel, you must first install the package from NuGet.

dotnet add package SemanticKernel.Agents.DatabaseAgent

Usage

To use the Database Agent for Semantic Kernel, you must first create an instance of the DatabaseAgent class and provide the necessary configuration settings.

using Microsoft.KernelMemory;
using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.ChatCompletion;
using SemanticKernel.Agents.DatabaseAgent;

var kernelBuilder = Kernel.CreateBuilder()
                ...
                .Build();

kernelBuilder.Services.AddSingleton<DbConnection>((sp) =>
            {
                // Configure the database connection
                return new SqliteConnection(configuration.GetConnectionString("DefaultConnection"));
            });

var kernel = kernelBuilder.Build();

var agent = await DBMSAgentFactory.CreateAgentAsync(kernel);

// execute the NL2SQL query
var responses = agent.InvokeAsync([new ChatMessageContent { Content = question, Role = AuthorRole.User }], thread: null)
                                            .ConfigureAwait(false);

Install the MCP Server as a Docker Image

The database agent MCP server can be run as a Docker image. This allows you to run the server in a containerized environment, making it easy to deploy and manage to expose it SSE (Server-Sent Events) and HTTP endpoints.

To run the MCP server as a Docker image, you can use the following command:

docker run -it --rm \
  -p 8080:5000 \
  -e AGENT__TRANSPORT__KIND=Sse \
  -e ASPNETCORE_URLS=http://+:5000 \
  -e DATABASE_PROVIDER=sqlite \
  -e DATABASE_CONNECTION_STRING="Data Source=northwind.db;Mode=ReadWrite" \
  -e MEMORY_KIND=Volatile \
  -e KERNEL_COMPLETION=gpt4omini \
  -e KERNEL_EMBEDDING=textembeddingada002 \
  -e SERVICES_GPT4OMINI_TYPE=AzureOpenAI \
  -e SERVICES_GPT4OMINI_ENDPOINT=https://xxx.openai.azure.com/ \
  -e SERVICES_GPT4OMINI_AUTH=APIKey \
  -e SERVICES_GPT4OMINI_API_KEY=xxx \
  -e SERVICES_GPT4OMINI_DEPLOYMENT=gpt-4o-mini \
  -e SERVICES_TEXTEMBEDDINGADA002_TYPE=AzureOpenAI \
  -e SERVICES_TEXTEMBEDDINGADA002_ENDPOINT=https://xxx.openai.azure.com/ \
  -e SERVICES_TEXTEMBEDDINGADA002_AUTH=APIKey \
  -e SERVICES_TEXTEMBEDDINGADA002_API_KEY=xxx \
  -e SERVICES_TEXTEMBEDDINGADA002_DEPLOYMENT=text-embedding-ada-002 \
  ghcr.io/kbeaugrand/database-mcp-server

Then you can configure your favorite MCP Client like Claude Desktop with this settings:

{
  "mcpServers": {
    "mcp-database-agent": {
      "command": "npx",
      "args": [
        "mcp-remote",
        "http://localhost:8080/sse",
        "--allow-http"
      ]
    }
  }
}

Behind the scenes

Here is a simplified sequence diagram of how the Database Agent is constructed using the Semantic Kernel before it can be used:

sequenceDiagram
    autonumber
    participant Client
    participant DatabaseAgentFactory
    participant SemanticKernel
    participant Database

    Client->>DatabaseAgentFactory: CreateAgentAsync(kernel)
    DatabaseAgentFactory->>SemanticKernel: Access services (vector store, embedding, prompts)

    DatabaseAgentFactory->>DatabaseAgentFactory: MemorizeAgentSchema()
    DatabaseAgentFactory->>Database: Fetch list of tables
    loop For each table
        DatabaseAgentFactory->>Database: Get structure and data sample
        DatabaseAgentFactory->>SemanticKernel: Generate table description
        DatabaseAgentFactory->>SemanticKernel: Embed and store definition
    end

    DatabaseAgentFactory->>SemanticKernel: Generate agent description
    DatabaseAgentFactory->>SemanticKernel: Generate name and instructions
    DatabaseAgentFactory->>SemanticKernel: Embed and store agent

    DatabaseAgentFactory-->>Client: Return DatabaseKernelAgent

Then, once the agent is created, the client can use it to execute queries.

sequenceDiagram
    autonumber
    participant User
    participant DatabasePlugin
    participant SemanticKernel
    participant Database

    User->>DatabasePlugin: ExecuteQueryAsync(prompt)

    DatabasePlugin->>SemanticKernel: Generate embedding for prompt
    SemanticKernel-->>DatabasePlugin: Embedding

    DatabasePlugin->>SemanticKernel: Vector search for related tables
    SemanticKernel-->>DatabasePlugin: Matching table definitions

    DatabasePlugin->>SemanticKernel: Generate SQL query (WriteSQLQuery prompt)
    SemanticKernel-->>DatabasePlugin: SQL query string

    DatabasePlugin->>DatabasePlugin: Check query filters (optional)
    alt Query is allowed
        DatabasePlugin->>Database: Execute SQL query
        Database-->>DatabasePlugin: Query result
        DatabasePlugin-->>User: Markdown-formatted result
    else Query is blocked
        DatabasePlugin-->>User: Filter message
    end

    Note over DatabasePlugin: Logs and error handling during the process

Quality insurance

Using LLM agents to write and execute its own queries into a database might lead to risks such as unintended data exposure, security vulnerabilities, and inefficient query execution, potentially compromising system integrity and compliance requirements. To mitigate these risks, the Database Agent for Semantic Kernel provides a set of quality assurance features to ensure the safety and reliability of the queries executed by the agent.

Additional Configuration

First, you must add the QualityAssurance package for DatabaseAgent to your project.

dotnet add package SemanticKernel.Agents.DatabaseAgent.QualityAssurance

Next, you must configure the quality insurance settings for the Database Agent.

    kernelBuilder.Services.UseDatabaseAgentQualityAssurance(opts =>
                            {
                                opts.EnableQueryRelevancyFilter = true;
                                opts.QueryRelevancyThreshold = .8f;
                            });

Quality Assurance Features

The Database Agent for Semantic Kernel provides the following quality assurance features: QueryRelevancyFilter: Ensures that only relevant queries are executed by the agent. The filter uses LLM to generate the description of the query that is intended to be executed, then compute the cosine similarity between the user prompt and the generated description. If the similarity score is below the threshold, the query is rejected.

C