Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
120 changes: 120 additions & 0 deletions blog/2026-07-08-ivorysql-agent/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,120 @@
---
slug: ivorysql-agent
title: "IvorySQL Agent: AI-Powered Database Management"
authors: [Oreo Yang]
category: IvorySQL
image: img/blog/covers/ivorysql-agent-en.svg
tags: [IvorySQL, AI, Agent, RAG, NL2SQL, PostgreSQL]
---

> Based on Oreo Yang's presentation at HOW 2026.
> Video replay: https://www.youtube.com/watch?v=e5QJUBXgJ7k

## 1. Project Background

### 1.1 What is IvorySQL

IvorySQL is an open-source relational database built on PostgreSQL with Oracle compatibility, providing flexible, high-performance data management. Key features include Oracle type/PL/SQL/Package support, an active open-source community, and multiple deployment options (traditional, containerized, cloud, sandbox).

### 1.2 Project Goals

Database operations face multiple challenges: scattered metrics, difficult diagnostics, high labor costs, cumbersome documentation. The IvorySQL Agent leverages LLM technology for intelligent conversations and automated analysis, building a unified monitoring, diagnostics, and operations platform. Six core problems: intelligent documentation retrieval, database operations, performance tuning, SQL efficiency, enterprise knowledge management, transitioning from "passive search" to "proactive service."

### 1.3 What is an Agent

Agent = LLM + Tools + Reasoning. An agent combines language models with tool capabilities to reason about tasks, decide which tools to use, and iterate toward solutions.

IvorySQL Agent tech stack:
- **Model Layer**: OpenAI GPT, Anthropic Claude, Zhipu AI, and other backends
- **Tool Layer**: 21 database tools across 7 expert domains
- **Framework Layer**: LangGraph workflow orchestration with ReAct loop iteration
- **Knowledge Layer**: Matrix knowledge base covering multiple versions and modes

## 2. System Architecture

### 2.1 Overall Architecture

Four-layer design:

| Layer | Tech Stack | Responsibility |
|-------|-----------|----------------|
| Web Access | FastAPI + static + CORS | User interface |
| Smart Router | LangGraph + state management | Intent recognition & dispatch |
| Expert Agents | NL2SQL / Analysis / Ops / Backup / Install / Knowledge / General | Domain tasks |
| Tools & Storage | DB tools + LLM + vector store + TSDB | Data access & execution |

### 2.2 Core Data Flows

**Monitoring collection flow**: Collector covers 50+ metrics (connections, cache, locks, transactions, queries, table bloat, etc.). Oracle-compatible mode dynamically registers custom type codecs. Data stored as JSONB in PostgreSQL; APScheduler runs every 30s with Delta incremental computation.

**Conversation flow**: User input routed to corresponding Agent, context built, tool chain invoked for reasoning, response generated.

### 2.3 Anomaly Detection: Rule Engine + LLM

Dual-track mechanism: predefined threshold rules trigger LLM analysis; Claude/OpenAI/Zhipu AI for intelligent interpretation; 512-dim local vectors (configurable 256-2048) with pgvector indexing; historical case retrieval reuses diagnostic experience.

## 3. Key Technical Implementation

### 3.1 Two-Stage Smart Routing

- **Phase 1 (Deterministic)**: Keyword + regex matching, ~0ms, ~70% coverage
- **Phase 2 (LLM Classification)**: LLM intent recognition on rule mismatch, ~300ms
- **Post-Confirm**: Cache hit reuse, 0ms

Key design decisions: mutually exclusive keywords preventing cross-domain conflicts, bilingual regex, progressive fallback (keyword → LLM → general), mode-aware tool table pruning based on compat_mode, LLM fault tolerance (exponential backoff ×2, timeout fallback to general), domain isolation to prevent tool pollution.

### 3.2 Local Vector Retrieval (Zero External Dependencies)

BAAI/bge-small-zh-v1.5 embedding model, 512-dim, fully offline. pgvector + IVFFlat index. Knowledge base includes IvorySQL docs, PostgreSQL docs, and historical analysis. RAG uses "weighted vector search + BM25 reranking + MMR diversification." Document chunks managed by chapter with YAML frontmatter metadata including version and mode tags.

### 3.3 Context Management & Smart Compression

Schema cache: 1-hour TTL, auto-invalidate on DDL. Package cache: IvorySQL Oracle mode package info. Real-time metrics: dynamic session, lock, slow query loading. Smart compression for 15+ turn conversations: DDL detection → selective retention (DDL + last 6 turns) → token check → LLM summary → context injection.

### 3.4 Tool System & Security

21 built-in tools across domains. Four-layer SQL validation: prompt pre-filtering (80% invalid SQL blocked) → syntax cache verification → pre-confirmation validation → error feedback retry (max 3 attempts). API Token + SHA256 authentication with hot-reload. Audit logging. Oracle/PG mode tool differentiation.

### 3.5 Chat Flow: Plan · Generate · Validate · Stream

LLM generates JSON step plans. SSE streaming pushes token, tool_start, tool_end events in real time. State round-tripping maintains multi-step plan coherence across rounds.

### 3.6 Deployment

Docker containerized: PostgreSQL storage + Agent service + Grafana + knowledge import. settings.json runtime config with hot updates. Multi-target monitoring. docker-compose one-click launch.

## 4. Technical Features

- Fully offline local operation, no external API dependencies
- Multi-RAG domain expansion (7 Agents currently, extensible)
- Intelligent context caching and compression
- Multi-layer security validation and audit logging
- Modular extensible design supporting custom tools and knowledge bases

## 5. Application Scenarios

| Scenario | User Query | Agent |
|----------|-----------|-------|
| Connection spike diagnosis | "Why are connections so high?" | Ops Agent analyzes active sessions, long transactions |
| Slow query optimization | "Find my slowest queries" | NL2SQL Agent retrieves slow query logs |
| Backup status check | "Is backup config OK?" | Backup Agent checks WAL archiving |
| Knowledge base Q&A | "How to use %rowtype?" | Knowledge Agent queries RAG |

## 6. Future Plans

### 6.1 Short-Term

- Refine Agent framework with multi-model backend and streaming
- Dual-mode IvorySQL/PostgreSQL compatibility
- Multi-modal input: file analysis, screenshot recognition, voice
- Knowledge base management: online CRUD for entries

### 6.2 Long-Term

Multi-version support, automated low-risk repairs, multi-instance management, granular access control, alerting (email/WeChat/Slack), self-learning from user preferences, more domain Agents (security, HA, performance tuning).

**Vision**: IvorySQL as AI data infrastructure — native pgvector integration, multi-language SDK (3-line RAG), Agent development kit with LangGraph/LlamaIndex templates, one-click Docker Compose deployment.

## 7. Summary

The IvorySQL Agent is a systematic exploration of AI-powered database management. Built on the ReAct framework with LangGraph orchestration, it delivers end-to-end automation from natural language to database operations, covering routing, retrieval, context management, tool invocation, and security auditing. As the project evolves toward multi-modal interaction, automated repairs, and an SDK ecosystem, IvorySQL aims to be not just a great database, but the infrastructure for intelligent data management.
8 changes: 7 additions & 1 deletion blog/authors.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,4 +17,10 @@ Yasir Hussain Shah:
ShunWah:
name: ShunWah
杨宇:
name: 杨宇
name: 杨宇
ZhangChen:
name: ZhangChen
陶郑:
name: 陶郑
Oreo Yang:
name: Oreo Yang
Original file line number Diff line number Diff line change
@@ -0,0 +1,133 @@
---
slug: ivorysql-agent
title: "IvorySQL Agent 探索与实践"
authors: [Oreo Yang]
category: IvorySQL
image: img/blog/covers/ivorysql-agent-zh.png
tags: [IvorySQL, AI, Agent, RAG, NL2SQL, PostgreSQL]
---

> 本文基于 HOW 2026 Oreo Yang 的演讲内容整理。
> 视频回放:https://www.bilibili.com/video/BV1CFLB6DEHV/

## 一、项目背景

### 1.1 什么是 IvorySQL

IvorySQL 是一个基于 PostgreSQL 并兼容 Oracle 的开源关系型数据库项目,致力于提供灵活、高性能的数据管理解决方案。其核心特性包括:

- **Oracle 兼容**:支持 Oracle 数据类型、PL/SQL 语法、内置函数及 Package 包,显著降低从 Oracle 迁移或混合部署的技术门槛与成本。
- **社区生态**:拥有活跃的开源社区,通过社区协作持续推动产品迭代与技术创新。
- **部署灵活性**:支持传统部署、容器化、云服务、在线体验、沙盒环境等多种部署方式。

### 1.2 项目背景与目标

数据库运维面临多重挑战:指标分散、诊断困难、人力成本高、文档查找繁琐。IvorySQL Agent 的核心理念是利用大模型技术实现智能对话与自动化分析,构建一站式监控、诊断、运维辅助平台。项目要解决的六大核心问题包括:智能文档检索、数据库运维、性能调优、SQL 编写效率提升、企业知识管理,最终通过 AI 技术实现从"被动检索"到"主动服务"的转型。

### 1.3 什么是 Agent

Agent = 大模型 + 工具 + 推理。智能体将语言模型与工具能力结合,能够对任务进行推理、决定使用哪些工具,并通过迭代寻求解决方案。

IvorySQL Agent 的技术选型:
- **模型层**:支持 OpenAI GPT 系列、Anthropic Claude 系列、智谱 AI 等多模型后端
- **工具层**:21 个数据库工具,覆盖 7 大领域专家
- **框架层**:基于 LangGraph 工作流编排,实现 ReAct 循环迭代
- **知识层**:矩阵式知识库,覆盖多版本、多模式

## 二、系统架构

### 2.1 整体架构

系统采用四层架构设计:

| 层级 | 技术栈 | 职责 |
|------|--------|------|
| Web 接入层 | FastAPI + 静态资源 + CORS | 用户界面交互 |
| 智能路由层 | LangGraph 工作流 + 状态管理 | 意图识别与请求分发 |
| 专家 Agent 层 | NL2SQL / Analysis / Ops / Backup / Install / Knowledge / General | 领域任务处理 |
| 工具与存储层 | 数据库工具集 + LLM 接口 + 向量存储 + 时序数据库 | 数据访问与工具执行 |

### 2.2 核心数据流

**监控采集流**:通过 Collector 设计,覆盖 50+ 监控指标(连接、缓存、锁、事务、查询、表膨胀等)。针对 IvorySQL 的 Oracle 兼容模式,动态注册自定义类型编解码器。采集数据以 JSONB 格式存储于 PostgreSQL,后台采用 APScheduler 每 30 秒执行一次采集任务,并通过 Delta 增量计算减少开销。

**智能对话流**:用户输入经路由分发至对应 Agent,构建上下文后调用工具链执行推理,最终生成回答并返回用户。

### 2.3 异常检测:规则引擎 + LLM 协同

异常检测采用规则引擎与 LLM 协同工作的双轨机制:
- **规则引擎**:预定义阈值规则,命中后自动触发 LLM 分析
- **LLM 分析**:支持 Claude/OpenAI/智谱 AI 等多模型,对异常进行智能解读
- **向量化存储**:512 维本地向量(支持 256~2048 维自定义),配合 pgvector 索引加速检索
- **历史学习**:通过相似案例检索复用诊断经验

## 三、关键技术实现

### 3.1 两阶段智能路由

- **Phase 1(确定性路由)**:基于关键词与正则表达式匹配,响应时间约 0ms,覆盖约 70% 的场景
- **Phase 2(LLM 分类)**:规则匹配失败时调用大模型进行意图识别,响应时间约 300ms
- **Post-Confirm**:短路复用机制,0ms 完成缓存命中

关键设计决策:关键词互斥防止跨域冲突、中英双语正则、渐进降级(关键词 → LLM → general 三层兜底)、模式感知根据 compat_mode 动态裁剪工具表、LLM 容错(指数退避重试 ×2,超时降级 general)、域隔离防污染。

### 3.2 本地向量检索(零外部依赖)

采用 BAAI/bge-small-zh-v1.5 嵌入模型,512 维,完全离线运行。pgvector 扩展 + IVFFlat 索引加速。知识库来源包括 IvorySQL 官方文档、PostgreSQL 官方文档和历史分析结果。RAG 检索采用"加权向量搜索 + BM25 重排 + MMR 多样化"组合方案。切片管理按文档章节划分,包含版本和模式标记防止跨版本知识污染。

### 3.3 上下文管理与智能压缩

- Schema 缓存:1 小时 TTL,DDL 后自动失效
- 包定义缓存:IvorySQL Oracle 模式包信息
- 实时指标:动态加载当前会话、锁、慢查询
- 历史异常:RAG 检索相关案例,top-K=5

智能压缩机制应对 15+ 轮长对话:DDL 判断 → 选择性保留(DDL + 最近 6 轮)→ Token 检查 → LLM 摘要生成 → 注入优化上下文。

### 3.4 工具系统与安全控制

21 个内置工具,分领域管理。SQL 验证四层防护:Prompt 预过滤(拦截 80% 无效 SQL)→ 语法缓存校验 → 预确认验证 → 错误反馈重试(最多 3 次)。安全认证采用 API Token + SHA256 + 配置热重载。审计系统记录操作日志与错误追踪。IvorySQL Oracle/PG 模式工具差异化。

### 3.5 Chat 流程:规划 · 生成 · 验证 · 流式输出

Plan 规划由 LLM 生成 JSON 格式步骤计划。SSE 流式输出实时推送 token、tool_start、tool_end 事件。状态往返保证多步计划跨轮次连贯性,无需重建上下文。

### 3.6 部署架构

Docker 容器化设计:服务拆分(存储 PostgreSQL + Agent 服务 + Grafana 可视化 + knowledge 批量导入)、settings.json 运行时配置热更新、多目标支持监控多个实例、环境隔离、docker-compose 一键启动。

## 四、技术特色

- 纯离线本地运行,无需外部 API
- 多 RAG 领域扩展(当前 7 个 Agent,可继续扩展)
- 智能上下文缓存与压缩机制
- 多层次安全校验与审计日志
- 模块化可扩展设计,支持自定义工具与知识库

## 五、应用场景

| 场景 | 用户问题 | 处理 Agent |
|------|----------|------------|
| 连接数异常诊断 | "为什么连接数这么高?" | Ops Agent 分析活跃会话、长事务 |
| 慢查询优化 | "帮我找出最慢的查询" | NL2SQL Agent 检索慢查询日志 |
| 备份配置查询 | "备份配置正常吗?" | Backup Agent 检查 WAL 归档 |
| 知识库问答 | "如何使用 %rowtype?" | Knowledge Agent 检索 RAG 知识库 |

## 六、未来规划

### 6.1 短期规划

- 完善智能 Agent 框架,支持多 Model 后端与流式输出
- 实现 IvorySQL 与 PostgreSQL 双模式兼容
- 多模态输入支持:文件分析、截图识别、语音输入
- 知识库管理:在线新增、编辑、删除知识条目与切片

### 6.2 长期规划

**功能扩展**:多模式多版本支持、自动化修复、多实例管理、安全与访问控制、告警通知集成、自学习能力、更多领域 Agent。

**愿景**:原生集成 pgvector 成为 Agent 首选向量存储、多语言 SDK 开箱即用、Agent 开发套件、一键 Docker Compose 部署完整环境。核心能力:3 行代码实现 RAG 应用,无需配置外部向量服务。

## 七、总结

IvorySQL Agent 是 IvorySQL 社区在 AI 赋能数据库管理领域的一次系统性探索。项目基于 ReAct 框架与 LangGraph 工作流编排,构建了覆盖路由、检索、上下文管理、工具调用和安全审计的完整技术体系,实现了从自然语言到数据库运维动作的端到端自动化。未来,项目将持续向多模态交互、自动化修复、SDK 生态等方向演进,降低数据库的使用门槛,提升运维效率,让 IvorySQL 不仅是优秀的数据库,更成为智能数据管理的基础设施。
8 changes: 7 additions & 1 deletion i18n/zh-CN/docusaurus-plugin-content-blog/authors.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,4 +17,10 @@ Yasir Hussain Shah:
ShunWah:
name: ShunWah
杨宇:
name: 杨宇
name: 杨宇
ZhangChen:
name: ZhangChen
陶郑:
name: 陶郑
Oreo Yang:
name: Oreo Yang
56 changes: 56 additions & 0 deletions static/img/blog/covers/ivorysql-agent-en.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added static/img/blog/covers/ivorysql-agent-zh.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.