欢迎光临
我们一直在努力

Qdrant 向量数据库完全指南:从入门到 Spring AI/LangChain4J 集成实践

前言

在人工智能和大语言模型(LLM)应用日益普及的今天,向量数据库成为了构建 AI 应用的关键基础设施。Qdrant 作为一款高性能的开源向量数据库,以其卓越的性能、易用性和丰富的功能特性,正在成为越来越多开发者的首选。

本文将详细介绍 Qdrant 的核心特性,并展示如何在 Spring Boot 项目中集成 Qdrant,以及如何配合 Spring AI 和 LangChain 等主流 AI 框架构建智能应用。


一、Qdrant 简介与核心特性

1.1 什么是 Qdrant?

Qdrant(读音:quadrant)是一个用 Rust 编写的开源向量相似度搜索引擎,专门用于存储、搜索和管理向量嵌入(Vector Embeddings)。它提供了高性能的向量搜索能力,支持过滤、负载均衡等功能,非常适合构建推荐系统、语义搜索、AI 助手等应用。

1.2 核心特性

✅ 高性能搜索
  • 使用 HNSW(Hierarchical Navigable Small World)算法实现高效的近似最近邻搜索
  • 支持实时索引更新,不影响搜索性能
  • 单节点可处理数十亿级别的向量
✅ 丰富的过滤能力
  • 支持在向量搜索时结合元数据进行过滤
  • 提供类似 SQL 的过滤语法
  • 支持复杂的布尔查询
✅ 易于部署和扩展
  • 提供 Docker、Kubernetes 等多种部署方式
  • 支持水平扩展和分片
  • 提供 RESTful API 和 gRPC 接口
✅ 企业级特性
  • 支持数据持久化
  • 提供快照和备份功能
  • 内置负载均衡和复制
  • 支持访问控制和身份验证

二、Qdrant 快速开始

2.1 使用 Docker 启动 Qdrant

最简单的启动方式是使用 Docker:

# 启动 Qdrant 实例
docker run -p 6333:6333 -p 6334:6334 \\
-v $(pwd)/qdrant_storage:/qdrant/storage \\
qdrant/qdrant

# 或者使用 docker-compose
version: '3.8'
services:
qdrant:
image: qdrant/qdrant
ports:
"6333:6333"
"6334:6334"
volumes:
– ./qdrant_storage:/qdrant/storage

启动后:

  • Web UI:http://localhost:6333/dashboard
  • HTTP API:http://localhost:6333
  • gRPC:localhost:6334

2.2 基本概念

  • Collection(集合):向量的集合,类似于关系数据库的表
  • Point(点):单个向量及其关联的 payload(元数据)
  • Vector(向量):数值数组,表示数据的嵌入表示
  • Payload(负载):与向量关联的元数据,可用于过滤

三、Spring Boot 集成 Qdrant

3.1 添加依赖

在 pom.xml 中添加 Qdrant Java 客户端依赖:

<dependencies>
<!– Qdrant Java Client –>
<dependency>
<groupId>io.qdrant</groupId>
<artifactId>qdrant-java-client</artifactId>
<version>1.7.0</version>
</dependency>

<!– Spring Boot Web (可选,用于构建 REST API) –>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>

<!– Lombok (简化代码) –>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<optional>true</optional>
</dependency>
</dependencies>

3.2 配置 Qdrant 客户端

创建配置类来初始化 Qdrant 客户端:

package com.example.qdrant.config;

import io.qdrant.client.QdrantClient;
import io.qdrant.client.QdrantGrpcClient;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
public class QdrantConfig {

@Value("${qdrant.host:localhost}")
private String host;

@Value("${qdrant.port:6334}")
private int port;

@Value("${qdrant.api-key:}")
private String apiKey;

@Bean
public QdrantClient qdrantClient() {
QdrantGrpcClient.Builder builder = QdrantGrpcClient.newBuilder()
.host(host)
.port(port);

if (!apiKey.isEmpty()) {
builder.withApiKey(apiKey);
}

return builder.build();
}
}

在 application.yml 中添加配置:

qdrant:
host: localhost
port: 6334
api-key: # 如果设置了 API Key

spring:
application:
name: qdrantdemo

3.3 创建 Collection 服务

package com.example.qdrant.service;

import io.qdrant.client.QdrantClient;
import io.qdrant.client.grpc.Collections;
import io.qdrant.client.grpc.Points;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.springframework.stereotype.Service;

import java.util.List;

@Slf4j
@Service
@RequiredArgsConstructor
public class QdrantCollectionService {

private final QdrantClient qdrantClient;

/**
* 创建集合
*/

public void createCollection(String collectionName, int vectorSize) throws Exception {
Collections.VectorParams vectorParams = Collections.VectorParams.newBuilder()
.setSize(vectorSize)
.setDistance(Collections.Distance.Cosine)
.build();

qdrantClient.createCollectionAsync(
collectionName,
Collections.CreateCollection.newBuilder()
.setVectorsConfig(vectorParams)
.build()
).get();

log.info("Collection '{}' created successfully", collectionName);
}

/**
* 删除集合
*/

public void deleteCollection(String collectionName) throws Exception {
qdrantClient.deleteCollectionAsync(collectionName).get();
log.info("Collection '{}' deleted successfully", collectionName);
}

/**
* 检查集合是否存在
*/

public boolean collectionExists(String collectionName) throws Exception {
Collections.CollectionInfo info = qdrantClient.getCollectionInfoAsync(collectionName).get();
return info != null;
}

/**
* 获取集合信息
*/

public Collections.CollectionInfo getCollectionInfo(String collectionName) throws Exception {
return qdrantClient.getCollectionInfoAsync(collectionName).get();
}
}

3.4 创建 Point 管理服务

package com.example.qdrant.service;

import io.qdrant.client.QdrantClient;
import io.qdrant.client.grpc.Points;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.springframework.stereotype.Service;

import java.util.List;
import java.util.Map;

@Slf4j
@Service
@RequiredArgsConstructor
public class QdrantPointService {

private final QdrantClient qdrantClient;

/**
* 插入/更新向量点
*/

public void upsertPoints(String collectionName, List<PointData> points) throws Exception {
List<Points.PointStruct> pointStructs = points.stream()
.map(this::convertToPointStruct)
.toList();

qdrantClient.upsertPointAsync(
collectionName,
Points.UpsertPoints.newBuilder()
.addAllPoints(pointStructs)
.build()
).get();

log.info("Successfully upserted {} points to collection '{}'", points.size(), collectionName);
}

/**
* 搜索向量
*/

public List<SearchResult> search(String collectionName,
List<Float> vector,
int limit,
Map<String, String> filter) throws Exception {
Points.SearchPoints.Builder searchBuilder = Points.SearchPoints.newBuilder()
.addAllVector(vector)
.setLimit(limit)
.withVectorSelector(Points.QueryVector.newBuilder().build());

// 添加过滤条件
if (filter != null && !filter.isEmpty()) {
Points.Filter filterBuilder = Points.Filter.newBuilder()
.addMust(Points.Condition.newBuilder()
.setField(Points.FieldCondition.newBuilder()
.setKey("category")
.setMatch(Points.Match.newBuilder()
.setTextValue(filter.get("category"))
.build())
.build())
.build())
.build();
searchBuilder.setFilter(filterBuilder);
}

List<Points.RetrievedPoint> results = qdrantClient.searchPointAsync(
collectionName,
searchBuilder.build()
).get();

return results.stream()
.map(this::convertToSearchResult)
.toList();
}

/**
* 删除点
*/

public void deletePoints(String collectionName, List<Long> ids) throws Exception {
Points.PointsSelector selector = Points.PointsSelector.newBuilder()
.setPointsSelector(Points.PointsIdsList.newBuilder()
.addAllIds(ids.stream()
.map(id -> Points.PointId.newBuilder().setNum(id).build())
.toList())
.build())
.build();

qdrantClient.deletePointAsync(collectionName, selector).get();
log.info("Successfully deleted {} points from collection '{}'", ids.size(), collectionName);
}

private Points.PointStruct convertToPointStruct(PointData pointData) {
Points.PointStruct.Builder builder = Points.PointStruct.newBuilder()
.setId(Points.PointId.newBuilder().setNum(pointData.getId()).build())
.addAllVector(pointData.getVector());

// 添加 payload
if (pointData.getPayload() != null) {
pointData.getPayload().forEach((key, value) -> {
builder.putPayload(key, Points.Value.newBuilder()
.setStringValue(value.toString())
.build());
});
}

return builder.build();
}

private SearchResult convertToSearchResult(Points.RetrievedPoint retrievedPoint) {
return SearchResult.builder()
.id(retrievedPoint.getId().getNum())
.score(retrievedPoint.getScore())
.payload(retrievedPoint.getPayloadMap())
.build();
}
}

// DTO 类
@Data
@Builder
public class PointData {
private Long id;
private List<Float> vector;
private Map<String, Object> payload;
}

@Data
@Builder
public class SearchResult {
private Long id;
private float score;
private Map<String, Points.Value> payload;
}


四、与 Spring AI 集成

Spring AI 是 Spring 生态系统中新兴的 AI 框架,提供了与各种 LLM 和向量数据库集成的统一接口。

4.1 添加依赖

<dependency>
<groupId>org.springframework.ai</groupId>
<artifactId>spring-ai-qdrant-spring-boot-starter</artifactId>
<version>1.0.0-M4</version>
</dependency>

<dependency>
<groupId>org.springframework.ai</groupId>
<artifactId>spring-ai-openai-spring-boot-starter</artifactId>
<version>1.0.0-M4</version>
</dependency>

4.2 配置 Spring AI

spring:
ai:
# OpenAI 配置(用于生成嵌入)
openai:
api-key: ${OPENAI_API_KEY}

# Qdrant 配置
vectorstore:
qdrant:
host: localhost
port: 6334
collection-name: documents
initialize-schema: true

4.3 创建 RAG 服务

package com.example.qdrant.service;

import org.springframework.ai.document.Document;
import org.springframework.ai.qdrant.QdrantVectorStore;
import org.springframework.ai.reader.TextReader;
import org.springframework.ai.transformer.splitter.TokenTextSplitter;
import org.springframework.ai.vectorstore.VectorStore;
import org.springframework.core.io.Resource;
import org.springframework.stereotype.Service;

import java.util.List;
import java.util.Map;

@Service
@RequiredArgsConstructor
public class RAGService {

private final VectorStore vectorStore;
private final QdrantVectorStore qdrantVectorStore;

/**
* 加载并存储文档
*/

public void loadAndStoreDocuments(Resource resource) throws Exception {
// 读取文档
TextReader textReader = new TextReader(resource);
List<Document> documents = textReader.get();

// 分割文档
TokenTextSplitter splitter = new TokenTextSplitter();
List<Document> splitDocuments = splitter.apply(documents);

// 存储到向量数据库
vectorStore.add(splitDocuments);

log.info("Stored {} document chunks in Qdrant", splitDocuments.size());
}

/**
* 相似度搜索
*/

public List<Document> similaritySearch(String query, int topK) {
return vectorStore.similaritySearch(
org.springframework.ai.vectorstore.SearchRequest.query(query)
.withTopK(topK)
);
}

/**
* 带过滤的相似度搜索
*/

public List<Document> similaritySearchWithFilter(String query,
int topK,
String category) {
return vectorStore.similaritySearch(
org.springframework.ai.vectorstore.SearchRequest.query(query)
.withTopK(topK)
.withFilterExpression("category == '" + category + "'")
);
}

/**
* 删除文档
*/

public void deleteDocuments(List<String> ids) {
vectorStore.delete(ids);
}
}

4.4 创建聊天控制器

package com.example.qdrant.controller;

import org.springframework.ai.chat.messages.UserMessage;
import org.springframework.ai.chat.model.ChatResponse;
import org.springframework.ai.chat.prompt.Prompt;
import org.springframework.ai.openai.OpenAiChatModel;
import org.springframework.ai.vectorstore.VectorStore;
import org.springframework.ai.vectorstore.SearchRequest;
import org.springframework.web.bind.annotation.*;

@RestController
@RequestMapping("/api/chat")
@RequiredArgsConstructor
public class ChatController {

private final OpenAiChatModel chatModel;
private final VectorStore vectorStore;

@PostMapping
public String chat(@RequestBody ChatRequest request) {
// 1. 检索相关文档
List<Document> relevantDocs = vectorStore.similaritySearch(
SearchRequest.query(request.getMessage())
.withTopK(3)
);

// 2. 构建增强提示词
String context = relevantDocs.stream()
.map(Document::getContent)
.collect(Collectors.joining("\\n\\n"));

String enhancedPrompt = String.format(
"Based on the following context:\\n\\n%s\\n\\nAnswer the question: %s",
context, request.getMessage()
);

// 3. 调用 LLM
ChatResponse response = chatModel.call(new Prompt(new UserMessage(enhancedPrompt)));

return response.getResult().getOutput().getContent();
}
}


五、与 LangChain4j 集成

LangChain4j 是 LangChain 的 Java 实现,提供了丰富的 AI 应用构建能力。

5.1 添加依赖

<dependencies>
<!– LangChain4j Qdrant –>
<dependency>
<groupId>dev.langchain4j</groupId>
<artifactId>langchain4j-qdrant</artifactId>
<version>0.34.0</version>
</dependency>

<!– LangChain4j OpenAI –>
<dependency>
<groupId>dev.langchain4j</groupId>
<artifactId>langchain4j-open-ai</artifactId>
<version>0.34.0</version>
</dependency>
</dependencies>

5.2 配置 Qdrant Embedding Store

package com.example.qdrant.config;

import dev.langchain4j.data.segment.TextSegment;
import dev.langchain4j.model.openai.OpenAiEmbeddingModel;
import dev.langchain4j.model.openai.OpenAiChatModel;
import dev.langchain4j.store.embedding.qdrant.QdrantEmbeddingStore;
import io.qdrant.client.QdrantClient;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
public class LangChainConfig {

@Value("${langchain4j.openai.api-key}")
private String openAiApiKey;

@Value("${qdrant.host:localhost}")
private String qdrantHost;

@Value("${qdrant.port:6334}")
private int qdrantPort;

@Bean
public QdrantEmbeddingStore qdrantEmbeddingStore(QdrantClient qdrantClient) {
return QdrantEmbeddingStore.builder()
.host(qdrantHost)
.port(qdrantPort)
.collectionName("langchain_docs")
.build();
}

@Bean
public OpenAiEmbeddingModel embeddingModel() {
return OpenAiEmbeddingModel.builder()
.apiKey(openAiApiKey)
.build();
}

@Bean
public OpenAiChatModel chatModel() {
return OpenAiChatModel.builder()
.apiKey(openAiApiKey)
.build();
}

@Bean
public ConversationRetriever conversationRetriever(
QdrantEmbeddingStore embeddingStore,
OpenAiEmbeddingModel embeddingModel) {

return EmbeddingStoreRetriever.builder()
.embeddingStore(embeddingStore)
.embeddingModel(embeddingModel)
.maxResults(3)
.minScore(0.7)
.build();
}
}

5.3 实现 RAG 服务

package com.example.qdrant.service;

import dev.langchain4j.data.document.Document;
import dev.langchain4j.data.document.DocumentSplitter;
import dev.langchain4j.data.document.splitter.DocumentSplitters;
import dev.langchain4j.data.segment.TextSegment;
import dev.langchain4j.model.openai.OpenAiChatModel;
import dev.langchain4j.model.openai.OpenAiEmbeddingModel;
import dev.langchain4j.store.embedding.EmbeddingStore;
import dev.langchain4j.store.embedding.EmbeddingStoreIngestor;
import dev.langchain4j.store.embedding.qdrant.QdrantEmbeddingStore;
import dev.langchain4j.service.AiServices;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.springframework.stereotype.Service;

import java.nio.file.Paths;
import java.util.List;

@Slf4j
@Service
@RequiredArgsConstructor
public class LangChainRAGService {

private final QdrantEmbeddingStore embeddingStore;
private final OpenAiEmbeddingModel embeddingModel;
private final OpenAiChatModel chatModel;

/**
* 加载文档到向量存储
*/

public void ingestDocuments(String filePath) throws Exception {
// 创建文档摄入器
EmbeddingStoreIngestor ingestor = EmbeddingStoreIngestor.builder()
.embeddingStore(embeddingStore)
.embeddingModel(embeddingModel)
.textSegmentSplitter(DocumentSplitters.recursive(300, 30))
.build();

// 加载文档
Document document = FileSystemDocumentLoader.loadDocument(Paths.get(filePath));

// 摄入文档
ingestor.ingest(document);

log.info("Document ingested successfully");
}

/**
* RAG 聊天助手
*/

public String chat(String message) {
// 定义 AI 服务接口
interface Assistant {
String chat(String userMessage);
}

// 构建带检索的 AI 服务
Assistant assistant = AiServices.builder(Assistant.class)
.chatLanguageModel(chatModel)
.retriever(EmbeddingStoreRetriever.from(
embeddingStore,
embeddingModel,
3,
0.7
))
.build();

return assistant.chat(message);
}

/**
* 语义搜索
*/

public List<TextSegment> semanticSearch(String query, int topK) {
List<EmbeddingMatch<TextSegment>> matches = embeddingStore.findRelevant(
embeddingModel.embed(query).content(),
topK
);

return matches.stream()
.map(EmbeddingMatch::embedded)
.toList();
}
}


六、实战案例:构建智能文档问答系统

6.1 系统架构

┌─────────────┐
│ 用户查询 │
└──────┬──────┘


┌─────────────────────┐
│ REST API 层 │
├─────────────────────┤
│ /api/chat │
│ /api/documents │
└──────┬──────────────┘


┌─────────────────────┐
│ 业务服务层 │
├─────────────────────┤
│ – RAGService │
│ – DocumentService │
└──────┬──────────────┘


┌──────────────────────────┐
│ AI 框架层 │
├──────────────────────────┤
│ – Spring AI / LangChain4j│
│ – Embedding Model │
│ – Chat Model │
└──────┬───────────────────┘


┌──────────────────────────┐
│ Qdrant 向量数据库 │
├──────────────────────────┤
│ – Collection: documents │
│ – Vector Search │
└──────────────────────────┘

6.2 完整实现

package com.example.qdrant;

@SpringBootApplication
@EnableAiServices // LangChain4j
public class QdrantDemoApplication {
public static void main(String[] args) {
SpringApplication.run(QdrantDemoApplication.class, args);
}
}

@RestController
@RequestMapping("/api")
@RequiredArgsConstructor
public class DocumentController {

private final RAGService ragService;
private final LangChainRAGService langChainRAGService;

/**
* 上传文档
*/

@PostMapping("/documents/upload")
public ResponseEntity<String> uploadDocument(@RequestParam("file") MultipartFile file) {
try {
Path tempFile = Files.createTempFile("upload", ".txt");
file.transferTo(tempFile);

// 使用 Spring AI 方式
// ragService.loadAndStoreDocuments(new FileSystemResource(tempFile));

// 使用 LangChain4j 方式
langChainRAGService.ingestDocuments(tempFile.toString());

return ResponseEntity.ok("Document uploaded and processed successfully");
} catch (Exception e) {
log.error("Error processing document", e);
return ResponseEntity.status(500).body("Error processing document");
}
}

/**
* 问答接口
*/

@PostMapping("/chat")
public ResponseEntity<ChatResponse> chat(@RequestBody ChatRequest request) {
String answer = langChainRAGService.chat(request.getMessage());
return ResponseEntity.ok(new ChatResponse(answer));
}

/**
* 语义搜索
*/

@GetMapping("/search")
public ResponseEntity<List<SearchResult>> search(
@RequestParam String query,
@RequestParam(defaultValue = "5") int topK) {

List<TextSegment> results = langChainRAGService.semanticSearch(query, topK);

List<SearchResult> searchResults = results.stream()
.map(segment -> new SearchResult(segment.text(), null))
.toList();

return ResponseEntity.ok(searchResults);
}
}


七、最佳实践与性能优化

7.1 向量维度选择

  • text-embedding-ada-002 (OpenAI): 1536 维
  • all-MiniLM-L6-v2: 384 维
  • paraphrase-multilingual-MiniLM-L12-v2: 384 维(多语言)

建议:根据模型选择合适的维度,维度越高精度越高但存储和搜索成本也越高。

7.2 分片策略

// 创建分片集合
Collections.CreateCollection createCollection = Collections.CreateCollection.newBuilder()
.setVectorsConfig(vectorParams)
.setShardNumber(4) // 4 个分片
.setReplicationFactor(2) // 每个分片 2 个副本
.build();

7.3 索引参数调优

Collections.HnswConfigDiff hnswConfig = Collections.HnswConfigDiff.newBuilder()
.setM(16) // 每个节点连接数(范围 2-100)
.setEfConstruct(100) // 构建索引时的搜索深度
.setFullScanThreshold(10000) // 触发全扫描的向量数量阈值
.build();

7.4 批量操作优化

// 批量插入时控制批次大小
public void batchUpsert(String collectionName, List<PointData> allPoints) throws Exception {
int batchSize = 100;
List<List<PointData>> batches = Lists.partition(allPoints, batchSize);

for (List<PointData> batch : batches) {
upsertPoints(collectionName, batch);
Thread.sleep(100); // 避免 QPS 过高
}
}


八、总结

Qdrant 作为一款现代化的向量数据库,具有以下优势:

  • 高性能:基于 Rust 实现,性能出色
  • 易集成:提供多语言客户端,与 Spring AI、LangChain 等框架集成良好
  • 功能丰富:支持过滤、分片、复制等企业级特性
  • 开源免费:完全开源,无供应商锁定
  • 通过本文的介绍,你应该能够:

    • 理解 Qdrant 的核心概念和特性
    • 在 Spring Boot 项目中集成 Qdrant
    • 配合 Spring AI 和 LangChain4j 构建 RAG 应用
    • 掌握基本的性能优化技巧

    接下来,建议你:

  • 从简单的语义搜索开始实践
  • 逐步构建完整的 RAG 应用
  • 根据业务需求优化向量维度、索引参数等配置
  • 参考资源

    • Qdrant 官方文档: https://qdrant.tech/documentation/
    • Spring AI 文档: https://docs.spring.io/spring-ai/reference/
    • LangChain4j 文档: https://docs.langchain4j.dev/
    • Qdrant Java Client: https://github.com/qdrant/qdrant-client-java

    赞(0)
    未经允许不得转载:171主机测评 » Qdrant 向量数据库完全指南:从入门到 Spring AI/LangChain4J 集成实践
    分享到: 更多 (0)

    评论 抢沙发

    • 昵称 (必填)
    • 邮箱 (必填)
    • 网址