Elasticsearch 相关性评分：BM25、Function Score 与自定义评分实战应用

1. BM25 算法解析：Elasticsearch 的默认相关性评分机制

BM25 (Best Match 25) 是 Elasticsearch 默认使用的相关性评分算法，是 BM 算法的改进版本。BM25 基于概率检索模型，通过计算查询词在文档中的出现频率与文档长度之间的关系来评估文档相关性。

BM25 的核心公式为：

score(D,Q) = ∑ IDF(q_i) * (f(q_i) * (k1 + 1)) / (f(q_i) + k1 * (1 – b + b * |D|/avgdl))

其中：

• IDF(q_i) 是查询词 q_i 的逆文档频率
• f(q_i) 是词 q_i 在文档 D 中的出现频率
• |D| 是文档 D 的长度
• avgdl 是平均文档长度
• k1 和 b 是调节参数，Elasticsearch 默认值分别为 1.2 和 0.75

BM25 的主要优势在于平衡了词频和文档长度对相关性的影响，避免了传统 TF-IDF 算法中长文档获得不公平高评分的问题。

在 Elasticsearch 中，我们可以通过查询 API 直接使用 BM25 算法：

GET /my_index/_search
{
"query": {
"match": {
"content": "elasticsearch search algorithm"
}
}
}

BM25 算法的参数可以通过 similarities 设置进行调整：

PUT /my_index
{
"settings": {
"similarity": {
"default": {
"type": "BM25",
"k1": 1.2,
"b": 0.75
}
}
}
}

2. Function Score 扩展：灵活调整文档评分的方法

虽然 BM25 提供了基础的评分机制，但在实际业务场景中，我们可能需要根据特定规则调整文档的相关性评分。Elasticsearch 的 Function Score 允许我们在查询过程中对文档进行二次评分和调整。

Function Score 的基本结构如下：

GET /my_index/_search
{
"query": {
"function_score": {
"query": {
"match": {
"content": "elasticsearch"
}
},
"functions": [
{
"filter": { "term": { "category": "tech" } },
"weight": 2
},
{
"filter": { "range": { "date": { "gte": "now-30d" } } },
"weight": 1.5
}
],
"score_mode": "sum",
"boost_mode": "multiply"
}
}
}

Function Score 支持多种函数类型：

Function Score 通过 score_mode 和 boost_mode 参数控制如何组合多个函数的结果：

• score_mode: 定义如何组合多个函数的分数（sum, multiply, max, min, replace）
• boost_mode: 定义如何将函数结果与原始查询分数结合（multiply, replace, sum, avg, max, min）

3. 自定义评分模型：Script Score 与高级评分策略

当 Function Score 提供的函数无法满足复杂业务需求时，Elasticsearch 的 Script Score 允许我们使用自定义脚本实现更灵活的评分策略。

Script Score 的基本语法：

GET /my_index/_search
{
"query": {
"function_score": {
"query": { "match_all": {} },
"functions": [
{
"script_score": {
"script": {
"source": "doc['rating'].value * doc['review_count'].value / 1000"
}
}
}
]
}
}
}

Elasticsearch 支持多种脚本语言（Painless, JavaScript, 等），其中 Painless 是 Elasticsearch 5.0+ 默认且推荐使用的脚本语言。

自定义评分模型的设计应遵循以下原则：

4. 实战案例：综合应用评分优化搜索体验

下面是一个综合应用 BM25、Function Score 和 Script Score 的实战案例，实现一个电商产品搜索的相关性优化：

业务场景：

一个电商平台需要根据用户搜索"笔记本电脑"返回最相关的产品，同时考虑以下因素：

实现方案：

GET /products/_search
{
"query": {
"function_score": {
"query": {
"multi_match": {
"query": "笔记本电脑",
"fields": [
"name^4",
"description^2",
"category^3",
"brand^2"
],
"type": "best_fields",
"minimum_should_match": "30%"
}
},
"functions": [
{
"filter": { "term": { "category": "computer" } },
"weight": 2
},
{
"filter": { "term": { "category": "electronics" } },
"weight": 1.5
},
{
"filter": { "term": { "in_stock": true } },
"weight": 1.8
},
{
"field_value_factor": {
"field": "rating",
"modifier": "log1p",
"factor": 1.5,
"missing": 1
}
},
{
"gauss": {
"上架时间": {
"origin": "now",
"scale": "30d",
"offset": "7d",
"decay": 0.5
}
}
}
],
"score_mode": "sum",
"boost_mode": "multiply",
"max_boost": 10
}
},
"size": 20
}

效果分析：

通过上述评分策略，我们实现了：

5. 最佳实践与注意事项

在使用 Elasticsearch 相关性评分时，需要注意以下事项：

Elasticsearch 相关性评分处理流程图：

#publish-mermaid-0-1789103252384{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#publish-mermaid-0-1789103252384 .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#publish-mermaid-0-1789103252384 .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#publish-mermaid-0-1789103252384 .error-icon{fill:#552222;}#publish-mermaid-0-1789103252384 .error-text{fill:#552222;stroke:#552222;}#publish-mermaid-0-1789103252384 .edge-thickness-normal{stroke-width:1px;}#publish-mermaid-0-1789103252384 .edge-thickness-thick{stroke-width:3.5px;}#publish-mermaid-0-1789103252384 .edge-pattern-solid{stroke-dasharray:0;}#publish-mermaid-0-1789103252384 .edge-thickness-invisible{stroke-width:0;fill:none;}#publish-mermaid-0-1789103252384 .edge-pattern-dashed{stroke-dasharray:3;}#publish-mermaid-0-1789103252384 .edge-pattern-dotted{stroke-dasharray:2;}#publish-mermaid-0-1789103252384 .marker{fill:#333333;stroke:#333333;}#publish-mermaid-0-1789103252384 .marker.cross{stroke:#333333;}#publish-mermaid-0-1789103252384 svg{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;}#publish-mermaid-0-1789103252384 p{margin:0;}#publish-mermaid-0-1789103252384 .label{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;color:#333;}#publish-mermaid-0-1789103252384 .cluster-label text{fill:#333;}#publish-mermaid-0-1789103252384 .cluster-label span{color:#333;}#publish-mermaid-0-1789103252384 .cluster-label span p{background-color:transparent;}#publish-mermaid-0-1789103252384 .label text,#publish-mermaid-0-1789103252384 span{fill:#333;color:#333;}#publish-mermaid-0-1789103252384 .node rect,#publish-mermaid-0-1789103252384 .node circle,#publish-mermaid-0-1789103252384 .node ellipse,#publish-mermaid-0-1789103252384 .node polygon,#publish-mermaid-0-1789103252384 .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#publish-mermaid-0-1789103252384 .rough-node .label text,#publish-mermaid-0-1789103252384 .node .label text,#publish-mermaid-0-1789103252384 .image-shape .label,#publish-mermaid-0-1789103252384 .icon-shape .label{text-anchor:middle;}#publish-mermaid-0-1789103252384 .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#publish-mermaid-0-1789103252384 .rough-node .label,#publish-mermaid-0-1789103252384 .node .label,#publish-mermaid-0-1789103252384 .image-shape .label,#publish-mermaid-0-1789103252384 .icon-shape .label{text-align:center;}#publish-mermaid-0-1789103252384 .node.clickable{cursor:pointer;}#publish-mermaid-0-1789103252384 .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#publish-mermaid-0-1789103252384 .arrowheadPath{fill:#333333;}#publish-mermaid-0-1789103252384 .edgePath .path{stroke:#333333;stroke-width:1px;}#publish-mermaid-0-1789103252384 .flowchart-link{stroke:#333333;fill:none;}#publish-mermaid-0-1789103252384 .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#publish-mermaid-0-1789103252384 .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#publish-mermaid-0-1789103252384 .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#publish-mermaid-0-1789103252384 .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#publish-mermaid-0-1789103252384 .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#publish-mermaid-0-1789103252384 .cluster text{fill:#333;}#publish-mermaid-0-1789103252384 .cluster span{color:#333;}#publish-mermaid-0-1789103252384 div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#publish-mermaid-0-1789103252384 .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#publish-mermaid-0-1789103252384 rect.text{fill:none;stroke-width:0;}#publish-mermaid-0-1789103252384 .icon-shape,#publish-mermaid-0-1789103252384 .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#publish-mermaid-0-1789103252384 .icon-shape p,#publish-mermaid-0-1789103252384 .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#publish-mermaid-0-1789103252384 .icon-shape .label rect,#publish-mermaid-0-1789103252384 .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#publish-mermaid-0-1789103252384 .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#publish-mermaid-0-1789103252384 .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#publish-mermaid-0-1789103252384 .node .neo-node{stroke:#9370DB;}#publish-mermaid-0-1789103252384 [data-look=\”neo\”].node rect,#publish-mermaid-0-1789103252384 [data-look=\”neo\”].cluster rect,#publish-mermaid-0-1789103252384 [data-look=\”neo\”].node polygon{stroke:#9370DB;filter:drop-shadow(1px 2px 2px rgba(185, 185, 185, 1));}#publish-mermaid-0-1789103252384 [data-look=\”neo\”].swimlane.cluster rect{filter:none;}#publish-mermaid-0-1789103252384 [data-look=\”neo\”].node path{stroke:#9370DB;stroke-width:1px;}#publish-mermaid-0-1789103252384 [data-look=\”neo\”].node .outer-path{filter:drop-shadow(1px 2px 2px rgba(185, 185, 185, 1));}#publish-mermaid-0-1789103252384 [data-look=\”neo\”].node .neo-line path{stroke:#9370DB;filter:none;}#publish-mermaid-0-1789103252384 [data-look=\”neo\”].node circle{stroke:#9370DB;filter:drop-shadow(1px 2px 2px rgba(185, 185, 185, 1));}#publish-mermaid-0-1789103252384 [data-look=\”neo\”].node circle .state-start{fill:#000000;}#publish-mermaid-0-1789103252384 [data-look=\”neo\”].icon-shape .icon{fill:#9370DB;filter:drop-shadow(1px 2px 2px rgba(185, 185, 185, 1));}#publish-mermaid-0-1789103252384 [data-look=\”neo\”].icon-shape .icon-neo path{stroke:#9370DB;filter:drop-shadow(1px 2px 2px rgba(185, 185, 185, 1));}#publish-mermaid-0-1789103252384 :root{–mermaid-font-family:\”trebuchet ms\”,verdana,arial,sans-serif;}

用户输入查询

文档分词处理

计算 BM25 基础分

应用 Function Score 调整

评估自定义评分脚本

综合计算最终相关性得分

按相关性排序返回结果

最小示例代码

// 基础 BM25 查询
GET /products/_search
{
"query": {
"match": {
"description": "laptop computer"
}
}
}
// 使用 Function Score 调整
GET /products/_search
{
"query": {
"function_score": {
"query": {
"match": {
"description": "laptop computer"
}
},
"functions": [
{
"filter": { "term": { "in_stock": true } },
"weight": 1.5
}
],
"boost_mode": "multiply"
}
}
}
// 使用 Script Score 完全自定义
GET /products/_search
{
"query": {
"function_score": {
"query": { "match_all": {} },
"functions": [
{
"script_score": {
"script": {
"source": "
double score = 0;
score += Math.min(doc['description.keyword'].value.length(), 200) / 200;
score += doc['rating'].value / 5;
score += Math.log(doc['sales_count'].value + 1) / 10;
return score;
",
"lang": "painless"
}
}
}
],
"boost_mode": "replace"
}
}
}