深入剖析 Kafka 事务消息：从设计到实现

深入剖析 Kafka 事务消息：从设计到实现

• • 1. 引言：为什么 Kafka 需要事务？
  • 2. 事务核心概念
  • • 2.1 事务的四大目标
    • 2.2 关键术语解析
  • 3. 事务全景架构
  • 4. 事务核心机制
  • • 4.1 幂等性基础：事务的基石
    • 4.2 僵尸实例防护：Epoch 机制
    • 4.3 控制消息：事务边界的标记
  • 5. 事务完整流程详解
  • • 5.1 阶段一：事务初始化
    • 5.2 阶段二：消息发送与分区注册
    • 5.3 阶段三：事务提交（两阶段提交）
    • 5.4 消费者隔离：消息可见性控制
  • 6. 事务使用示例
  • • 6.1 生产者配置与代码
    • 6.2 Consume-Process-Produce 模式
    • 6.3 消费者配置
  • 7. 事务限制与性能考量
  • • 7.1 功能限制
    • 7.2 性能开销分析
  • 8. 总结

跨分区原子写入与流处理 Exactly-Once 的底层密码

1. 引言：为什么 Kafka 需要事务？

在 Kafka 0.11.0 之前，开发者长期面临两大痛点：

Kafka 在 0.11.0 版本引入的事务机制，正是为了解决上述问题，实现 Exactly-Once 语义（精确一次处理） 的核心基础设施。

2. 事务核心概念

2.1 事务的四大目标

2.2 关键术语解析

#mermaid-svg-ilBeJLguDDwnCGxo{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-ilBeJLguDDwnCGxo .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-ilBeJLguDDwnCGxo .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-ilBeJLguDDwnCGxo .error-icon{fill:#552222;}#mermaid-svg-ilBeJLguDDwnCGxo .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-ilBeJLguDDwnCGxo .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-ilBeJLguDDwnCGxo .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-ilBeJLguDDwnCGxo .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-ilBeJLguDDwnCGxo .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-ilBeJLguDDwnCGxo .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-ilBeJLguDDwnCGxo .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-ilBeJLguDDwnCGxo .marker{fill:#333333;stroke:#333333;}#mermaid-svg-ilBeJLguDDwnCGxo .marker.cross{stroke:#333333;}#mermaid-svg-ilBeJLguDDwnCGxo svg{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-ilBeJLguDDwnCGxo p{margin:0;}#mermaid-svg-ilBeJLguDDwnCGxo .label{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;color:#333;}#mermaid-svg-ilBeJLguDDwnCGxo .cluster-label text{fill:#333;}#mermaid-svg-ilBeJLguDDwnCGxo .cluster-label span{color:#333;}#mermaid-svg-ilBeJLguDDwnCGxo .cluster-label span p{background-color:transparent;}#mermaid-svg-ilBeJLguDDwnCGxo .label text,#mermaid-svg-ilBeJLguDDwnCGxo span{fill:#333;color:#333;}#mermaid-svg-ilBeJLguDDwnCGxo .node rect,#mermaid-svg-ilBeJLguDDwnCGxo .node circle,#mermaid-svg-ilBeJLguDDwnCGxo .node ellipse,#mermaid-svg-ilBeJLguDDwnCGxo .node polygon,#mermaid-svg-ilBeJLguDDwnCGxo .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-ilBeJLguDDwnCGxo .rough-node .label text,#mermaid-svg-ilBeJLguDDwnCGxo .node .label text,#mermaid-svg-ilBeJLguDDwnCGxo .image-shape .label,#mermaid-svg-ilBeJLguDDwnCGxo .icon-shape .label{text-anchor:middle;}#mermaid-svg-ilBeJLguDDwnCGxo .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-ilBeJLguDDwnCGxo .rough-node .label,#mermaid-svg-ilBeJLguDDwnCGxo .node .label,#mermaid-svg-ilBeJLguDDwnCGxo .image-shape .label,#mermaid-svg-ilBeJLguDDwnCGxo .icon-shape .label{text-align:center;}#mermaid-svg-ilBeJLguDDwnCGxo .node.clickable{cursor:pointer;}#mermaid-svg-ilBeJLguDDwnCGxo .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-ilBeJLguDDwnCGxo .arrowheadPath{fill:#333333;}#mermaid-svg-ilBeJLguDDwnCGxo .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-ilBeJLguDDwnCGxo .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-ilBeJLguDDwnCGxo .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-ilBeJLguDDwnCGxo .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-ilBeJLguDDwnCGxo .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-ilBeJLguDDwnCGxo .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-ilBeJLguDDwnCGxo .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-ilBeJLguDDwnCGxo .cluster text{fill:#333;}#mermaid-svg-ilBeJLguDDwnCGxo .cluster span{color:#333;}#mermaid-svg-ilBeJLguDDwnCGxo div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-ilBeJLguDDwnCGxo .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-ilBeJLguDDwnCGxo rect.text{fill:none;stroke-width:0;}#mermaid-svg-ilBeJLguDDwnCGxo .icon-shape,#mermaid-svg-ilBeJLguDDwnCGxo .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-ilBeJLguDDwnCGxo .icon-shape p,#mermaid-svg-ilBeJLguDDwnCGxo .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-ilBeJLguDDwnCGxo .icon-shape .label rect,#mermaid-svg-ilBeJLguDDwnCGxo .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-ilBeJLguDDwnCGxo .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-ilBeJLguDDwnCGxo .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-ilBeJLguDDwnCGxo :root{–mermaid-font-family:\”trebuchet ms\”,verdana,arial,sans-serif;}

服务端管理

用户可见

1:1 映射

关联

关联

Transactional.Id用户配置，业务标识

Producer ID服务端分配，内部标识

Producer Epoch版本号，防僵尸实例

Sequence Number消息序列号，幂等保证

3. 事务全景架构

#mermaid-svg-GUE0SwMa2fkUAVkO{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-GUE0SwMa2fkUAVkO .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-GUE0SwMa2fkUAVkO .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-GUE0SwMa2fkUAVkO .error-icon{fill:#552222;}#mermaid-svg-GUE0SwMa2fkUAVkO .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-GUE0SwMa2fkUAVkO .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-GUE0SwMa2fkUAVkO .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-GUE0SwMa2fkUAVkO .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-GUE0SwMa2fkUAVkO .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-GUE0SwMa2fkUAVkO .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-GUE0SwMa2fkUAVkO .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-GUE0SwMa2fkUAVkO .marker{fill:#333333;stroke:#333333;}#mermaid-svg-GUE0SwMa2fkUAVkO .marker.cross{stroke:#333333;}#mermaid-svg-GUE0SwMa2fkUAVkO svg{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-GUE0SwMa2fkUAVkO p{margin:0;}#mermaid-svg-GUE0SwMa2fkUAVkO .label{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;color:#333;}#mermaid-svg-GUE0SwMa2fkUAVkO .cluster-label text{fill:#333;}#mermaid-svg-GUE0SwMa2fkUAVkO .cluster-label span{color:#333;}#mermaid-svg-GUE0SwMa2fkUAVkO .cluster-label span p{background-color:transparent;}#mermaid-svg-GUE0SwMa2fkUAVkO .label text,#mermaid-svg-GUE0SwMa2fkUAVkO span{fill:#333;color:#333;}#mermaid-svg-GUE0SwMa2fkUAVkO .node rect,#mermaid-svg-GUE0SwMa2fkUAVkO .node circle,#mermaid-svg-GUE0SwMa2fkUAVkO .node ellipse,#mermaid-svg-GUE0SwMa2fkUAVkO .node polygon,#mermaid-svg-GUE0SwMa2fkUAVkO .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-GUE0SwMa2fkUAVkO .rough-node .label text,#mermaid-svg-GUE0SwMa2fkUAVkO .node .label text,#mermaid-svg-GUE0SwMa2fkUAVkO .image-shape .label,#mermaid-svg-GUE0SwMa2fkUAVkO .icon-shape .label{text-anchor:middle;}#mermaid-svg-GUE0SwMa2fkUAVkO .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-GUE0SwMa2fkUAVkO .rough-node .label,#mermaid-svg-GUE0SwMa2fkUAVkO .node .label,#mermaid-svg-GUE0SwMa2fkUAVkO .image-shape .label,#mermaid-svg-GUE0SwMa2fkUAVkO .icon-shape .label{text-align:center;}#mermaid-svg-GUE0SwMa2fkUAVkO .node.clickable{cursor:pointer;}#mermaid-svg-GUE0SwMa2fkUAVkO .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-GUE0SwMa2fkUAVkO .arrowheadPath{fill:#333333;}#mermaid-svg-GUE0SwMa2fkUAVkO .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-GUE0SwMa2fkUAVkO .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-GUE0SwMa2fkUAVkO .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-GUE0SwMa2fkUAVkO .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-GUE0SwMa2fkUAVkO .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-GUE0SwMa2fkUAVkO .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-GUE0SwMa2fkUAVkO .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-GUE0SwMa2fkUAVkO .cluster text{fill:#333;}#mermaid-svg-GUE0SwMa2fkUAVkO .cluster span{color:#333;}#mermaid-svg-GUE0SwMa2fkUAVkO div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-GUE0SwMa2fkUAVkO .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-GUE0SwMa2fkUAVkO rect.text{fill:none;stroke-width:0;}#mermaid-svg-GUE0SwMa2fkUAVkO .icon-shape,#mermaid-svg-GUE0SwMa2fkUAVkO .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-GUE0SwMa2fkUAVkO .icon-shape p,#mermaid-svg-GUE0SwMa2fkUAVkO .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-GUE0SwMa2fkUAVkO .icon-shape .label rect,#mermaid-svg-GUE0SwMa2fkUAVkO .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-GUE0SwMa2fkUAVkO .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-GUE0SwMa2fkUAVkO .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-GUE0SwMa2fkUAVkO :root{–mermaid-font-family:\”trebuchet ms\”,verdana,arial,sans-serif;}

消费者

Broker 集群

事务生产者

数据 Broker

1.beginTransaction

2.FindCoordinator

3.InitProducerId

4.持久化

5.send 消息

6.Produce 请求

7.AddPartitionsToTxn

8.持久化分区信息

9.commitTransaction

10.EndTxn

11.WriteTxnMarker

12.持久化最终状态

13.过滤已提交消息

业务应用

TransactionManager

Transaction Coordinator事务协调者

__transaction_state事务日志 Topic

Partition Leader 0

Partition Leader 1

.txnindex事务索引文件

Consumerisolation.level=read_committed

4. 事务核心机制

4.1 幂等性基础：事务的基石

事务机制强依赖幂等性，配置 enable.idempotence=true 会自动生效以下约束：

#mermaid-svg-FTsIBSsplYoYCHks{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-FTsIBSsplYoYCHks .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-FTsIBSsplYoYCHks .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-FTsIBSsplYoYCHks .error-icon{fill:#552222;}#mermaid-svg-FTsIBSsplYoYCHks .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-FTsIBSsplYoYCHks .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-FTsIBSsplYoYCHks .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-FTsIBSsplYoYCHks .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-FTsIBSsplYoYCHks .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-FTsIBSsplYoYCHks .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-FTsIBSsplYoYCHks .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-FTsIBSsplYoYCHks .marker{fill:#333333;stroke:#333333;}#mermaid-svg-FTsIBSsplYoYCHks .marker.cross{stroke:#333333;}#mermaid-svg-FTsIBSsplYoYCHks svg{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-FTsIBSsplYoYCHks p{margin:0;}#mermaid-svg-FTsIBSsplYoYCHks .label{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;color:#333;}#mermaid-svg-FTsIBSsplYoYCHks .cluster-label text{fill:#333;}#mermaid-svg-FTsIBSsplYoYCHks .cluster-label span{color:#333;}#mermaid-svg-FTsIBSsplYoYCHks .cluster-label span p{background-color:transparent;}#mermaid-svg-FTsIBSsplYoYCHks .label text,#mermaid-svg-FTsIBSsplYoYCHks span{fill:#333;color:#333;}#mermaid-svg-FTsIBSsplYoYCHks .node rect,#mermaid-svg-FTsIBSsplYoYCHks .node circle,#mermaid-svg-FTsIBSsplYoYCHks .node ellipse,#mermaid-svg-FTsIBSsplYoYCHks .node polygon,#mermaid-svg-FTsIBSsplYoYCHks .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-FTsIBSsplYoYCHks .rough-node .label text,#mermaid-svg-FTsIBSsplYoYCHks .node .label text,#mermaid-svg-FTsIBSsplYoYCHks .image-shape .label,#mermaid-svg-FTsIBSsplYoYCHks .icon-shape .label{text-anchor:middle;}#mermaid-svg-FTsIBSsplYoYCHks .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-FTsIBSsplYoYCHks .rough-node .label,#mermaid-svg-FTsIBSsplYoYCHks .node .label,#mermaid-svg-FTsIBSsplYoYCHks .image-shape .label,#mermaid-svg-FTsIBSsplYoYCHks .icon-shape .label{text-align:center;}#mermaid-svg-FTsIBSsplYoYCHks .node.clickable{cursor:pointer;}#mermaid-svg-FTsIBSsplYoYCHks .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-FTsIBSsplYoYCHks .arrowheadPath{fill:#333333;}#mermaid-svg-FTsIBSsplYoYCHks .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-FTsIBSsplYoYCHks .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-FTsIBSsplYoYCHks .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-FTsIBSsplYoYCHks .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-FTsIBSsplYoYCHks .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-FTsIBSsplYoYCHks .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-FTsIBSsplYoYCHks .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-FTsIBSsplYoYCHks .cluster text{fill:#333;}#mermaid-svg-FTsIBSsplYoYCHks .cluster span{color:#333;}#mermaid-svg-FTsIBSsplYoYCHks div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-FTsIBSsplYoYCHks .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-FTsIBSsplYoYCHks rect.text{fill:none;stroke-width:0;}#mermaid-svg-FTsIBSsplYoYCHks .icon-shape,#mermaid-svg-FTsIBSsplYoYCHks .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-FTsIBSsplYoYCHks .icon-shape p,#mermaid-svg-FTsIBSsplYoYCHks .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-FTsIBSsplYoYCHks .icon-shape .label rect,#mermaid-svg-FTsIBSsplYoYCHks .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-FTsIBSsplYoYCHks .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-FTsIBSsplYoYCHks .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-FTsIBSsplYoYCHks :root{–mermaid-font-family:\”trebuchet ms\”,verdana,arial,sans-serif;}

幂等性保障

seq_num > last_seq

seq_num <= last_seq

Sequence Number

序号检查

接受并更新 last_seq

拒绝OutOfOrderSequenceException

幂等性约束：Broker 为每个 <PID, TopicPartition> 维护当前最大 Sequence Number，消息序号必须严格递增，否则视为重复消息拒绝写入。

4.2 僵尸实例防护：Epoch 机制

为防止故障恢复后出现多个活跃 Producer 实例（僵尸），引入 Epoch 机制：

Transaction Coordinator

Producer实例2

(Epoch=2)

Producer实例1

(Epoch=1)

Transaction Coordinator

Producer实例2

(Epoch=2)

Producer实例1

(Epoch=1)

#mermaid-svg-Hn8DqwewPc4JYu0m{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-Hn8DqwewPc4JYu0m .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-Hn8DqwewPc4JYu0m .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-Hn8DqwewPc4JYu0m .error-icon{fill:#552222;}#mermaid-svg-Hn8DqwewPc4JYu0m .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-Hn8DqwewPc4JYu0m .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-Hn8DqwewPc4JYu0m .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-Hn8DqwewPc4JYu0m .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-Hn8DqwewPc4JYu0m .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-Hn8DqwewPc4JYu0m .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-Hn8DqwewPc4JYu0m .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-Hn8DqwewPc4JYu0m .marker{fill:#333333;stroke:#333333;}#mermaid-svg-Hn8DqwewPc4JYu0m .marker.cross{stroke:#333333;}#mermaid-svg-Hn8DqwewPc4JYu0m svg{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-Hn8DqwewPc4JYu0m p{margin:0;}#mermaid-svg-Hn8DqwewPc4JYu0m .actor{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-Hn8DqwewPc4JYu0m text.actor>tspan{fill:black;stroke:none;}#mermaid-svg-Hn8DqwewPc4JYu0m .actor-line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);}#mermaid-svg-Hn8DqwewPc4JYu0m .innerArc{stroke-width:1.5;stroke-dasharray:none;}#mermaid-svg-Hn8DqwewPc4JYu0m .messageLine0{stroke-width:1.5;stroke-dasharray:none;stroke:#333;}#mermaid-svg-Hn8DqwewPc4JYu0m .messageLine1{stroke-width:1.5;stroke-dasharray:2,2;stroke:#333;}#mermaid-svg-Hn8DqwewPc4JYu0m #arrowhead path{fill:#333;stroke:#333;}#mermaid-svg-Hn8DqwewPc4JYu0m .sequenceNumber{fill:white;}#mermaid-svg-Hn8DqwewPc4JYu0m #sequencenumber{fill:#333;}#mermaid-svg-Hn8DqwewPc4JYu0m #crosshead path{fill:#333;stroke:#333;}#mermaid-svg-Hn8DqwewPc4JYu0m .messageText{fill:#333;stroke:none;}#mermaid-svg-Hn8DqwewPc4JYu0m .labelBox{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-Hn8DqwewPc4JYu0m .labelText,#mermaid-svg-Hn8DqwewPc4JYu0m .labelText>tspan{fill:black;stroke:none;}#mermaid-svg-Hn8DqwewPc4JYu0m .loopText,#mermaid-svg-Hn8DqwewPc4JYu0m .loopText>tspan{fill:black;stroke:none;}#mermaid-svg-Hn8DqwewPc4JYu0m .loopLine{stroke-width:2px;stroke-dasharray:2,2;stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);}#mermaid-svg-Hn8DqwewPc4JYu0m .note{stroke:#aaaa33;fill:#fff5ad;}#mermaid-svg-Hn8DqwewPc4JYu0m .noteText,#mermaid-svg-Hn8DqwewPc4JYu0m .noteText>tspan{fill:black;stroke:none;}#mermaid-svg-Hn8DqwewPc4JYu0m .activation0{fill:#f4f4f4;stroke:#666;}#mermaid-svg-Hn8DqwewPc4JYu0m .activation1{fill:#f4f4f4;stroke:#666;}#mermaid-svg-Hn8DqwewPc4JYu0m .activation2{fill:#f4f4f4;stroke:#666;}#mermaid-svg-Hn8DqwewPc4JYu0m .actorPopupMenu{position:absolute;}#mermaid-svg-Hn8DqwewPc4JYu0m .actorPopupMenuPanel{position:absolute;fill:#ECECFF;box-shadow:0px 8px 16px 0px rgba(0,0,0,0.2);filter:drop-shadow(3px 5px 2px rgb(0 0 0 / 0.4));}#mermaid-svg-Hn8DqwewPc4JYu0m .actor-man line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-Hn8DqwewPc4JYu0m .actor-man circle,#mermaid-svg-Hn8DqwewPc4JYu0m line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;stroke-width:2px;}#mermaid-svg-Hn8DqwewPc4JYu0m :root{–mermaid-font-family:\”trebuchet ms\”,verdana,arial,sans-serif;}

实例1 正常运行中

实例1 故障

连接断开

initTransactions(Transactional.Id)

Epoch: 1 → 2

返回新 PID, Epoch=2

僵尸恢复，发送请求(Epoch=1)

拒绝请求

FENCED

核心机制：

• 每个 Transactional.Id 对应的 Epoch 单调递增
• Coordinator 拒绝 Epoch 过小的请求
• 旧的 Producer 实例被“围栏”（Fenced），无法继续操作

4.3 控制消息：事务边界的标记

Control Message 是写入数据日志的特殊消息，对消费者不可见，用于标记事务状态：

#mermaid-svg-NU6mV92OpumBZLPM{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-NU6mV92OpumBZLPM .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-NU6mV92OpumBZLPM .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-NU6mV92OpumBZLPM .error-icon{fill:#552222;}#mermaid-svg-NU6mV92OpumBZLPM .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-NU6mV92OpumBZLPM .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-NU6mV92OpumBZLPM .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-NU6mV92OpumBZLPM .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-NU6mV92OpumBZLPM .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-NU6mV92OpumBZLPM .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-NU6mV92OpumBZLPM .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-NU6mV92OpumBZLPM .marker{fill:#333333;stroke:#333333;}#mermaid-svg-NU6mV92OpumBZLPM .marker.cross{stroke:#333333;}#mermaid-svg-NU6mV92OpumBZLPM svg{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-NU6mV92OpumBZLPM p{margin:0;}#mermaid-svg-NU6mV92OpumBZLPM .label{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;color:#333;}#mermaid-svg-NU6mV92OpumBZLPM .cluster-label text{fill:#333;}#mermaid-svg-NU6mV92OpumBZLPM .cluster-label span{color:#333;}#mermaid-svg-NU6mV92OpumBZLPM .cluster-label span p{background-color:transparent;}#mermaid-svg-NU6mV92OpumBZLPM .label text,#mermaid-svg-NU6mV92OpumBZLPM span{fill:#333;color:#333;}#mermaid-svg-NU6mV92OpumBZLPM .node rect,#mermaid-svg-NU6mV92OpumBZLPM .node circle,#mermaid-svg-NU6mV92OpumBZLPM .node ellipse,#mermaid-svg-NU6mV92OpumBZLPM .node polygon,#mermaid-svg-NU6mV92OpumBZLPM .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-NU6mV92OpumBZLPM .rough-node .label text,#mermaid-svg-NU6mV92OpumBZLPM .node .label text,#mermaid-svg-NU6mV92OpumBZLPM .image-shape .label,#mermaid-svg-NU6mV92OpumBZLPM .icon-shape .label{text-anchor:middle;}#mermaid-svg-NU6mV92OpumBZLPM .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-NU6mV92OpumBZLPM .rough-node .label,#mermaid-svg-NU6mV92OpumBZLPM .node .label,#mermaid-svg-NU6mV92OpumBZLPM .image-shape .label,#mermaid-svg-NU6mV92OpumBZLPM .icon-shape .label{text-align:center;}#mermaid-svg-NU6mV92OpumBZLPM .node.clickable{cursor:pointer;}#mermaid-svg-NU6mV92OpumBZLPM .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-NU6mV92OpumBZLPM .arrowheadPath{fill:#333333;}#mermaid-svg-NU6mV92OpumBZLPM .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-NU6mV92OpumBZLPM .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-NU6mV92OpumBZLPM .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-NU6mV92OpumBZLPM .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-NU6mV92OpumBZLPM .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-NU6mV92OpumBZLPM .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-NU6mV92OpumBZLPM .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-NU6mV92OpumBZLPM .cluster text{fill:#333;}#mermaid-svg-NU6mV92OpumBZLPM .cluster span{color:#333;}#mermaid-svg-NU6mV92OpumBZLPM div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-NU6mV92OpumBZLPM .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-NU6mV92OpumBZLPM rect.text{fill:none;stroke-width:0;}#mermaid-svg-NU6mV92OpumBZLPM .icon-shape,#mermaid-svg-NU6mV92OpumBZLPM .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-NU6mV92OpumBZLPM .icon-shape p,#mermaid-svg-NU6mV92OpumBZLPM .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-NU6mV92OpumBZLPM .icon-shape .label rect,#mermaid-svg-NU6mV92OpumBZLPM .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-NU6mV92OpumBZLPM .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-NU6mV92OpumBZLPM .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-NU6mV92OpumBZLPM :root{–mermaid-font-family:\”trebuchet ms\”,verdana,arial,sans-serif;}

消费过滤

Partition日志

read_committed

read_uncommitted

消息1TxA

消息2TxA

COMMITControl Msg

消息3TxB

停止消费

继续消费

5. 事务完整流程详解

5.1 阶段一：事务初始化

__transaction_state

Transaction Coordinator

Producer

__transaction_state

Transaction Coordinator

Producer

#mermaid-svg-lso0uGDUENK1qkLn{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-lso0uGDUENK1qkLn .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-lso0uGDUENK1qkLn .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-lso0uGDUENK1qkLn .error-icon{fill:#552222;}#mermaid-svg-lso0uGDUENK1qkLn .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-lso0uGDUENK1qkLn .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-lso0uGDUENK1qkLn .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-lso0uGDUENK1qkLn .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-lso0uGDUENK1qkLn .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-lso0uGDUENK1qkLn .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-lso0uGDUENK1qkLn .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-lso0uGDUENK1qkLn .marker{fill:#333333;stroke:#333333;}#mermaid-svg-lso0uGDUENK1qkLn .marker.cross{stroke:#333333;}#mermaid-svg-lso0uGDUENK1qkLn svg{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-lso0uGDUENK1qkLn p{margin:0;}#mermaid-svg-lso0uGDUENK1qkLn .actor{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-lso0uGDUENK1qkLn text.actor>tspan{fill:black;stroke:none;}#mermaid-svg-lso0uGDUENK1qkLn .actor-line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);}#mermaid-svg-lso0uGDUENK1qkLn .innerArc{stroke-width:1.5;stroke-dasharray:none;}#mermaid-svg-lso0uGDUENK1qkLn .messageLine0{stroke-width:1.5;stroke-dasharray:none;stroke:#333;}#mermaid-svg-lso0uGDUENK1qkLn .messageLine1{stroke-width:1.5;stroke-dasharray:2,2;stroke:#333;}#mermaid-svg-lso0uGDUENK1qkLn #arrowhead path{fill:#333;stroke:#333;}#mermaid-svg-lso0uGDUENK1qkLn .sequenceNumber{fill:white;}#mermaid-svg-lso0uGDUENK1qkLn #sequencenumber{fill:#333;}#mermaid-svg-lso0uGDUENK1qkLn #crosshead path{fill:#333;stroke:#333;}#mermaid-svg-lso0uGDUENK1qkLn .messageText{fill:#333;stroke:none;}#mermaid-svg-lso0uGDUENK1qkLn .labelBox{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-lso0uGDUENK1qkLn .labelText,#mermaid-svg-lso0uGDUENK1qkLn .labelText>tspan{fill:black;stroke:none;}#mermaid-svg-lso0uGDUENK1qkLn .loopText,#mermaid-svg-lso0uGDUENK1qkLn .loopText>tspan{fill:black;stroke:none;}#mermaid-svg-lso0uGDUENK1qkLn .loopLine{stroke-width:2px;stroke-dasharray:2,2;stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);}#mermaid-svg-lso0uGDUENK1qkLn .note{stroke:#aaaa33;fill:#fff5ad;}#mermaid-svg-lso0uGDUENK1qkLn .noteText,#mermaid-svg-lso0uGDUENK1qkLn .noteText>tspan{fill:black;stroke:none;}#mermaid-svg-lso0uGDUENK1qkLn .activation0{fill:#f4f4f4;stroke:#666;}#mermaid-svg-lso0uGDUENK1qkLn .activation1{fill:#f4f4f4;stroke:#666;}#mermaid-svg-lso0uGDUENK1qkLn .activation2{fill:#f4f4f4;stroke:#666;}#mermaid-svg-lso0uGDUENK1qkLn .actorPopupMenu{position:absolute;}#mermaid-svg-lso0uGDUENK1qkLn .actorPopupMenuPanel{position:absolute;fill:#ECECFF;box-shadow:0px 8px 16px 0px rgba(0,0,0,0.2);filter:drop-shadow(3px 5px 2px rgb(0 0 0 / 0.4));}#mermaid-svg-lso0uGDUENK1qkLn .actor-man line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-lso0uGDUENK1qkLn .actor-man circle,#mermaid-svg-lso0uGDUENK1qkLn line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;stroke-width:2px;}#mermaid-svg-lso0uGDUENK1qkLn :root{–mermaid-font-family:\”trebuchet ms\”,verdana,arial,sans-serif;}

配置 transactional.id

alt

[首次初始化]

[已有事务未完成]

1. FindCoordinator(transactional.id)

返回 Coordinator 地址

2. InitProducerId(transactional.id)

生成新的 PID, Epoch=1

回滚未完成事务

加载已有 PID, Epoch+1

3. 写入 PID 分配记录

4. 返回 PID, Epoch

关键点：

• beginTransaction() 仅是客户端状态标记，不产生 RPC
• 实际事务开启发生在第一条消息发送时

5.2 阶段二：消息发送与分区注册

Data Broker (分区 Leader)

__transaction_state

Transaction Coordinator

Producer

Data Broker (分区 Leader)

__transaction_state

Transaction Coordinator

Producer

#mermaid-svg-kaOhZuZedPqCWvQV{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-kaOhZuZedPqCWvQV .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-kaOhZuZedPqCWvQV .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-kaOhZuZedPqCWvQV .error-icon{fill:#552222;}#mermaid-svg-kaOhZuZedPqCWvQV .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-kaOhZuZedPqCWvQV .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-kaOhZuZedPqCWvQV .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-kaOhZuZedPqCWvQV .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-kaOhZuZedPqCWvQV .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-kaOhZuZedPqCWvQV .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-kaOhZuZedPqCWvQV .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-kaOhZuZedPqCWvQV .marker{fill:#333333;stroke:#333333;}#mermaid-svg-kaOhZuZedPqCWvQV .marker.cross{stroke:#333333;}#mermaid-svg-kaOhZuZedPqCWvQV svg{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-kaOhZuZedPqCWvQV p{margin:0;}#mermaid-svg-kaOhZuZedPqCWvQV .actor{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-kaOhZuZedPqCWvQV text.actor>tspan{fill:black;stroke:none;}#mermaid-svg-kaOhZuZedPqCWvQV .actor-line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);}#mermaid-svg-kaOhZuZedPqCWvQV .innerArc{stroke-width:1.5;stroke-dasharray:none;}#mermaid-svg-kaOhZuZedPqCWvQV .messageLine0{stroke-width:1.5;stroke-dasharray:none;stroke:#333;}#mermaid-svg-kaOhZuZedPqCWvQV .messageLine1{stroke-width:1.5;stroke-dasharray:2,2;stroke:#333;}#mermaid-svg-kaOhZuZedPqCWvQV #arrowhead path{fill:#333;stroke:#333;}#mermaid-svg-kaOhZuZedPqCWvQV .sequenceNumber{fill:white;}#mermaid-svg-kaOhZuZedPqCWvQV #sequencenumber{fill:#333;}#mermaid-svg-kaOhZuZedPqCWvQV #crosshead path{fill:#333;stroke:#333;}#mermaid-svg-kaOhZuZedPqCWvQV .messageText{fill:#333;stroke:none;}#mermaid-svg-kaOhZuZedPqCWvQV .labelBox{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-kaOhZuZedPqCWvQV .labelText,#mermaid-svg-kaOhZuZedPqCWvQV .labelText>tspan{fill:black;stroke:none;}#mermaid-svg-kaOhZuZedPqCWvQV .loopText,#mermaid-svg-kaOhZuZedPqCWvQV .loopText>tspan{fill:black;stroke:none;}#mermaid-svg-kaOhZuZedPqCWvQV .loopLine{stroke-width:2px;stroke-dasharray:2,2;stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);}#mermaid-svg-kaOhZuZedPqCWvQV .note{stroke:#aaaa33;fill:#fff5ad;}#mermaid-svg-kaOhZuZedPqCWvQV .noteText,#mermaid-svg-kaOhZuZedPqCWvQV .noteText>tspan{fill:black;stroke:none;}#mermaid-svg-kaOhZuZedPqCWvQV .activation0{fill:#f4f4f4;stroke:#666;}#mermaid-svg-kaOhZuZedPqCWvQV .activation1{fill:#f4f4f4;stroke:#666;}#mermaid-svg-kaOhZuZedPqCWvQV .activation2{fill:#f4f4f4;stroke:#666;}#mermaid-svg-kaOhZuZedPqCWvQV .actorPopupMenu{position:absolute;}#mermaid-svg-kaOhZuZedPqCWvQV .actorPopupMenuPanel{position:absolute;fill:#ECECFF;box-shadow:0px 8px 16px 0px rgba(0,0,0,0.2);filter:drop-shadow(3px 5px 2px rgb(0 0 0 / 0.4));}#mermaid-svg-kaOhZuZedPqCWvQV .actor-man line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-kaOhZuZedPqCWvQV .actor-man circle,#mermaid-svg-kaOhZuZedPqCWvQV line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;stroke-width:2px;}#mermaid-svg-kaOhZuZedPqCWvQV :root{–mermaid-font-family:\”trebuchet ms\”,verdana,arial,sans-serif;}

beginTransaction()

loop

[每条消息]

1. Produce 请求 (第一条消息)

检测到新分区

响应

2. AddPartitionsToTxn

(partitions=[topic1-0, topic2-1])

3. 写入分区信息

状态: Ongoing

4. 确认

5. Produce 请求

(PID, Epoch, SeqNo, 消息体)

持久化消息

(消费者不可见)

5.3 阶段三：事务提交（两阶段提交）

Data Broker

__transaction_state

Transaction Coordinator

Producer

Data Broker

__transaction_state

Transaction Coordinator

Producer

#mermaid-svg-ISVHZVBg7rpEwIxY{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-ISVHZVBg7rpEwIxY .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-ISVHZVBg7rpEwIxY .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-ISVHZVBg7rpEwIxY .error-icon{fill:#552222;}#mermaid-svg-ISVHZVBg7rpEwIxY .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-ISVHZVBg7rpEwIxY .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-ISVHZVBg7rpEwIxY .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-ISVHZVBg7rpEwIxY .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-ISVHZVBg7rpEwIxY .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-ISVHZVBg7rpEwIxY .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-ISVHZVBg7rpEwIxY .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-ISVHZVBg7rpEwIxY .marker{fill:#333333;stroke:#333333;}#mermaid-svg-ISVHZVBg7rpEwIxY .marker.cross{stroke:#333333;}#mermaid-svg-ISVHZVBg7rpEwIxY svg{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-ISVHZVBg7rpEwIxY p{margin:0;}#mermaid-svg-ISVHZVBg7rpEwIxY .actor{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-ISVHZVBg7rpEwIxY text.actor>tspan{fill:black;stroke:none;}#mermaid-svg-ISVHZVBg7rpEwIxY .actor-line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);}#mermaid-svg-ISVHZVBg7rpEwIxY .innerArc{stroke-width:1.5;stroke-dasharray:none;}#mermaid-svg-ISVHZVBg7rpEwIxY .messageLine0{stroke-width:1.5;stroke-dasharray:none;stroke:#333;}#mermaid-svg-ISVHZVBg7rpEwIxY .messageLine1{stroke-width:1.5;stroke-dasharray:2,2;stroke:#333;}#mermaid-svg-ISVHZVBg7rpEwIxY #arrowhead path{fill:#333;stroke:#333;}#mermaid-svg-ISVHZVBg7rpEwIxY .sequenceNumber{fill:white;}#mermaid-svg-ISVHZVBg7rpEwIxY #sequencenumber{fill:#333;}#mermaid-svg-ISVHZVBg7rpEwIxY #crosshead path{fill:#333;stroke:#333;}#mermaid-svg-ISVHZVBg7rpEwIxY .messageText{fill:#333;stroke:none;}#mermaid-svg-ISVHZVBg7rpEwIxY .labelBox{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-ISVHZVBg7rpEwIxY .labelText,#mermaid-svg-ISVHZVBg7rpEwIxY .labelText>tspan{fill:black;stroke:none;}#mermaid-svg-ISVHZVBg7rpEwIxY .loopText,#mermaid-svg-ISVHZVBg7rpEwIxY .loopText>tspan{fill:black;stroke:none;}#mermaid-svg-ISVHZVBg7rpEwIxY .loopLine{stroke-width:2px;stroke-dasharray:2,2;stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);}#mermaid-svg-ISVHZVBg7rpEwIxY .note{stroke:#aaaa33;fill:#fff5ad;}#mermaid-svg-ISVHZVBg7rpEwIxY .noteText,#mermaid-svg-ISVHZVBg7rpEwIxY .noteText>tspan{fill:black;stroke:none;}#mermaid-svg-ISVHZVBg7rpEwIxY .activation0{fill:#f4f4f4;stroke:#666;}#mermaid-svg-ISVHZVBg7rpEwIxY .activation1{fill:#f4f4f4;stroke:#666;}#mermaid-svg-ISVHZVBg7rpEwIxY .activation2{fill:#f4f4f4;stroke:#666;}#mermaid-svg-ISVHZVBg7rpEwIxY .actorPopupMenu{position:absolute;}#mermaid-svg-ISVHZVBg7rpEwIxY .actorPopupMenuPanel{position:absolute;fill:#ECECFF;box-shadow:0px 8px 16px 0px rgba(0,0,0,0.2);filter:drop-shadow(3px 5px 2px rgb(0 0 0 / 0.4));}#mermaid-svg-ISVHZVBg7rpEwIxY .actor-man line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-ISVHZVBg7rpEwIxY .actor-man circle,#mermaid-svg-ISVHZVBg7rpEwIxY line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;stroke-width:2px;}#mermaid-svg-ISVHZVBg7rpEwIxY :root{–mermaid-font-family:\”trebuchet ms\”,verdana,arial,sans-serif;}

第一阶段: Prepare

第二阶段: 写 Marker

loop

[每个参与分区]

第三阶段: 完成

1. commitTransaction()

2. 写入 PREPARE_COMMIT 状态

3. 返回响应

4. WriteTxnMarker(COMMIT)

写入 COMMIT Control Message

确认

5. 写入 COMMITTED 状态

完整的状态转换路径：

#mermaid-svg-dBvPpWU9UOl2gxXt{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-dBvPpWU9UOl2gxXt .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-dBvPpWU9UOl2gxXt .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-dBvPpWU9UOl2gxXt .error-icon{fill:#552222;}#mermaid-svg-dBvPpWU9UOl2gxXt .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-dBvPpWU9UOl2gxXt .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-dBvPpWU9UOl2gxXt .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-dBvPpWU9UOl2gxXt .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-dBvPpWU9UOl2gxXt .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-dBvPpWU9UOl2gxXt .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-dBvPpWU9UOl2gxXt .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-dBvPpWU9UOl2gxXt .marker{fill:#333333;stroke:#333333;}#mermaid-svg-dBvPpWU9UOl2gxXt .marker.cross{stroke:#333333;}#mermaid-svg-dBvPpWU9UOl2gxXt svg{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-dBvPpWU9UOl2gxXt p{margin:0;}#mermaid-svg-dBvPpWU9UOl2gxXt defs #statediagram-barbEnd{fill:#333333;stroke:#333333;}#mermaid-svg-dBvPpWU9UOl2gxXt g.stateGroup text{fill:#9370DB;stroke:none;font-size:10px;}#mermaid-svg-dBvPpWU9UOl2gxXt g.stateGroup text{fill:#333;stroke:none;font-size:10px;}#mermaid-svg-dBvPpWU9UOl2gxXt g.stateGroup .state-title{font-weight:bolder;fill:#131300;}#mermaid-svg-dBvPpWU9UOl2gxXt g.stateGroup rect{fill:#ECECFF;stroke:#9370DB;}#mermaid-svg-dBvPpWU9UOl2gxXt g.stateGroup line{stroke:#333333;stroke-width:1;}#mermaid-svg-dBvPpWU9UOl2gxXt .transition{stroke:#333333;stroke-width:1;fill:none;}#mermaid-svg-dBvPpWU9UOl2gxXt .stateGroup .composit{fill:white;border-bottom:1px;}#mermaid-svg-dBvPpWU9UOl2gxXt .stateGroup .alt-composit{fill:#e0e0e0;border-bottom:1px;}#mermaid-svg-dBvPpWU9UOl2gxXt .state-note{stroke:#aaaa33;fill:#fff5ad;}#mermaid-svg-dBvPpWU9UOl2gxXt .state-note text{fill:black;stroke:none;font-size:10px;}#mermaid-svg-dBvPpWU9UOl2gxXt .stateLabel .box{stroke:none;stroke-width:0;fill:#ECECFF;opacity:0.5;}#mermaid-svg-dBvPpWU9UOl2gxXt .edgeLabel .label rect{fill:#ECECFF;opacity:0.5;}#mermaid-svg-dBvPpWU9UOl2gxXt .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-dBvPpWU9UOl2gxXt .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-dBvPpWU9UOl2gxXt .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-dBvPpWU9UOl2gxXt .edgeLabel .label text{fill:#333;}#mermaid-svg-dBvPpWU9UOl2gxXt .label div .edgeLabel{color:#333;}#mermaid-svg-dBvPpWU9UOl2gxXt .stateLabel text{fill:#131300;font-size:10px;font-weight:bold;}#mermaid-svg-dBvPpWU9UOl2gxXt .node circle.state-start{fill:#333333;stroke:#333333;}#mermaid-svg-dBvPpWU9UOl2gxXt .node .fork-join{fill:#333333;stroke:#333333;}#mermaid-svg-dBvPpWU9UOl2gxXt .node circle.state-end{fill:#9370DB;stroke:white;stroke-width:1.5;}#mermaid-svg-dBvPpWU9UOl2gxXt .end-state-inner{fill:white;stroke-width:1.5;}#mermaid-svg-dBvPpWU9UOl2gxXt .node rect{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-dBvPpWU9UOl2gxXt .node polygon{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-dBvPpWU9UOl2gxXt #statediagram-barbEnd{fill:#333333;}#mermaid-svg-dBvPpWU9UOl2gxXt .statediagram-cluster rect{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-dBvPpWU9UOl2gxXt .cluster-label,#mermaid-svg-dBvPpWU9UOl2gxXt .nodeLabel{color:#131300;}#mermaid-svg-dBvPpWU9UOl2gxXt .statediagram-cluster rect.outer{rx:5px;ry:5px;}#mermaid-svg-dBvPpWU9UOl2gxXt .statediagram-state .divider{stroke:#9370DB;}#mermaid-svg-dBvPpWU9UOl2gxXt .statediagram-state .title-state{rx:5px;ry:5px;}#mermaid-svg-dBvPpWU9UOl2gxXt .statediagram-cluster.statediagram-cluster .inner{fill:white;}#mermaid-svg-dBvPpWU9UOl2gxXt .statediagram-cluster.statediagram-cluster-alt .inner{fill:#f0f0f0;}#mermaid-svg-dBvPpWU9UOl2gxXt .statediagram-cluster .inner{rx:0;ry:0;}#mermaid-svg-dBvPpWU9UOl2gxXt .statediagram-state rect.basic{rx:5px;ry:5px;}#mermaid-svg-dBvPpWU9UOl2gxXt .statediagram-state rect.divider{stroke-dasharray:10,10;fill:#f0f0f0;}#mermaid-svg-dBvPpWU9UOl2gxXt .note-edge{stroke-dasharray:5;}#mermaid-svg-dBvPpWU9UOl2gxXt .statediagram-note rect{fill:#fff5ad;stroke:#aaaa33;stroke-width:1px;rx:0;ry:0;}#mermaid-svg-dBvPpWU9UOl2gxXt .statediagram-note rect{fill:#fff5ad;stroke:#aaaa33;stroke-width:1px;rx:0;ry:0;}#mermaid-svg-dBvPpWU9UOl2gxXt .statediagram-note text{fill:black;}#mermaid-svg-dBvPpWU9UOl2gxXt .statediagram-note .nodeLabel{color:black;}#mermaid-svg-dBvPpWU9UOl2gxXt .statediagram .edgeLabel{color:red;}#mermaid-svg-dBvPpWU9UOl2gxXt #dependencyStart,#mermaid-svg-dBvPpWU9UOl2gxXt #dependencyEnd{fill:#333333;stroke:#333333;stroke-width:1;}#mermaid-svg-dBvPpWU9UOl2gxXt .statediagramTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-dBvPpWU9UOl2gxXt :root{–mermaid-font-family:\”trebuchet ms\”,verdana,arial,sans-serif;}

initTransactions

第一条消息发送

commitTransaction

WriteTxnMarker 完成

事务结束

abortTransaction

WriteTxnMarker 完成

事务回滚

Empty

Ongoing

PrepareCommit

CompleteCommit

PrepareAbort

CompleteAbort

5.4 消费者隔离：消息可见性控制

消费者通过 isolation.level 配置控制可见性：

#mermaid-svg-3MmeaNEDLffxkYu0{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-3MmeaNEDLffxkYu0 .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-3MmeaNEDLffxkYu0 .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-3MmeaNEDLffxkYu0 .error-icon{fill:#552222;}#mermaid-svg-3MmeaNEDLffxkYu0 .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-3MmeaNEDLffxkYu0 .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-3MmeaNEDLffxkYu0 .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-3MmeaNEDLffxkYu0 .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-3MmeaNEDLffxkYu0 .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-3MmeaNEDLffxkYu0 .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-3MmeaNEDLffxkYu0 .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-3MmeaNEDLffxkYu0 .marker{fill:#333333;stroke:#333333;}#mermaid-svg-3MmeaNEDLffxkYu0 .marker.cross{stroke:#333333;}#mermaid-svg-3MmeaNEDLffxkYu0 svg{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-3MmeaNEDLffxkYu0 p{margin:0;}#mermaid-svg-3MmeaNEDLffxkYu0 .label{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;color:#333;}#mermaid-svg-3MmeaNEDLffxkYu0 .cluster-label text{fill:#333;}#mermaid-svg-3MmeaNEDLffxkYu0 .cluster-label span{color:#333;}#mermaid-svg-3MmeaNEDLffxkYu0 .cluster-label span p{background-color:transparent;}#mermaid-svg-3MmeaNEDLffxkYu0 .label text,#mermaid-svg-3MmeaNEDLffxkYu0 span{fill:#333;color:#333;}#mermaid-svg-3MmeaNEDLffxkYu0 .node rect,#mermaid-svg-3MmeaNEDLffxkYu0 .node circle,#mermaid-svg-3MmeaNEDLffxkYu0 .node ellipse,#mermaid-svg-3MmeaNEDLffxkYu0 .node polygon,#mermaid-svg-3MmeaNEDLffxkYu0 .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-3MmeaNEDLffxkYu0 .rough-node .label text,#mermaid-svg-3MmeaNEDLffxkYu0 .node .label text,#mermaid-svg-3MmeaNEDLffxkYu0 .image-shape .label,#mermaid-svg-3MmeaNEDLffxkYu0 .icon-shape .label{text-anchor:middle;}#mermaid-svg-3MmeaNEDLffxkYu0 .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-3MmeaNEDLffxkYu0 .rough-node .label,#mermaid-svg-3MmeaNEDLffxkYu0 .node .label,#mermaid-svg-3MmeaNEDLffxkYu0 .image-shape .label,#mermaid-svg-3MmeaNEDLffxkYu0 .icon-shape .label{text-align:center;}#mermaid-svg-3MmeaNEDLffxkYu0 .node.clickable{cursor:pointer;}#mermaid-svg-3MmeaNEDLffxkYu0 .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-3MmeaNEDLffxkYu0 .arrowheadPath{fill:#333333;}#mermaid-svg-3MmeaNEDLffxkYu0 .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-3MmeaNEDLffxkYu0 .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-3MmeaNEDLffxkYu0 .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-3MmeaNEDLffxkYu0 .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-3MmeaNEDLffxkYu0 .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-3MmeaNEDLffxkYu0 .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-3MmeaNEDLffxkYu0 .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-3MmeaNEDLffxkYu0 .cluster text{fill:#333;}#mermaid-svg-3MmeaNEDLffxkYu0 .cluster span{color:#333;}#mermaid-svg-3MmeaNEDLffxkYu0 div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-3MmeaNEDLffxkYu0 .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-3MmeaNEDLffxkYu0 rect.text{fill:none;stroke-width:0;}#mermaid-svg-3MmeaNEDLffxkYu0 .icon-shape,#mermaid-svg-3MmeaNEDLffxkYu0 .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-3MmeaNEDLffxkYu0 .icon-shape p,#mermaid-svg-3MmeaNEDLffxkYu0 .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-3MmeaNEDLffxkYu0 .icon-shape .label rect,#mermaid-svg-3MmeaNEDLffxkYu0 .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-3MmeaNEDLffxkYu0 .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-3MmeaNEDLffxkYu0 .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-3MmeaNEDLffxkYu0 :root{–mermaid-font-family:\”trebuchet ms\”,verdana,arial,sans-serif;}

read_committed

read_uncommitted

消费者配置

read_uncommitted

read_committed

isolation.level

可见所有消息包括未提交事务

可能导致脏读

仅可见已提交事务

LSO – Log Stable Offset第一条未提交事务的位置

.txnindex 过滤已回滚消息

LSO（Log Stable Offset）：Broker 维护的 offset，指向第一条未提交事务消息的位置。read_committed 消费者只能消费 LSO 之前的消息。

6. 事务使用示例

6.1 生产者配置与代码

// 事务生产者配置
Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");
props.put("transactional.id", "order-tx-" + UUID.randomUUID()); // 关键：事务ID
props.put("enable.idempotence", "true"); // 必须开启
props.put("acks", "all"); // 必须 all
props.put("max.in.flight.requests.per.connection", "1"); // 幂等自动限制

KafkaProducer<String, String> producer = new KafkaProducer<>(props);
producer.initTransactions(); // 初始化

try {
producer.beginTransaction();

// 发送多条消息到不同分区
producer.send(new ProducerRecord<>("topic1", "key1", "value1"));
producer.send(new ProducerRecord<>("topic2", "key2", "value2"));

producer.commitTransaction();
} catch (Exception e) {
producer.abortTransaction();
}

6.2 Consume-Process-Produce 模式

// 典型流处理场景
producer.initTransactions();

while (true) {
ConsumerRecords<String, String> records = consumer.poll(Duration.ofMillis(100));
if (records.isEmpty()) continue;

producer.beginTransaction();
try {
// 处理并发送结果
for (ConsumerRecord<String, String> record : records) {
String processed = transform(record.value());
producer.send(new ProducerRecord<>("output-topic", record.key(), processed));
}

// 关键：将消费位点作为事务的一部分提交
producer.sendOffsetsToTransaction(getOffsets(records),
new ConsumerGroupMetadata("my-consumer-group"));

producer.commitTransaction();
} catch (Exception e) {
producer.abortTransaction();
}
}

核心：sendOffsetsToTransaction 将消费位移作为事务的一部分，实现消费与生产的原子性。

6.3 消费者配置

Properties props = new Properties();
props.put("isolation.level", "read_committed"); // 只读已提交消息
props.put("enable.auto.commit", "false"); // 必须关闭自动提交

7. 事务限制与性能考量

7.1 功能限制

7.2 性能开销分析

事务引入的额外 RPC 开销：

性能优化建议：

• 事务中包含的消息越多，单条消息均摊开销越小
• 避免高频小事务，建议批量处理
• 生产环境务必设置 max.transaction.timeout.ms 防止长事务阻塞

8. 总结

#mermaid-svg-zROCwoxOfBjYBGzk{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-zROCwoxOfBjYBGzk .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-zROCwoxOfBjYBGzk .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-zROCwoxOfBjYBGzk .error-icon{fill:#552222;}#mermaid-svg-zROCwoxOfBjYBGzk .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-zROCwoxOfBjYBGzk .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-zROCwoxOfBjYBGzk .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-zROCwoxOfBjYBGzk .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-zROCwoxOfBjYBGzk .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-zROCwoxOfBjYBGzk .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-zROCwoxOfBjYBGzk .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-zROCwoxOfBjYBGzk .marker{fill:#333333;stroke:#333333;}#mermaid-svg-zROCwoxOfBjYBGzk .marker.cross{stroke:#333333;}#mermaid-svg-zROCwoxOfBjYBGzk svg{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-zROCwoxOfBjYBGzk p{margin:0;}#mermaid-svg-zROCwoxOfBjYBGzk .label{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;color:#333;}#mermaid-svg-zROCwoxOfBjYBGzk .cluster-label text{fill:#333;}#mermaid-svg-zROCwoxOfBjYBGzk .cluster-label span{color:#333;}#mermaid-svg-zROCwoxOfBjYBGzk .cluster-label span p{background-color:transparent;}#mermaid-svg-zROCwoxOfBjYBGzk .label text,#mermaid-svg-zROCwoxOfBjYBGzk span{fill:#333;color:#333;}#mermaid-svg-zROCwoxOfBjYBGzk .node rect,#mermaid-svg-zROCwoxOfBjYBGzk .node circle,#mermaid-svg-zROCwoxOfBjYBGzk .node ellipse,#mermaid-svg-zROCwoxOfBjYBGzk .node polygon,#mermaid-svg-zROCwoxOfBjYBGzk .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-zROCwoxOfBjYBGzk .rough-node .label text,#mermaid-svg-zROCwoxOfBjYBGzk .node .label text,#mermaid-svg-zROCwoxOfBjYBGzk .image-shape .label,#mermaid-svg-zROCwoxOfBjYBGzk .icon-shape .label{text-anchor:middle;}#mermaid-svg-zROCwoxOfBjYBGzk .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-zROCwoxOfBjYBGzk .rough-node .label,#mermaid-svg-zROCwoxOfBjYBGzk .node .label,#mermaid-svg-zROCwoxOfBjYBGzk .image-shape .label,#mermaid-svg-zROCwoxOfBjYBGzk .icon-shape .label{text-align:center;}#mermaid-svg-zROCwoxOfBjYBGzk .node.clickable{cursor:pointer;}#mermaid-svg-zROCwoxOfBjYBGzk .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-zROCwoxOfBjYBGzk .arrowheadPath{fill:#333333;}#mermaid-svg-zROCwoxOfBjYBGzk .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-zROCwoxOfBjYBGzk .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-zROCwoxOfBjYBGzk .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-zROCwoxOfBjYBGzk .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-zROCwoxOfBjYBGzk .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-zROCwoxOfBjYBGzk .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-zROCwoxOfBjYBGzk .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-zROCwoxOfBjYBGzk .cluster text{fill:#333;}#mermaid-svg-zROCwoxOfBjYBGzk .cluster span{color:#333;}#mermaid-svg-zROCwoxOfBjYBGzk div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-zROCwoxOfBjYBGzk .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-zROCwoxOfBjYBGzk rect.text{fill:none;stroke-width:0;}#mermaid-svg-zROCwoxOfBjYBGzk .icon-shape,#mermaid-svg-zROCwoxOfBjYBGzk .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-zROCwoxOfBjYBGzk .icon-shape p,#mermaid-svg-zROCwoxOfBjYBGzk .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-zROCwoxOfBjYBGzk .icon-shape .label rect,#mermaid-svg-zROCwoxOfBjYBGzk .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-zROCwoxOfBjYBGzk .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-zROCwoxOfBjYBGzk .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-zROCwoxOfBjYBGzk :root{–mermaid-font-family:\”trebuchet ms\”,verdana,arial,sans-serif;}

事务三大支柱

幂等性Sequence Number

Exactly-Once基础

僵尸防护Epoch 机制

两阶段提交2PC + Control Message

Kafka 事务的核心设计哲学：

适用场景：

• ✅ 流处理（Kafka Streams）：消费-处理-生产的原子性保证
• ✅ 跨分区原子写入：多 Topic/Partition 的原子写入
• ❌ 跨系统分布式事务：需要 XA 或 TCC 方案
• ❌ 高频小事务：性能开销过高

原创不易，欢迎点赞收藏 💡 评论区聊聊：你在什么场景下使用过 Kafka 事务？

标签：Kafka 事务消息 Exactly-Once 分布式事务 消息队列