好的!我用 Mermaid 流程图 + 文字说明 + 代码示例 的方式,把 RabbitMQ 和 Kafka 的完整交互流程讲清楚。
一、RabbitMQ 完整交互流程
1.1 整体流程图
消费者队列(Queue)交换机(Exchange)生产者RabbitMQ Server管理员消费者队列(Queue)交换机(Exchange)生产者RabbitMQ Server管理员#mermaid-svg-p7Zq0JbUlbSdBQEV{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-p7Zq0JbUlbSdBQEV .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-p7Zq0JbUlbSdBQEV .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-p7Zq0JbUlbSdBQEV .error-icon{fill:#552222;}#mermaid-svg-p7Zq0JbUlbSdBQEV .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-p7Zq0JbUlbSdBQEV .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-p7Zq0JbUlbSdBQEV .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-p7Zq0JbUlbSdBQEV .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-p7Zq0JbUlbSdBQEV .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-p7Zq0JbUlbSdBQEV .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-p7Zq0JbUlbSdBQEV .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-p7Zq0JbUlbSdBQEV .marker{fill:#333333;stroke:#333333;}#mermaid-svg-p7Zq0JbUlbSdBQEV .marker.cross{stroke:#333333;}#mermaid-svg-p7Zq0JbUlbSdBQEV svg{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-p7Zq0JbUlbSdBQEV p{margin:0;}#mermaid-svg-p7Zq0JbUlbSdBQEV .actor{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-p7Zq0JbUlbSdBQEV text.actor>tspan{fill:black;stroke:none;}#mermaid-svg-p7Zq0JbUlbSdBQEV .actor-line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);}#mermaid-svg-p7Zq0JbUlbSdBQEV .innerArc{stroke-width:1.5;stroke-dasharray:none;}#mermaid-svg-p7Zq0JbUlbSdBQEV .messageLine0{stroke-width:1.5;stroke-dasharray:none;stroke:#333;}#mermaid-svg-p7Zq0JbUlbSdBQEV .messageLine1{stroke-width:1.5;stroke-dasharray:2,2;stroke:#333;}#mermaid-svg-p7Zq0JbUlbSdBQEV #arrowhead path{fill:#333;stroke:#333;}#mermaid-svg-p7Zq0JbUlbSdBQEV .sequenceNumber{fill:white;}#mermaid-svg-p7Zq0JbUlbSdBQEV #sequencenumber{fill:#333;}#mermaid-svg-p7Zq0JbUlbSdBQEV #crosshead path{fill:#333;stroke:#333;}#mermaid-svg-p7Zq0JbUlbSdBQEV .messageText{fill:#333;stroke:none;}#mermaid-svg-p7Zq0JbUlbSdBQEV .labelBox{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-p7Zq0JbUlbSdBQEV .labelText,#mermaid-svg-p7Zq0JbUlbSdBQEV .labelText>tspan{fill:black;stroke:none;}#mermaid-svg-p7Zq0JbUlbSdBQEV .loopText,#mermaid-svg-p7Zq0JbUlbSdBQEV .loopText>tspan{fill:black;stroke:none;}#mermaid-svg-p7Zq0JbUlbSdBQEV .loopLine{stroke-width:2px;stroke-dasharray:2,2;stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);}#mermaid-svg-p7Zq0JbUlbSdBQEV .note{stroke:#aaaa33;fill:#fff5ad;}#mermaid-svg-p7Zq0JbUlbSdBQEV .noteText,#mermaid-svg-p7Zq0JbUlbSdBQEV .noteText>tspan{fill:black;stroke:none;}#mermaid-svg-p7Zq0JbUlbSdBQEV .activation0{fill:#f4f4f4;stroke:#666;}#mermaid-svg-p7Zq0JbUlbSdBQEV .activation1{fill:#f4f4f4;stroke:#666;}#mermaid-svg-p7Zq0JbUlbSdBQEV .activation2{fill:#f4f4f4;stroke:#666;}#mermaid-svg-p7Zq0JbUlbSdBQEV .actorPopupMenu{position:absolute;}#mermaid-svg-p7Zq0JbUlbSdBQEV .actorPopupMenuPanel{position:absolute;fill:#ECECFF;box-shadow:0px 8px 16px 0px rgba(0,0,0,0.2);filter:drop-shadow(3px 5px 2px rgb(0 0 0 / 0.4));}#mermaid-svg-p7Zq0JbUlbSdBQEV .actor-man line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-p7Zq0JbUlbSdBQEV .actor-man circle,#mermaid-svg-p7Zq0JbUlbSdBQEV line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;stroke-width:2px;}#mermaid-svg-p7Zq0JbUlbSdBQEV :root{–mermaid-font-family:\”trebuchet ms\”,verdana,arial,sans-serif;}第1阶段:初始化(提前配置)第2阶段:消费者启动第3阶段:消息发送与消费创建队列 order_queue创建交换机 order_exchange绑定(交换机→队列, routing_key="order.created")启动,监听 order_queue确认监听成功basicPublish(exchange, routing_key="order.created", body)查找绑定表匹配 routing_key路由消息到 order_queue暂存消息等待消费推送消息处理业务逻辑basicAck(确认)删除已确认消息
1.2 详细文字说明
阶段1:初始化(管理员配置)
管理员提前创建好队列和交换机,并建立绑定关系。这就像在邮局提前设置好分拣规则。
阶段2:消费者启动
消费者启动后告诉RabbitMQ:“我要监听 order_queue 这个邮箱”。此时消费者进入等待状态。
阶段3:消息发送与消费
生产者发送消息时指定交换机和 routing_key,交换机根据绑定规则找到目标队列,把消息存进去,然后立即推送给等待中的消费者。消费者处理完必须发送确认,RabbitMQ 收到确认后才删除消息。
1.3 对应代码示例
// ============ 阶段1:管理员初始化 ============
// 创建队列
channel.queueDeclare("order_queue", true, false, false, null);
// 创建交换机
channel.exchangeDeclare("order_exchange", BuiltinExchangeType.DIRECT, true);
// 绑定(交换机 → 队列, routing_key)
channel.queueBind("order_queue", "order_exchange", "order.created");
// ============ 阶段2:消费者启动 ============
// 消费者监听队列
channel.basicConsume("order_queue", false, (consumerTag, delivery) -> {
// ============ 阶段3:消息处理 ============
String message = new String(delivery.getBody());
System.out.println("收到消息: " + message);
// 处理业务逻辑…
processOrder(message);
// 发送确认(告诉RabbitMQ可以删除了)
channel.basicAck(delivery.getEnvelope().getDeliveryTag(), false);
}, consumerTag -> {});
// ============ 生产者发送消息 ============
String message = "订单1001创建成功";
channel.basicPublish("order_exchange", "order.created", null, message.getBytes());
二、Kafka 完整交互流程
2.1 整体流程图
消费者消费者组(Consumer Group)Topic(含多个分区)生产者Kafka Broker管理员消费者消费者组(Consumer Group)Topic(含多个分区)生产者Kafka Broker管理员#mermaid-svg-R4AqyjOAS54i3X3Y{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-R4AqyjOAS54i3X3Y .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-R4AqyjOAS54i3X3Y .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-R4AqyjOAS54i3X3Y .error-icon{fill:#552222;}#mermaid-svg-R4AqyjOAS54i3X3Y .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-R4AqyjOAS54i3X3Y .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-R4AqyjOAS54i3X3Y .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-R4AqyjOAS54i3X3Y .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-R4AqyjOAS54i3X3Y .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-R4AqyjOAS54i3X3Y .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-R4AqyjOAS54i3X3Y .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-R4AqyjOAS54i3X3Y .marker{fill:#333333;stroke:#333333;}#mermaid-svg-R4AqyjOAS54i3X3Y .marker.cross{stroke:#333333;}#mermaid-svg-R4AqyjOAS54i3X3Y svg{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-R4AqyjOAS54i3X3Y p{margin:0;}#mermaid-svg-R4AqyjOAS54i3X3Y .actor{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-R4AqyjOAS54i3X3Y text.actor>tspan{fill:black;stroke:none;}#mermaid-svg-R4AqyjOAS54i3X3Y .actor-line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);}#mermaid-svg-R4AqyjOAS54i3X3Y .innerArc{stroke-width:1.5;stroke-dasharray:none;}#mermaid-svg-R4AqyjOAS54i3X3Y .messageLine0{stroke-width:1.5;stroke-dasharray:none;stroke:#333;}#mermaid-svg-R4AqyjOAS54i3X3Y .messageLine1{stroke-width:1.5;stroke-dasharray:2,2;stroke:#333;}#mermaid-svg-R4AqyjOAS54i3X3Y #arrowhead path{fill:#333;stroke:#333;}#mermaid-svg-R4AqyjOAS54i3X3Y .sequenceNumber{fill:white;}#mermaid-svg-R4AqyjOAS54i3X3Y #sequencenumber{fill:#333;}#mermaid-svg-R4AqyjOAS54i3X3Y #crosshead path{fill:#333;stroke:#333;}#mermaid-svg-R4AqyjOAS54i3X3Y .messageText{fill:#333;stroke:none;}#mermaid-svg-R4AqyjOAS54i3X3Y .labelBox{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-R4AqyjOAS54i3X3Y .labelText,#mermaid-svg-R4AqyjOAS54i3X3Y .labelText>tspan{fill:black;stroke:none;}#mermaid-svg-R4AqyjOAS54i3X3Y .loopText,#mermaid-svg-R4AqyjOAS54i3X3Y .loopText>tspan{fill:black;stroke:none;}#mermaid-svg-R4AqyjOAS54i3X3Y .loopLine{stroke-width:2px;stroke-dasharray:2,2;stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);}#mermaid-svg-R4AqyjOAS54i3X3Y .note{stroke:#aaaa33;fill:#fff5ad;}#mermaid-svg-R4AqyjOAS54i3X3Y .noteText,#mermaid-svg-R4AqyjOAS54i3X3Y .noteText>tspan{fill:black;stroke:none;}#mermaid-svg-R4AqyjOAS54i3X3Y .activation0{fill:#f4f4f4;stroke:#666;}#mermaid-svg-R4AqyjOAS54i3X3Y .activation1{fill:#f4f4f4;stroke:#666;}#mermaid-svg-R4AqyjOAS54i3X3Y .activation2{fill:#f4f4f4;stroke:#666;}#mermaid-svg-R4AqyjOAS54i3X3Y .actorPopupMenu{position:absolute;}#mermaid-svg-R4AqyjOAS54i3X3Y .actorPopupMenuPanel{position:absolute;fill:#ECECFF;box-shadow:0px 8px 16px 0px rgba(0,0,0,0.2);filter:drop-shadow(3px 5px 2px rgb(0 0 0 / 0.4));}#mermaid-svg-R4AqyjOAS54i3X3Y .actor-man line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-R4AqyjOAS54i3X3Y .actor-man circle,#mermaid-svg-R4AqyjOAS54i3X3Y line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;stroke-width:2px;}#mermaid-svg-R4AqyjOAS54i3X3Y :root{–mermaid-font-family:\”trebuchet ms\”,verdana,arial,sans-serif;}第1阶段:初始化(提前配置)第2阶段:消费者组订阅loop[持续拉取]第3阶段:生产者发送消息创建Topic orderspartitions=3加入消费者组订阅Topic orders分配分区(Consumer1→P0, Consumer2→P1, Consumer3→P2)poll() 拉取消息从分配的分区读取消息返回消息(带offset)返回消息处理业务逻辑commitSync() 提交offset计算分区(hash(key) % partitions)直接写入指定分区追加到日志文件末尾分配新的offset
2.2 详细文字说明
阶段1:初始化(管理员配置)
管理员创建 Topic 并指定分区数(比如3个)。分区是 Kafka 的并行单位,每个分区是一个独立的日志文件。
阶段2:消费者组订阅
消费者启动时加入一个消费者组,Kafka 的 Group Coordinator 负责给组内每个消费者分配分区(一个分区只能分配给组内的一个消费者)。消费者持续调用 poll() 拉取消息,处理完后提交 offset(记录读到了哪条消息)。
阶段3:生产者发送消息
生产者发送消息时可以指定 key,Kafka 根据 hash(key) % 分区数 决定写入哪个分区。如果不指定 key,则使用轮询方式。消息直接追加到目标分区的日志文件末尾,获得一个新的 offset。
2.3 对应代码示例
// ============ 阶段1:管理员初始化 ============
// 命令行创建Topic(3个分区)
// kafka-topics.sh –create –topic orders –partitions 3 –replication-factor 1
// ============ 阶段2:消费者组订阅 ============
Properties consumerProps = new Properties();
consumerProps.put("group.id", "order-group"); // 消费者组ID
consumerProps.put("enable.auto.commit", "false"); // 手动提交offset
KafkaConsumer<String, String> consumer = new KafkaConsumer<>(consumerProps);
consumer.subscribe(Arrays.asList("orders"));
// 持续拉取消息
while (true) {
ConsumerRecords<String, String> records = consumer.poll(Duration.ofMillis(1000));
for (ConsumerRecord<String, String> record : records) {
System.out.printf("收到消息: 分区=%d, offset=%d, value=%s%n",
record.partition(), record.offset(), record.value());
// 处理业务逻辑
processOrder(record.value());
}
// 手动提交offset(告诉Kafka我已处理完这批)
consumer.commitSync();
}
// ============ 阶段3:生产者发送消息 ============
KafkaProducer<String, String> producer = new KafkaProducer<>(producerProps);
// 有key:相同key进同一分区
String orderId = "order_1001";
producer.send(new ProducerRecord<>("orders", orderId, "订单数据"));
// 无key:轮询分区
producer.send(new ProducerRecord<>("orders", "订单数据"));
三、核心差异对比流程图
3.1 消息流向对比
#mermaid-svg-4KxZqPUVHYwVnEtH{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-4KxZqPUVHYwVnEtH .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-4KxZqPUVHYwVnEtH .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-4KxZqPUVHYwVnEtH .error-icon{fill:#552222;}#mermaid-svg-4KxZqPUVHYwVnEtH .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-4KxZqPUVHYwVnEtH .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-4KxZqPUVHYwVnEtH .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-4KxZqPUVHYwVnEtH .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-4KxZqPUVHYwVnEtH .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-4KxZqPUVHYwVnEtH .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-4KxZqPUVHYwVnEtH .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-4KxZqPUVHYwVnEtH .marker{fill:#333333;stroke:#333333;}#mermaid-svg-4KxZqPUVHYwVnEtH .marker.cross{stroke:#333333;}#mermaid-svg-4KxZqPUVHYwVnEtH svg{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-4KxZqPUVHYwVnEtH p{margin:0;}#mermaid-svg-4KxZqPUVHYwVnEtH .label{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;color:#333;}#mermaid-svg-4KxZqPUVHYwVnEtH .cluster-label text{fill:#333;}#mermaid-svg-4KxZqPUVHYwVnEtH .cluster-label span{color:#333;}#mermaid-svg-4KxZqPUVHYwVnEtH .cluster-label span p{background-color:transparent;}#mermaid-svg-4KxZqPUVHYwVnEtH .label text,#mermaid-svg-4KxZqPUVHYwVnEtH span{fill:#333;color:#333;}#mermaid-svg-4KxZqPUVHYwVnEtH .node rect,#mermaid-svg-4KxZqPUVHYwVnEtH .node circle,#mermaid-svg-4KxZqPUVHYwVnEtH .node ellipse,#mermaid-svg-4KxZqPUVHYwVnEtH .node polygon,#mermaid-svg-4KxZqPUVHYwVnEtH .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-4KxZqPUVHYwVnEtH .rough-node .label text,#mermaid-svg-4KxZqPUVHYwVnEtH .node .label text,#mermaid-svg-4KxZqPUVHYwVnEtH .image-shape .label,#mermaid-svg-4KxZqPUVHYwVnEtH .icon-shape .label{text-anchor:middle;}#mermaid-svg-4KxZqPUVHYwVnEtH .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-4KxZqPUVHYwVnEtH .rough-node .label,#mermaid-svg-4KxZqPUVHYwVnEtH .node .label,#mermaid-svg-4KxZqPUVHYwVnEtH .image-shape .label,#mermaid-svg-4KxZqPUVHYwVnEtH .icon-shape .label{text-align:center;}#mermaid-svg-4KxZqPUVHYwVnEtH .node.clickable{cursor:pointer;}#mermaid-svg-4KxZqPUVHYwVnEtH .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-4KxZqPUVHYwVnEtH .arrowheadPath{fill:#333333;}#mermaid-svg-4KxZqPUVHYwVnEtH .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-4KxZqPUVHYwVnEtH .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-4KxZqPUVHYwVnEtH .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-4KxZqPUVHYwVnEtH .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-4KxZqPUVHYwVnEtH .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-4KxZqPUVHYwVnEtH .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-4KxZqPUVHYwVnEtH .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-4KxZqPUVHYwVnEtH .cluster text{fill:#333;}#mermaid-svg-4KxZqPUVHYwVnEtH .cluster span{color:#333;}#mermaid-svg-4KxZqPUVHYwVnEtH div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-4KxZqPUVHYwVnEtH .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-4KxZqPUVHYwVnEtH rect.text{fill:none;stroke-width:0;}#mermaid-svg-4KxZqPUVHYwVnEtH .icon-shape,#mermaid-svg-4KxZqPUVHYwVnEtH .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-4KxZqPUVHYwVnEtH .icon-shape p,#mermaid-svg-4KxZqPUVHYwVnEtH .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-4KxZqPUVHYwVnEtH .icon-shape .label rect,#mermaid-svg-4KxZqPUVHYwVnEtH .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-4KxZqPUVHYwVnEtH .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-4KxZqPUVHYwVnEtH .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-4KxZqPUVHYwVnEtH :root{–mermaid-font-family:\”trebuchet ms\”,verdana,arial,sans-serif;}
Kafka
RabbitMQ
路由规则
路由规则
不同组
分区1
交换机Exchange
队列A
队列B
消费者
消费者
分区2
Topic
分区0
消费者组1
消费者组2
3.2 消费确认流程对比
Offset记录Kafka TopicKafka消费者RabbitMQ队列RabbitMQ消费者Offset记录Kafka TopicKafka消费者RabbitMQ队列RabbitMQ消费者#mermaid-svg-u4m3TCZdXxmr6MpI{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-u4m3TCZdXxmr6MpI .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-u4m3TCZdXxmr6MpI .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-u4m3TCZdXxmr6MpI .error-icon{fill:#552222;}#mermaid-svg-u4m3TCZdXxmr6MpI .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-u4m3TCZdXxmr6MpI .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-u4m3TCZdXxmr6MpI .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-u4m3TCZdXxmr6MpI .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-u4m3TCZdXxmr6MpI .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-u4m3TCZdXxmr6MpI .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-u4m3TCZdXxmr6MpI .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-u4m3TCZdXxmr6MpI .marker{fill:#333333;stroke:#333333;}#mermaid-svg-u4m3TCZdXxmr6MpI .marker.cross{stroke:#333333;}#mermaid-svg-u4m3TCZdXxmr6MpI svg{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-u4m3TCZdXxmr6MpI p{margin:0;}#mermaid-svg-u4m3TCZdXxmr6MpI .actor{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-u4m3TCZdXxmr6MpI text.actor>tspan{fill:black;stroke:none;}#mermaid-svg-u4m3TCZdXxmr6MpI .actor-line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);}#mermaid-svg-u4m3TCZdXxmr6MpI .innerArc{stroke-width:1.5;stroke-dasharray:none;}#mermaid-svg-u4m3TCZdXxmr6MpI .messageLine0{stroke-width:1.5;stroke-dasharray:none;stroke:#333;}#mermaid-svg-u4m3TCZdXxmr6MpI .messageLine1{stroke-width:1.5;stroke-dasharray:2,2;stroke:#333;}#mermaid-svg-u4m3TCZdXxmr6MpI #arrowhead path{fill:#333;stroke:#333;}#mermaid-svg-u4m3TCZdXxmr6MpI .sequenceNumber{fill:white;}#mermaid-svg-u4m3TCZdXxmr6MpI #sequencenumber{fill:#333;}#mermaid-svg-u4m3TCZdXxmr6MpI #crosshead path{fill:#333;stroke:#333;}#mermaid-svg-u4m3TCZdXxmr6MpI .messageText{fill:#333;stroke:none;}#mermaid-svg-u4m3TCZdXxmr6MpI .labelBox{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-u4m3TCZdXxmr6MpI .labelText,#mermaid-svg-u4m3TCZdXxmr6MpI .labelText>tspan{fill:black;stroke:none;}#mermaid-svg-u4m3TCZdXxmr6MpI .loopText,#mermaid-svg-u4m3TCZdXxmr6MpI .loopText>tspan{fill:black;stroke:none;}#mermaid-svg-u4m3TCZdXxmr6MpI .loopLine{stroke-width:2px;stroke-dasharray:2,2;stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);}#mermaid-svg-u4m3TCZdXxmr6MpI .note{stroke:#aaaa33;fill:#fff5ad;}#mermaid-svg-u4m3TCZdXxmr6MpI .noteText,#mermaid-svg-u4m3TCZdXxmr6MpI .noteText>tspan{fill:black;stroke:none;}#mermaid-svg-u4m3TCZdXxmr6MpI .activation0{fill:#f4f4f4;stroke:#666;}#mermaid-svg-u4m3TCZdXxmr6MpI .activation1{fill:#f4f4f4;stroke:#666;}#mermaid-svg-u4m3TCZdXxmr6MpI .activation2{fill:#f4f4f4;stroke:#666;}#mermaid-svg-u4m3TCZdXxmr6MpI .actorPopupMenu{position:absolute;}#mermaid-svg-u4m3TCZdXxmr6MpI .actorPopupMenuPanel{position:absolute;fill:#ECECFF;box-shadow:0px 8px 16px 0px rgba(0,0,0,0.2);filter:drop-shadow(3px 5px 2px rgb(0 0 0 / 0.4));}#mermaid-svg-u4m3TCZdXxmr6MpI .actor-man line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-u4m3TCZdXxmr6MpI .actor-man circle,#mermaid-svg-u4m3TCZdXxmr6MpI line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;stroke-width:2px;}#mermaid-svg-u4m3TCZdXxmr6MpI :root{–mermaid-font-family:\”trebuchet ms\”,verdana,arial,sans-serif;}RabbitMQ:先确认后删除Kafka:先处理,再记录位置消息继续保留(可重复消费)推送消息处理消息basicAck(确认)立即删除消息poll(拉取消息, offset=5)处理消息commit(offset=5)
四、完整时序对比表
| 1. 初始化 | 创建队列、交换机、绑定 | 创建Topic(指定分区数) |
| 2. 消费者启动 | 监听队列 | 订阅Topic,加入消费者组 |
| 3. 分区/队列分配 | 多个消费者共享一个队列 | 每个消费者分配固定分区 |
| 4. 发送消息 | 发到交换机,携带routing_key | 发到Topic,可选携带key |
| 5. 路由/分区 | 交换机根据binding路由到队列 | hash(key) % 分区数 |
| 6. 存储 | 队列暂存在内存(可持久化) | 分区追加到磁盘日志 |
| 7. 消费方式 | 推送(Push) | 拉取(Pull) |
| 8. 消费确认 | basicAck后立即删除 | commit offset,消息保留 |
| 9. 重复消费 | 不支持 | 支持(重置offset) |
五、常见问题流程解析
5.1 问题:如果消费者处理失败怎么办?
#mermaid-svg-zml2TeWA2HCQEs4O{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-zml2TeWA2HCQEs4O .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-zml2TeWA2HCQEs4O .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-zml2TeWA2HCQEs4O .error-icon{fill:#552222;}#mermaid-svg-zml2TeWA2HCQEs4O .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-zml2TeWA2HCQEs4O .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-zml2TeWA2HCQEs4O .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-zml2TeWA2HCQEs4O .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-zml2TeWA2HCQEs4O .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-zml2TeWA2HCQEs4O .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-zml2TeWA2HCQEs4O .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-zml2TeWA2HCQEs4O .marker{fill:#333333;stroke:#333333;}#mermaid-svg-zml2TeWA2HCQEs4O .marker.cross{stroke:#333333;}#mermaid-svg-zml2TeWA2HCQEs4O svg{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-zml2TeWA2HCQEs4O p{margin:0;}#mermaid-svg-zml2TeWA2HCQEs4O .label{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;color:#333;}#mermaid-svg-zml2TeWA2HCQEs4O .cluster-label text{fill:#333;}#mermaid-svg-zml2TeWA2HCQEs4O .cluster-label span{color:#333;}#mermaid-svg-zml2TeWA2HCQEs4O .cluster-label span p{background-color:transparent;}#mermaid-svg-zml2TeWA2HCQEs4O .label text,#mermaid-svg-zml2TeWA2HCQEs4O span{fill:#333;color:#333;}#mermaid-svg-zml2TeWA2HCQEs4O .node rect,#mermaid-svg-zml2TeWA2HCQEs4O .node circle,#mermaid-svg-zml2TeWA2HCQEs4O .node ellipse,#mermaid-svg-zml2TeWA2HCQEs4O .node polygon,#mermaid-svg-zml2TeWA2HCQEs4O .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-zml2TeWA2HCQEs4O .rough-node .label text,#mermaid-svg-zml2TeWA2HCQEs4O .node .label text,#mermaid-svg-zml2TeWA2HCQEs4O .image-shape .label,#mermaid-svg-zml2TeWA2HCQEs4O .icon-shape .label{text-anchor:middle;}#mermaid-svg-zml2TeWA2HCQEs4O .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-zml2TeWA2HCQEs4O .rough-node .label,#mermaid-svg-zml2TeWA2HCQEs4O .node .label,#mermaid-svg-zml2TeWA2HCQEs4O .image-shape .label,#mermaid-svg-zml2TeWA2HCQEs4O .icon-shape .label{text-align:center;}#mermaid-svg-zml2TeWA2HCQEs4O .node.clickable{cursor:pointer;}#mermaid-svg-zml2TeWA2HCQEs4O .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-zml2TeWA2HCQEs4O .arrowheadPath{fill:#333333;}#mermaid-svg-zml2TeWA2HCQEs4O .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-zml2TeWA2HCQEs4O .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-zml2TeWA2HCQEs4O .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-zml2TeWA2HCQEs4O .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-zml2TeWA2HCQEs4O .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-zml2TeWA2HCQEs4O .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-zml2TeWA2HCQEs4O .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-zml2TeWA2HCQEs4O .cluster text{fill:#333;}#mermaid-svg-zml2TeWA2HCQEs4O .cluster span{color:#333;}#mermaid-svg-zml2TeWA2HCQEs4O div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-zml2TeWA2HCQEs4O .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-zml2TeWA2HCQEs4O rect.text{fill:none;stroke-width:0;}#mermaid-svg-zml2TeWA2HCQEs4O .icon-shape,#mermaid-svg-zml2TeWA2HCQEs4O .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-zml2TeWA2HCQEs4O .icon-shape p,#mermaid-svg-zml2TeWA2HCQEs4O .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-zml2TeWA2HCQEs4O .icon-shape .label rect,#mermaid-svg-zml2TeWA2HCQEs4O .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-zml2TeWA2HCQEs4O .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-zml2TeWA2HCQEs4O .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-zml2TeWA2HCQEs4O :root{–mermaid-font-family:\”trebuchet ms\”,verdana,arial,sans-serif;}
Kafka
是
否/崩溃
消费者拉取消息
处理成功?
commit offset
下次从新offset开始
offset未提交
重启后重新消费同一条消息
RabbitMQ
是
否/崩溃
消费者收到消息
处理成功?
发送basicAck
消息从队列删除
连接断开
消息重新入队
交给其他消费者
5.2 问题:多个消费者如何协同?
#mermaid-svg-oKuyqh6Vxqwu1f0q{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-oKuyqh6Vxqwu1f0q .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-oKuyqh6Vxqwu1f0q .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-oKuyqh6Vxqwu1f0q .error-icon{fill:#552222;}#mermaid-svg-oKuyqh6Vxqwu1f0q .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-oKuyqh6Vxqwu1f0q .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-oKuyqh6Vxqwu1f0q .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-oKuyqh6Vxqwu1f0q .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-oKuyqh6Vxqwu1f0q .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-oKuyqh6Vxqwu1f0q .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-oKuyqh6Vxqwu1f0q .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-oKuyqh6Vxqwu1f0q .marker{fill:#333333;stroke:#333333;}#mermaid-svg-oKuyqh6Vxqwu1f0q .marker.cross{stroke:#333333;}#mermaid-svg-oKuyqh6Vxqwu1f0q svg{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-oKuyqh6Vxqwu1f0q p{margin:0;}#mermaid-svg-oKuyqh6Vxqwu1f0q .label{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;color:#333;}#mermaid-svg-oKuyqh6Vxqwu1f0q .cluster-label text{fill:#333;}#mermaid-svg-oKuyqh6Vxqwu1f0q .cluster-label span{color:#333;}#mermaid-svg-oKuyqh6Vxqwu1f0q .cluster-label span p{background-color:transparent;}#mermaid-svg-oKuyqh6Vxqwu1f0q .label text,#mermaid-svg-oKuyqh6Vxqwu1f0q span{fill:#333;color:#333;}#mermaid-svg-oKuyqh6Vxqwu1f0q .node rect,#mermaid-svg-oKuyqh6Vxqwu1f0q .node circle,#mermaid-svg-oKuyqh6Vxqwu1f0q .node ellipse,#mermaid-svg-oKuyqh6Vxqwu1f0q .node polygon,#mermaid-svg-oKuyqh6Vxqwu1f0q .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-oKuyqh6Vxqwu1f0q .rough-node .label text,#mermaid-svg-oKuyqh6Vxqwu1f0q .node .label text,#mermaid-svg-oKuyqh6Vxqwu1f0q .image-shape .label,#mermaid-svg-oKuyqh6Vxqwu1f0q .icon-shape .label{text-anchor:middle;}#mermaid-svg-oKuyqh6Vxqwu1f0q .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-oKuyqh6Vxqwu1f0q .rough-node .label,#mermaid-svg-oKuyqh6Vxqwu1f0q .node .label,#mermaid-svg-oKuyqh6Vxqwu1f0q .image-shape .label,#mermaid-svg-oKuyqh6Vxqwu1f0q .icon-shape .label{text-align:center;}#mermaid-svg-oKuyqh6Vxqwu1f0q .node.clickable{cursor:pointer;}#mermaid-svg-oKuyqh6Vxqwu1f0q .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-oKuyqh6Vxqwu1f0q .arrowheadPath{fill:#333333;}#mermaid-svg-oKuyqh6Vxqwu1f0q .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-oKuyqh6Vxqwu1f0q .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-oKuyqh6Vxqwu1f0q .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-oKuyqh6Vxqwu1f0q .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-oKuyqh6Vxqwu1f0q .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-oKuyqh6Vxqwu1f0q .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-oKuyqh6Vxqwu1f0q .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-oKuyqh6Vxqwu1f0q .cluster text{fill:#333;}#mermaid-svg-oKuyqh6Vxqwu1f0q .cluster span{color:#333;}#mermaid-svg-oKuyqh6Vxqwu1f0q div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-oKuyqh6Vxqwu1f0q .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-oKuyqh6Vxqwu1f0q rect.text{fill:none;stroke-width:0;}#mermaid-svg-oKuyqh6Vxqwu1f0q .icon-shape,#mermaid-svg-oKuyqh6Vxqwu1f0q .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-oKuyqh6Vxqwu1f0q .icon-shape p,#mermaid-svg-oKuyqh6Vxqwu1f0q .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-oKuyqh6Vxqwu1f0q .icon-shape .label rect,#mermaid-svg-oKuyqh6Vxqwu1f0q .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-oKuyqh6Vxqwu1f0q .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-oKuyqh6Vxqwu1f0q .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-oKuyqh6Vxqwu1f0q :root{–mermaid-font-family:\”trebuchet ms\”,verdana,arial,sans-serif;}
Kafka分区模式
Topic
分区0消息1,4,7
分区1消息2,5,8
分区2消息3,6,9
消费者A固定消费P0
消费者B固定消费P1
消费者C固定消费P2
RabbitMQ队列模式
队列10个消息
消费者A抢到3个
消费者B抢到3个
消费者C抢到4个
独占分区,分工消费
六、一句话总结(配合流程图记忆)
RabbitMQ 流程:
准备(队列+交换机+绑定) → 消费者监听 → 生产者发送(指定routing_key)
→ 交换机路由 → 队列存储 → 推送消费者 → 确认删除
Kafka 流程:
准备(Topic+分区) → 消费者组订阅 → 分配分区 → 生产者发送(可选key)
→ 计算分区 → 追加日志 → 消费者拉取 → 处理 → 提交offset
核心记忆点:
- RabbitMQ:先绑定,后路由,推送消费,确认即删
- Kafka:先分区,后分配,拉取消费,提交即记
如果你想深入了解某个具体的环节(比如 Kafka 的分区分配算法、RabbitMQ 的确认机制细节等),告诉我,我可以单独画出那个环节的详细流程图。


