一、MySQL Orchestrator 概述:自动化数据库集群管理的利器
MySQL Orchestrator 是一个由 GitHub 维护的开源项目,旨在简化 MySQL 复制拓扑的管理和监控。在传统的 MySQL 主从架构中,当主节点发生故障时,需要手动执行一系列复杂步骤来选举新主节点并重新配置从节点,这不仅耗时,而且容易出错。Orchestrator 的核心价值在于自动化这一过程,它能够自动发现集群拓扑、持续监控节点健康状态,并在主节点故障时快速、安全地执行故障转移,将业务影响降到最低。
1.1 Orchestrator 的核心价值
Orchestrator 通过自动化关键运维操作,为数据库管理员带来了显著的好处:
1.2 Orchestrator 的主要功能模块
Orchestrator 的功能主要围绕 MySQL 复制集群的生命周期管理,包括:
- 拓扑发现:自动扫描并构建 MySQL 复制集群的拓扑图。
- 健康检查:定期检查每个节点的可用性和复制状态。
- 故障转移:在检测到主节点故障时,自动执行故障转移。
- 复制管理:管理从节点的增删、复制通道的修复等。
- 元数据存储:将集群拓扑和状态信息持久化到后端数据库(如 MySQL 或 SQLite)。
二、Orchestrator 自动故障转移原理:从监控到切换的完整流程
Orchestrator 的自动故障转移是一个精密的自动化流程,它依赖于持续的监控和智能的决策算法。其核心原理可以分解为三个关键步骤:拓扑发现与状态监控、故障检测、以及故障转移决策与执行。
2.1 拓扑发现与集群状态监控
Orchestrator 启动后,首先会通过连接集群中的 MySQL 节点来发现整个复制拓扑。它会递归地查询每个节点的 SHOW MASTER STATUS、SHOW SLAVE STATUS 和 SHOW PROCESSLIST 等信息,从而构建一个包含所有主从关系的图结构。这个拓扑图会实时更新,确保 Orchestrator 始终掌握集群的最新状态。
#publish-mermaid-0-1792402916110{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#publish-mermaid-0-1792402916110 .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#publish-mermaid-0-1792402916110 .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#publish-mermaid-0-1792402916110 .error-icon{fill:#552222;}#publish-mermaid-0-1792402916110 .error-text{fill:#552222;stroke:#552222;}#publish-mermaid-0-1792402916110 .edge-thickness-normal{stroke-width:1px;}#publish-mermaid-0-1792402916110 .edge-thickness-thick{stroke-width:3.5px;}#publish-mermaid-0-1792402916110 .edge-pattern-solid{stroke-dasharray:0;}#publish-mermaid-0-1792402916110 .edge-thickness-invisible{stroke-width:0;fill:none;}#publish-mermaid-0-1792402916110 .edge-pattern-dashed{stroke-dasharray:3;}#publish-mermaid-0-1792402916110 .edge-pattern-dotted{stroke-dasharray:2;}#publish-mermaid-0-1792402916110 .marker{fill:#333333;stroke:#333333;}#publish-mermaid-0-1792402916110 .marker.cross{stroke:#333333;}#publish-mermaid-0-1792402916110 svg{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;}#publish-mermaid-0-1792402916110 p{margin:0;}#publish-mermaid-0-1792402916110 .label{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;color:#333;}#publish-mermaid-0-1792402916110 .cluster-label text{fill:#333;}#publish-mermaid-0-1792402916110 .cluster-label span{color:#333;}#publish-mermaid-0-1792402916110 .cluster-label span p{background-color:transparent;}#publish-mermaid-0-1792402916110 .label text,#publish-mermaid-0-1792402916110 span{fill:#333;color:#333;}#publish-mermaid-0-1792402916110 .node rect,#publish-mermaid-0-1792402916110 .node circle,#publish-mermaid-0-1792402916110 .node ellipse,#publish-mermaid-0-1792402916110 .node polygon,#publish-mermaid-0-1792402916110 .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#publish-mermaid-0-1792402916110 .rough-node .label text,#publish-mermaid-0-1792402916110 .node .label text,#publish-mermaid-0-1792402916110 .image-shape .label,#publish-mermaid-0-1792402916110 .icon-shape .label{text-anchor:middle;}#publish-mermaid-0-1792402916110 .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#publish-mermaid-0-1792402916110 .rough-node .label,#publish-mermaid-0-1792402916110 .node .label,#publish-mermaid-0-1792402916110 .image-shape .label,#publish-mermaid-0-1792402916110 .icon-shape .label{text-align:center;}#publish-mermaid-0-1792402916110 .node.clickable{cursor:pointer;}#publish-mermaid-0-1792402916110 .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#publish-mermaid-0-1792402916110 .arrowheadPath{fill:#333333;}#publish-mermaid-0-1792402916110 .edgePath .path{stroke:#333333;stroke-width:1px;}#publish-mermaid-0-1792402916110 .flowchart-link{stroke:#333333;fill:none;}#publish-mermaid-0-1792402916110 .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#publish-mermaid-0-1792402916110 .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#publish-mermaid-0-1792402916110 .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#publish-mermaid-0-1792402916110 .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#publish-mermaid-0-1792402916110 .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#publish-mermaid-0-1792402916110 .cluster text{fill:#333;}#publish-mermaid-0-1792402916110 .cluster span{color:#333;}#publish-mermaid-0-1792402916110 div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#publish-mermaid-0-1792402916110 .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#publish-mermaid-0-1792402916110 rect.text{fill:none;stroke-width:0;}#publish-mermaid-0-1792402916110 .icon-shape,#publish-mermaid-0-1792402916110 .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#publish-mermaid-0-1792402916110 .icon-shape p,#publish-mermaid-0-1792402916110 .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#publish-mermaid-0-1792402916110 .icon-shape .label rect,#publish-mermaid-0-1792402916110 .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#publish-mermaid-0-1792402916110 .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#publish-mermaid-0-1792402916110 .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#publish-mermaid-0-1792402916110 .node .neo-node{stroke:#9370DB;}#publish-mermaid-0-1792402916110 [data-look=\”neo\”].node rect,#publish-mermaid-0-1792402916110 [data-look=\”neo\”].cluster rect,#publish-mermaid-0-1792402916110 [data-look=\”neo\”].node polygon{stroke:#9370DB;filter:drop-shadow(1px 2px 2px rgba(185, 185, 185, 1));}#publish-mermaid-0-1792402916110 [data-look=\”neo\”].swimlane.cluster rect{filter:none;}#publish-mermaid-0-1792402916110 [data-look=\”neo\”].node path{stroke:#9370DB;stroke-width:1px;}#publish-mermaid-0-1792402916110 [data-look=\”neo\”].node .outer-path{filter:drop-shadow(1px 2px 2px rgba(185, 185, 185, 1));}#publish-mermaid-0-1792402916110 [data-look=\”neo\”].node .neo-line path{stroke:#9370DB;filter:none;}#publish-mermaid-0-1792402916110 [data-look=\”neo\”].node circle{stroke:#9370DB;filter:drop-shadow(1px 2px 2px rgba(185, 185, 185, 1));}#publish-mermaid-0-1792402916110 [data-look=\”neo\”].node circle .state-start{fill:#000000;}#publish-mermaid-0-1792402916110 [data-look=\”neo\”].icon-shape .icon{fill:#9370DB;filter:drop-shadow(1px 2px 2px rgba(185, 185, 185, 1));}#publish-mermaid-0-1792402916110 [data-look=\”neo\”].icon-shape .icon-neo path{stroke:#9370DB;filter:drop-shadow(1px 2px 2px rgba(185, 185, 185, 1));}#publish-mermaid-0-1792402916110 :root{–mermaid-font-family:\”trebuchet ms\”,verdana,arial,sans-serif;}
Orchestrator 启动
连接 MySQL 集群
执行 SHOW MASTER STATUS
执行 SHOW SLAVE STATUS
获取主节点信息
获取从节点信息
构建拓扑图
持续监控拓扑变化
此图展示了 Orchestrator 如何通过查询 MySQL 节点信息来构建和监控集群拓扑。它首先连接到集群,然后分别获取主从节点的状态信息,最后将这些信息整合成拓扑图并持续监控。
2.2 故障检测机制
Orchestrator 通过多种方式检测节点故障。最基本的是心跳检测,它会定期尝试连接每个节点。如果连接失败,Orchestrator 会标记该节点为“不可达”。此外,它还会检查复制状态,如果一个从节点长时间没有应用 binlog,也可能被视为异常。Orchestrator 允许配置故障检测的超时时间和重试次数,以避免误判。
Slave_NodeMaster_NodeOrchestratorSlave_NodeMaster_NodeOrchestrator#publish-mermaid-1-1792402916456{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#publish-mermaid-1-1792402916456 .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#publish-mermaid-1-1792402916456 .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#publish-mermaid-1-1792402916456 .error-icon{fill:#552222;}#publish-mermaid-1-1792402916456 .error-text{fill:#552222;stroke:#552222;}#publish-mermaid-1-1792402916456 .edge-thickness-normal{stroke-width:1px;}#publish-mermaid-1-1792402916456 .edge-thickness-thick{stroke-width:3.5px;}#publish-mermaid-1-1792402916456 .edge-pattern-solid{stroke-dasharray:0;}#publish-mermaid-1-1792402916456 .edge-thickness-invisible{stroke-width:0;fill:none;}#publish-mermaid-1-1792402916456 .edge-pattern-dashed{stroke-dasharray:3;}#publish-mermaid-1-1792402916456 .edge-pattern-dotted{stroke-dasharray:2;}#publish-mermaid-1-1792402916456 .marker{fill:#333333;stroke:#333333;}#publish-mermaid-1-1792402916456 .marker.cross{stroke:#333333;}#publish-mermaid-1-1792402916456 svg{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;}#publish-mermaid-1-1792402916456 p{margin:0;}#publish-mermaid-1-1792402916456 .actor{stroke:#9370DB;fill:#ECECFF;stroke-width:1;}#publish-mermaid-1-1792402916456 rect.actor.outer-path[data-look=\”neo\”]{filter:drop-shadow(1px 2px 2px rgba(185, 185, 185, 1));}#publish-mermaid-1-1792402916456 rect.note[data-look=\”neo\”]{stroke:#aaaa33;fill:#fff5ad;filter:drop-shadow(1px 2px 2px rgba(185, 185, 185, 1));}#publish-mermaid-1-1792402916456 text.actor>tspan{fill:black;stroke:none;}#publish-mermaid-1-1792402916456 .actor-line{stroke:#9370DB;}#publish-mermaid-1-1792402916456 .innerArc{stroke-width:1.5;stroke-dasharray:none;}#publish-mermaid-1-1792402916456 .messageLine0{stroke-width:1.5;stroke-dasharray:none;stroke:#333;}#publish-mermaid-1-1792402916456 .messageLine1{stroke-width:1.5;stroke-dasharray:2,2;stroke:#333;}#publish-mermaid-1-1792402916456 [id$=\”-arrowhead\”] path{fill:#333;stroke:#333;}#publish-mermaid-1-1792402916456 .sequenceNumber{fill:white;}#publish-mermaid-1-1792402916456 [id$=\”-sequencenumber\”]{fill:#333;}#publish-mermaid-1-1792402916456 [id$=\”-crosshead\”] path{fill:#333;stroke:#333;}#publish-mermaid-1-1792402916456 .messageText{fill:#333;stroke:none;}#publish-mermaid-1-1792402916456 .labelBox{stroke:#9370DB;fill:#ECECFF;filter:none;}#publish-mermaid-1-1792402916456 .labelText,#publish-mermaid-1-1792402916456 .labelText>tspan{fill:black;stroke:none;}#publish-mermaid-1-1792402916456 .loopText,#publish-mermaid-1-1792402916456 .loopText>tspan{fill:black;stroke:none;}#publish-mermaid-1-1792402916456 .sectionTitle,#publish-mermaid-1-1792402916456 .sectionTitle>tspan{fill:black;stroke:none;}#publish-mermaid-1-1792402916456 .loopLine{stroke-width:2px;stroke-dasharray:2,2;stroke:#9370DB;fill:#9370DB;}#publish-mermaid-1-1792402916456 .note{stroke:#aaaa33;fill:#fff5ad;}#publish-mermaid-1-1792402916456 .noteText,#publish-mermaid-1-1792402916456 .noteText>tspan{fill:black;stroke:none;font-weight:normal;}#publish-mermaid-1-1792402916456 .activation0{fill:#f4f4f4;stroke:#666;}#publish-mermaid-1-1792402916456 .activation1{fill:#f4f4f4;stroke:#666;}#publish-mermaid-1-1792402916456 .activation2{fill:#f4f4f4;stroke:#666;}#publish-mermaid-1-1792402916456 .actorPopupMenu{position:absolute;}#publish-mermaid-1-1792402916456 .actorPopupMenuPanel{position:absolute;fill:#ECECFF;box-shadow:0px 8px 16px 0px rgba(0,0,0,0.2);filter:drop-shadow(3px 5px 2px rgb(0 0 0 / 0.4));}#publish-mermaid-1-1792402916456 .actor-man circle,#publish-mermaid-1-1792402916456 line{fill:#ECECFF;stroke-width:2px;}#publish-mermaid-1-1792402916456 g rect.rect{filter:drop-shadow(1px 2px 2px rgba(185, 185, 185, 1));stroke:#9370DB;}#publish-mermaid-1-1792402916456 .node .neo-node{stroke:#9370DB;}#publish-mermaid-1-1792402916456 [data-look=\”neo\”].node rect,#publish-mermaid-1-1792402916456 [data-look=\”neo\”].cluster rect,#publish-mermaid-1-1792402916456 [data-look=\”neo\”].node polygon{stroke:#9370DB;filter:drop-shadow(1px 2px 2px rgba(185, 185, 185, 1));}#publish-mermaid-1-1792402916456 [data-look=\”neo\”].swimlane.cluster rect{filter:none;}#publish-mermaid-1-1792402916456 [data-look=\”neo\”].node path{stroke:#9370DB;stroke-width:1px;}#publish-mermaid-1-1792402916456 [data-look=\”neo\”].node .outer-path{filter:drop-shadow(1px 2px 2px rgba(185, 185, 185, 1));}#publish-mermaid-1-1792402916456 [data-look=\”neo\”].node .neo-line path{stroke:#9370DB;filter:none;}#publish-mermaid-1-1792402916456 [data-look=\”neo\”].node circle{stroke:#9370DB;filter:drop-shadow(1px 2px 2px rgba(185, 185, 185, 1));}#publish-mermaid-1-1792402916456 [data-look=\”neo\”].node circle .state-start{fill:#000000;}#publish-mermaid-1-1792402916456 [data-look=\”neo\”].icon-shape .icon{fill:#9370DB;filter:drop-shadow(1px 2px 2px rgba(185, 185, 185, 1));}#publish-mermaid-1-1792402916456 [data-look=\”neo\”].icon-shape .icon-neo path{stroke:#9370DB;filter:drop-shadow(1px 2px 2px rgba(185, 185, 185, 1));}#publish-mermaid-1-1792402916456 :root{–mermaid-font-family:\”trebuchet ms\”,verdana,arial,sans-serif;}loop[心跳检测]如果连续 N 次无响应,标记为故障执行 ping 或简单查询响应执行 ping 或简单查询响应
该时序图描述了 Orchestrator 与 MySQL 节点之间进行周期性心跳检测的过程。当节点连续多次无响应时,Orchestrator 会将其判定为故障节点。
2.3 故障转移决策与执行
当 Orchestrator 确认主节点故障后,它会进入决策阶段。它会根据预设的策略(默认是最靠前的可读从节点)从健康的从节点中选出一个作为新主节点。选择新主节点后,Orchestrator 会执行一系列操作:停止旧主节点的复制(如果它恢复),提升新主节点,并重新配置所有其他从节点指向新的主节点。整个过程对应用程序是透明的,因为 Orchestrator 会处理 DNS 或连接字符串的更新(如果配置了)。
#publish-mermaid-2-1792402916493{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#publish-mermaid-2-1792402916493 .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#publish-mermaid-2-1792402916493 .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#publish-mermaid-2-1792402916493 .error-icon{fill:#552222;}#publish-mermaid-2-1792402916493 .error-text{fill:#552222;stroke:#552222;}#publish-mermaid-2-1792402916493 .edge-thickness-normal{stroke-width:1px;}#publish-mermaid-2-1792402916493 .edge-thickness-thick{stroke-width:3.5px;}#publish-mermaid-2-1792402916493 .edge-pattern-solid{stroke-dasharray:0;}#publish-mermaid-2-1792402916493 .edge-thickness-invisible{stroke-width:0;fill:none;}#publish-mermaid-2-1792402916493 .edge-pattern-dashed{stroke-dasharray:3;}#publish-mermaid-2-1792402916493 .edge-pattern-dotted{stroke-dasharray:2;}#publish-mermaid-2-1792402916493 .marker{fill:#333333;stroke:#333333;}#publish-mermaid-2-1792402916493 .marker.cross{stroke:#333333;}#publish-mermaid-2-1792402916493 svg{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;}#publish-mermaid-2-1792402916493 p{margin:0;}#publish-mermaid-2-1792402916493 defs [id$=\”-barbEnd\”]{fill:#333333;stroke:#333333;}#publish-mermaid-2-1792402916493 g.stateGroup text{fill:#9370DB;stroke:none;font-size:10px;}#publish-mermaid-2-1792402916493 g.stateGroup text{fill:#333;stroke:none;font-size:10px;}#publish-mermaid-2-1792402916493 g.stateGroup .state-title{font-weight:bolder;fill:#131300;}#publish-mermaid-2-1792402916493 g.stateGroup rect{fill:#ECECFF;stroke:#9370DB;}#publish-mermaid-2-1792402916493 g.stateGroup line{stroke:#333333;stroke-width:1;}#publish-mermaid-2-1792402916493 .transition{stroke:#333333;stroke-width:1;fill:none;}#publish-mermaid-2-1792402916493 .stateGroup .composit{fill:white;border-bottom:1px;}#publish-mermaid-2-1792402916493 .stateGroup .alt-composit{fill:#e0e0e0;border-bottom:1px;}#publish-mermaid-2-1792402916493 .state-note{stroke:#aaaa33;fill:#fff5ad;}#publish-mermaid-2-1792402916493 .state-note text{fill:black;stroke:none;font-size:10px;}#publish-mermaid-2-1792402916493 .stateLabel .box{stroke:none;stroke-width:0;fill:#ECECFF;opacity:0.5;}#publish-mermaid-2-1792402916493 .edgeLabel .label rect{fill:#ECECFF;opacity:0.5;}#publish-mermaid-2-1792402916493 .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#publish-mermaid-2-1792402916493 .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#publish-mermaid-2-1792402916493 .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#publish-mermaid-2-1792402916493 .edgeLabel .label text{fill:#333;}#publish-mermaid-2-1792402916493 .label div .edgeLabel{color:#333;}#publish-mermaid-2-1792402916493 .stateLabel text{fill:#131300;font-size:10px;font-weight:bold;}#publish-mermaid-2-1792402916493 .node circle.state-start{fill:#333333;stroke:#333333;}#publish-mermaid-2-1792402916493 .node .fork-join{fill:#333333;stroke:#333333;}#publish-mermaid-2-1792402916493 .node circle.state-end{fill:#9370DB;stroke:white;stroke-width:1.5;}#publish-mermaid-2-1792402916493 .end-state-inner{fill:white;stroke-width:1.5;}#publish-mermaid-2-1792402916493 .node rect{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#publish-mermaid-2-1792402916493 .node polygon{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#publish-mermaid-2-1792402916493 [id$=\”-barbEnd\”]{fill:#333333;}#publish-mermaid-2-1792402916493 .statediagram-cluster rect{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#publish-mermaid-2-1792402916493 .cluster-label,#publish-mermaid-2-1792402916493 .nodeLabel{color:#131300;}#publish-mermaid-2-1792402916493 .statediagram-cluster rect.outer{rx:5px;ry:5px;}#publish-mermaid-2-1792402916493 .statediagram-state .divider{stroke:#9370DB;}#publish-mermaid-2-1792402916493 .statediagram-state .title-state{rx:5px;ry:5px;}#publish-mermaid-2-1792402916493 .statediagram-cluster.statediagram-cluster .inner{fill:white;}#publish-mermaid-2-1792402916493 .statediagram-cluster.statediagram-cluster-alt .inner{fill:#f0f0f0;}#publish-mermaid-2-1792402916493 .statediagram-cluster .inner{rx:0;ry:0;}#publish-mermaid-2-1792402916493 .statediagram-state rect.basic{rx:5px;ry:5px;}#publish-mermaid-2-1792402916493 .statediagram-state rect.divider{stroke-dasharray:10,10;fill:#f0f0f0;}#publish-mermaid-2-1792402916493 .note-edge{stroke-dasharray:5;}#publish-mermaid-2-1792402916493 .statediagram-note rect{fill:#fff5ad;stroke:#aaaa33;stroke-width:1px;rx:0;ry:0;}#publish-mermaid-2-1792402916493 .statediagram-note rect{fill:#fff5ad;stroke:#aaaa33;stroke-width:1px;rx:0;ry:0;}#publish-mermaid-2-1792402916493 .statediagram-note text{fill:black;}#publish-mermaid-2-1792402916493 .statediagram-note .nodeLabel{color:black;}#publish-mermaid-2-1792402916493 .statediagram .edgeLabel{color:red;}#publish-mermaid-2-1792402916493 [id$=\”-dependencyStart\”],#publish-mermaid-2-1792402916493 [id$=\”-dependencyEnd\”]{fill:#333333;stroke:#333333;stroke-width:1;}#publish-mermaid-2-1792402916493 .statediagramTitleText{text-anchor:middle;font-size:18px;fill:#333;}#publish-mermaid-2-1792402916493 [data-look=\”neo\”].statediagram-cluster rect{fill:#ECECFF;stroke:#9370DB;stroke-width:1;}#publish-mermaid-2-1792402916493 [data-look=\”neo\”].statediagram-cluster rect.outer{rx:5px;ry:5px;filter:drop-shadow(1px 2px 2px rgba(185, 185, 185, 1));}#publish-mermaid-2-1792402916493 .node .neo-node{stroke:#9370DB;}#publish-mermaid-2-1792402916493 [data-look=\”neo\”].node rect,#publish-mermaid-2-1792402916493 [data-look=\”neo\”].cluster rect,#publish-mermaid-2-1792402916493 [data-look=\”neo\”].node polygon{stroke:#9370DB;filter:drop-shadow(1px 2px 2px rgba(185, 185, 185, 1));}#publish-mermaid-2-1792402916493 [data-look=\”neo\”].swimlane.cluster rect{filter:none;}#publish-mermaid-2-1792402916493 [data-look=\”neo\”].node path{stroke:#9370DB;stroke-width:1px;}#publish-mermaid-2-1792402916493 [data-look=\”neo\”].node .outer-path{filter:drop-shadow(1px 2px 2px rgba(185, 185, 185, 1));}#publish-mermaid-2-1792402916493 [data-look=\”neo\”].node .neo-line path{stroke:#9370DB;filter:none;}#publish-mermaid-2-1792402916493 [data-look=\”neo\”].node circle{stroke:#9370DB;filter:drop-shadow(1px 2px 2px rgba(185, 185, 185, 1));}#publish-mermaid-2-1792402916493 [data-look=\”neo\”].node circle .state-start{fill:#000000;}#publish-mermaid-2-1792402916493 [data-look=\”neo\”].icon-shape .icon{fill:#9370DB;filter:drop-shadow(1px 2px 2px rgba(185, 185, 185, 1));}#publish-mermaid-2-1792402916493 [data-look=\”neo\”].icon-shape .icon-neo path{stroke:#9370DB;filter:drop-shadow(1px 2px 2px rgba(185, 185, 185, 1));}#publish-mermaid-2-1792402916493 :root{–mermaid-font-family:\”trebuchet ms\”,verdana,arial,sans-serif;}检测到主节点无响应分析健康从节点选择最优从节点提升新主节点重新配置从节点故障转移完成,继续监控
Monitoring
Detecting_Failure
Analyzing_Topology
Selecting_New_Master
Promoting_New_Master
Reconfiguring_Slaves
此状态图清晰地展示了 Orchestrator 在检测到故障后,如何从监控状态过渡到分析、选择、提升和重新配置,最终回到监控状态,完成一次完整的故障转移循环。
三、Orchestrator 部署准备:环境与依赖检查
在部署 Orchestrator 之前,需要确保运行环境和 MySQL 集群已经满足基本要求。这包括安装必要的软件依赖和正确配置 MySQL 集群本身。
3.1 环境要求
Orchestrator 是一个用 Go 语言编写的单二进制文件,部署非常简单。其基本环境要求如下:
| 操作系统 | Linux (推荐 CentOS 7+, Ubuntu 16.04+) |
| Go 语言 | Go 1.13 或更高版本 (用于编译源码) |
| MySQL 客户端 | MySQL 客户端库 (libmysqlclient) |
| 后端存储 | MySQL 5.6+ 或 SQLite 3 (用于存储元数据) |
3.2 MySQL 集群准备
为了使 Orchestrator 能够正常工作,MySQL 集群需要启用 GTID(全局事务标识符),这是 Orchestrator 进行精确拓扑发现和故障转移的关键。此外,还需要创建一个具有特定权限的复制用户,供 Orchestrator 连接和操作集群。
#publish-mermaid-3-1792402916578{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#publish-mermaid-3-1792402916578 .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#publish-mermaid-3-1792402916578 .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#publish-mermaid-3-1792402916578 .error-icon{fill:#552222;}#publish-mermaid-3-1792402916578 .error-text{fill:#552222;stroke:#552222;}#publish-mermaid-3-1792402916578 .edge-thickness-normal{stroke-width:1px;}#publish-mermaid-3-1792402916578 .edge-thickness-thick{stroke-width:3.5px;}#publish-mermaid-3-1792402916578 .edge-pattern-solid{stroke-dasharray:0;}#publish-mermaid-3-1792402916578 .edge-thickness-invisible{stroke-width:0;fill:none;}#publish-mermaid-3-1792402916578 .edge-pattern-dashed{stroke-dasharray:3;}#publish-mermaid-3-1792402916578 .edge-pattern-dotted{stroke-dasharray:2;}#publish-mermaid-3-1792402916578 .marker{fill:#333333;stroke:#333333;}#publish-mermaid-3-1792402916578 .marker.cross{stroke:#333333;}#publish-mermaid-3-1792402916578 svg{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;}#publish-mermaid-3-1792402916578 p{margin:0;}#publish-mermaid-3-1792402916578 .entityBox{fill:#ECECFF;stroke:#9370DB;}#publish-mermaid-3-1792402916578 .relationshipLabelBox{fill:hsl(80, 100%, 96.2745098039%);opacity:0.7;background-color:hsl(80, 100%, 96.2745098039%);}#publish-mermaid-3-1792402916578 .relationshipLabelBox rect{opacity:0.5;}#publish-mermaid-3-1792402916578 .labelBkg{background-color:rgba(248.6666666666, 255, 235.9999999999, 0.5);}#publish-mermaid-3-1792402916578 .edgeLabel{background-color:rgba(232,232,232, 0.8);}#publish-mermaid-3-1792402916578 .edgeLabel .label rect{fill:rgba(232,232,232, 0.8);}#publish-mermaid-3-1792402916578 .edgeLabel .label text{fill:#333;}#publish-mermaid-3-1792402916578 .edgeLabel .label{fill:#9370DB;font-size:14px;}#publish-mermaid-3-1792402916578 .label{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;color:#333;}#publish-mermaid-3-1792402916578 .edge-pattern-dashed{stroke-dasharray:8,8;}#publish-mermaid-3-1792402916578 .node rect,#publish-mermaid-3-1792402916578 .node circle,#publish-mermaid-3-1792402916578 .node ellipse,#publish-mermaid-3-1792402916578 .node polygon{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#publish-mermaid-3-1792402916578 .relationshipLine{stroke:#333333;stroke-width:1px;fill:none;}#publish-mermaid-3-1792402916578 .marker{fill:none!important;stroke:#333333!important;stroke-width:1;}#publish-mermaid-3-1792402916578 [data-look=neo].labelBkg{background-color:rgba(248.6666666666, 255, 235.9999999999, 0.5);}#publish-mermaid-3-1792402916578 .node .neo-node{stroke:#9370DB;}#publish-mermaid-3-1792402916578 [data-look=\”neo\”].node rect,#publish-mermaid-3-1792402916578 [data-look=\”neo\”].cluster rect,#publish-mermaid-3-1792402916578 [data-look=\”neo\”].node polygon{stroke:#9370DB;filter:drop-shadow(1px 2px 2px rgba(185, 185, 185, 1));}#publish-mermaid-3-1792402916578 [data-look=\”neo\”].swimlane.cluster rect{filter:none;}#publish-mermaid-3-1792402916578 [data-look=\”neo\”].node path{stroke:#9370DB;stroke-width:1px;}#publish-mermaid-3-1792402916578 [data-look=\”neo\”].node .outer-path{filter:drop-shadow(1px 2px 2px rgba(185, 185, 185, 1));}#publish-mermaid-3-1792402916578 [data-look=\”neo\”].node .neo-line path{stroke:#9370DB;filter:none;}#publish-mermaid-3-1792402916578 [data-look=\”neo\”].node circle{stroke:#9370DB;filter:drop-shadow(1px 2px 2px rgba(185, 185, 185, 1));}#publish-mermaid-3-1792402916578 [data-look=\”neo\”].node circle .state-start{fill:#000000;}#publish-mermaid-3-1792402916578 [data-look=\”neo\”].icon-shape .icon{fill:#9370DB;filter:drop-shadow(1px 2px 2px rgba(185, 185, 185, 1));}#publish-mermaid-3-1792402916578 [data-look=\”neo\”].icon-shape .icon-neo path{stroke:#9370DB;filter:drop-shadow(1px 2px 2px rgba(185, 185, 185, 1));}#publish-mermaid-3-1792402916578 :root{–mermaid-font-family:\”trebuchet ms\”,verdana,arial,sans-serif;}管理与监控包含节点
Orchestrator_Config
string
dsn
string
username
string
password
MySQL_Cluster
string
cluster_name
string
topology
MySQL_Node
string
node_id
string
host
string
port
string
role
该 ER 图展示了 Orchestrator 配置与它所管理的 MySQL 集群之间的关系。Orchestrator 通过配置信息(DSN、用户名、密码)连接到 MySQL 集群,并管理其内部的节点。
四、Orchestrator 部署与配置:从安装到启动
部署 Orchestrator 主要包括下载二进制文件、配置核心参数以及启动服务三个步骤。Orchestrator 支持多种部署方式,最常用的是直接使用预编译的二进制文件。
4.1 下载与安装
最简单的方式是直接从 GitHub 下载最新的预编译二进制文件。对于 Linux 系统,可以直接下载并赋予执行权限。
#publish-mermaid-4-1792402916672{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#publish-mermaid-4-1792402916672 .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#publish-mermaid-4-1792402916672 .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#publish-mermaid-4-1792402916672 .error-icon{fill:#552222;}#publish-mermaid-4-1792402916672 .error-text{fill:#552222;stroke:#552222;}#publish-mermaid-4-1792402916672 .edge-thickness-normal{stroke-width:1px;}#publish-mermaid-4-1792402916672 .edge-thickness-thick{stroke-width:3.5px;}#publish-mermaid-4-1792402916672 .edge-pattern-solid{stroke-dasharray:0;}#publish-mermaid-4-1792402916672 .edge-thickness-invisible{stroke-width:0;fill:none;}#publish-mermaid-4-1792402916672 .edge-pattern-dashed{stroke-dasharray:3;}#publish-mermaid-4-1792402916672 .edge-pattern-dotted{stroke-dasharray:2;}#publish-mermaid-4-1792402916672 .marker{fill:#333333;stroke:#333333;}#publish-mermaid-4-1792402916672 .marker.cross{stroke:#333333;}#publish-mermaid-4-1792402916672 svg{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;}#publish-mermaid-4-1792402916672 p{margin:0;}#publish-mermaid-4-1792402916672 .label{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;color:#333;}#publish-mermaid-4-1792402916672 .cluster-label text{fill:#333;}#publish-mermaid-4-1792402916672 .cluster-label span{color:#333;}#publish-mermaid-4-1792402916672 .cluster-label span p{background-color:transparent;}#publish-mermaid-4-1792402916672 .label text,#publish-mermaid-4-1792402916672 span{fill:#333;color:#333;}#publish-mermaid-4-1792402916672 .node rect,#publish-mermaid-4-1792402916672 .node circle,#publish-mermaid-4-1792402916672 .node ellipse,#publish-mermaid-4-1792402916672 .node polygon,#publish-mermaid-4-1792402916672 .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#publish-mermaid-4-1792402916672 .rough-node .label text,#publish-mermaid-4-1792402916672 .node .label text,#publish-mermaid-4-1792402916672 .image-shape .label,#publish-mermaid-4-1792402916672 .icon-shape .label{text-anchor:middle;}#publish-mermaid-4-1792402916672 .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#publish-mermaid-4-1792402916672 .rough-node .label,#publish-mermaid-4-1792402916672 .node .label,#publish-mermaid-4-1792402916672 .image-shape .label,#publish-mermaid-4-1792402916672 .icon-shape .label{text-align:center;}#publish-mermaid-4-1792402916672 .node.clickable{cursor:pointer;}#publish-mermaid-4-1792402916672 .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#publish-mermaid-4-1792402916672 .arrowheadPath{fill:#333333;}#publish-mermaid-4-1792402916672 .edgePath .path{stroke:#333333;stroke-width:1px;}#publish-mermaid-4-1792402916672 .flowchart-link{stroke:#333333;fill:none;}#publish-mermaid-4-1792402916672 .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#publish-mermaid-4-1792402916672 .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#publish-mermaid-4-1792402916672 .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#publish-mermaid-4-1792402916672 .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#publish-mermaid-4-1792402916672 .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#publish-mermaid-4-1792402916672 .cluster text{fill:#333;}#publish-mermaid-4-1792402916672 .cluster span{color:#333;}#publish-mermaid-4-1792402916672 div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#publish-mermaid-4-1792402916672 .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#publish-mermaid-4-1792402916672 rect.text{fill:none;stroke-width:0;}#publish-mermaid-4-1792402916672 .icon-shape,#publish-mermaid-4-1792402916672 .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#publish-mermaid-4-1792402916672 .icon-shape p,#publish-mermaid-4-1792402916672 .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#publish-mermaid-4-1792402916672 .icon-shape .label rect,#publish-mermaid-4-1792402916672 .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#publish-mermaid-4-1792402916672 .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#publish-mermaid-4-1792402916672 .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#publish-mermaid-4-1792402916672 .node .neo-node{stroke:#9370DB;}#publish-mermaid-4-1792402916672 [data-look=\”neo\”].node rect,#publish-mermaid-4-1792402916672 [data-look=\”neo\”].cluster rect,#publish-mermaid-4-1792402916672 [data-look=\”neo\”].node polygon{stroke:#9370DB;filter:drop-shadow(1px 2px 2px rgba(185, 185, 185, 1));}#publish-mermaid-4-1792402916672 [data-look=\”neo\”].swimlane.cluster rect{filter:none;}#publish-mermaid-4-1792402916672 [data-look=\”neo\”].node path{stroke:#9370DB;stroke-width:1px;}#publish-mermaid-4-1792402916672 [data-look=\”neo\”].node .outer-path{filter:drop-shadow(1px 2px 2px rgba(185, 185, 185, 1));}#publish-mermaid-4-1792402916672 [data-look=\”neo\”].node .neo-line path{stroke:#9370DB;filter:none;}#publish-mermaid-4-1792402916672 [data-look=\”neo\”].node circle{stroke:#9370DB;filter:drop-shadow(1px 2px 2px rgba(185, 185, 185, 1));}#publish-mermaid-4-1792402916672 [data-look=\”neo\”].node circle .state-start{fill:#000000;}#publish-mermaid-4-1792402916672 [data-look=\”neo\”].icon-shape .icon{fill:#9370DB;filter:drop-shadow(1px 2px 2px rgba(185, 185, 185, 1));}#publish-mermaid-4-1792402916672 [data-look=\”neo\”].icon-shape .icon-neo path{stroke:#9370DB;filter:drop-shadow(1px 2px 2px rgba(185, 185, 185, 1));}#publish-mermaid-4-1792402916672 :root{–mermaid-font-family:\”trebuchet ms\”,verdana,arial,sans-serif;}
访问 GitHub Releases 页面
下载对应系统的二进制文件
解压或移动到 PATH 目录
赋予执行权限: chmod +x orchestrator
此流程图展示了下载和准备 Orchestrator 二进制文件的标准步骤,确保其可以在系统路径中直接运行。
4.2 配置文件详解
Orchestrator 的行为主要通过配置文件 orchestrator.conf.json 控制。以下是一些关键配置项:
- MySQLOrchestratorDSN: 指定 Orchestrator 元数据存储的 MySQL 连接字符串。
- MySQLOrchestratorUser & MySQLOrchestratorPassword: 用于连接元数据存储的用户名和密码。
- MySQLOrchestratorMap: 定义 Orchestrator 要管理的 MySQL 集群,格式为 "集群名: MySQL 连接字符串"。
- PrimaryCluster: 指定默认管理的集群。
- DiscovererInterval: 拓扑发现的间隔时间(秒)。
4.3 启动 Orchestrator 服务
配置完成后,可以通过命令行启动 Orchestrator 服务。默认情况下,它会启动一个 Web 服务,监听 3000 端口。
#publish-mermaid-5-1792402916727{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#publish-mermaid-5-1792402916727 .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#publish-mermaid-5-1792402916727 .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#publish-mermaid-5-1792402916727 .error-icon{fill:#552222;}#publish-mermaid-5-1792402916727 .error-text{fill:#552222;stroke:#552222;}#publish-mermaid-5-1792402916727 .edge-thickness-normal{stroke-width:1px;}#publish-mermaid-5-1792402916727 .edge-thickness-thick{stroke-width:3.5px;}#publish-mermaid-5-1792402916727 .edge-pattern-solid{stroke-dasharray:0;}#publish-mermaid-5-1792402916727 .edge-thickness-invisible{stroke-width:0;fill:none;}#publish-mermaid-5-1792402916727 .edge-pattern-dashed{stroke-dasharray:3;}#publish-mermaid-5-1792402916727 .edge-pattern-dotted{stroke-dasharray:2;}#publish-mermaid-5-1792402916727 .marker{fill:#333333;stroke:#333333;}#publish-mermaid-5-1792402916727 .marker.cross{stroke:#333333;}#publish-mermaid-5-1792402916727 svg{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;}#publish-mermaid-5-1792402916727 p{margin:0;}#publish-mermaid-5-1792402916727 .label{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;color:#333;}#publish-mermaid-5-1792402916727 .cluster-label text{fill:#333;}#publish-mermaid-5-1792402916727 .cluster-label span{color:#333;}#publish-mermaid-5-1792402916727 .cluster-label span p{background-color:transparent;}#publish-mermaid-5-1792402916727 .label text,#publish-mermaid-5-1792402916727 span{fill:#333;color:#333;}#publish-mermaid-5-1792402916727 .node rect,#publish-mermaid-5-1792402916727 .node circle,#publish-mermaid-5-1792402916727 .node ellipse,#publish-mermaid-5-1792402916727 .node polygon,#publish-mermaid-5-1792402916727 .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#publish-mermaid-5-1792402916727 .rough-node .label text,#publish-mermaid-5-1792402916727 .node .label text,#publish-mermaid-5-1792402916727 .image-shape .label,#publish-mermaid-5-1792402916727 .icon-shape .label{text-anchor:middle;}#publish-mermaid-5-1792402916727 .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#publish-mermaid-5-1792402916727 .rough-node .label,#publish-mermaid-5-1792402916727 .node .label,#publish-mermaid-5-1792402916727 .image-shape .label,#publish-mermaid-5-1792402916727 .icon-shape .label{text-align:center;}#publish-mermaid-5-1792402916727 .node.clickable{cursor:pointer;}#publish-mermaid-5-1792402916727 .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#publish-mermaid-5-1792402916727 .arrowheadPath{fill:#333333;}#publish-mermaid-5-1792402916727 .edgePath .path{stroke:#333333;stroke-width:1px;}#publish-mermaid-5-1792402916727 .flowchart-link{stroke:#333333;fill:none;}#publish-mermaid-5-1792402916727 .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#publish-mermaid-5-1792402916727 .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#publish-mermaid-5-1792402916727 .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#publish-mermaid-5-1792402916727 .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#publish-mermaid-5-1792402916727 .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#publish-mermaid-5-1792402916727 .cluster text{fill:#333;}#publish-mermaid-5-1792402916727 .cluster span{color:#333;}#publish-mermaid-5-1792402916727 div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#publish-mermaid-5-1792402916727 .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#publish-mermaid-5-1792402916727 rect.text{fill:none;stroke-width:0;}#publish-mermaid-5-1792402916727 .icon-shape,#publish-mermaid-5-1792402916727 .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#publish-mermaid-5-1792402916727 .icon-shape p,#publish-mermaid-5-1792402916727 .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#publish-mermaid-5-1792402916727 .icon-shape .label rect,#publish-mermaid-5-1792402916727 .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#publish-mermaid-5-1792402916727 .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#publish-mermaid-5-1792402916727 .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#publish-mermaid-5-1792402916727 .node .neo-node{stroke:#9370DB;}#publish-mermaid-5-1792402916727 [data-look=\”neo\”].node rect,#publish-mermaid-5-1792402916727 [data-look=\”neo\”].cluster rect,#publish-mermaid-5-1792402916727 [data-look=\”neo\”].node polygon{stroke:#9370DB;filter:drop-shadow(1px 2px 2px rgba(185, 185, 185, 1));}#publish-mermaid-5-1792402916727 [data-look=\”neo\”].swimlane.cluster rect{filter:none;}#publish-mermaid-5-1792402916727 [data-look=\”neo\”].node path{stroke:#9370DB;stroke-width:1px;}#publish-mermaid-5-1792402916727 [data-look=\”neo\”].node .outer-path{filter:drop-shadow(1px 2px 2px rgba(185, 185, 185, 1));}#publish-mermaid-5-1792402916727 [data-look=\”neo\”].node .neo-line path{stroke:#9370DB;filter:none;}#publish-mermaid-5-1792402916727 [data-look=\”neo\”].node circle{stroke:#9370DB;filter:drop-shadow(1px 2px 2px rgba(185, 185, 185, 1));}#publish-mermaid-5-1792402916727 [data-look=\”neo\”].node circle .state-start{fill:#000000;}#publish-mermaid-5-1792402916727 [data-look=\”neo\”].icon-shape .icon{fill:#9370DB;filter:drop-shadow(1px 2px 2px rgba(185, 185, 185, 1));}#publish-mermaid-5-1792402916727 [data-look=\”neo\”].icon-shape .icon-neo path{stroke:#9370DB;filter:drop-shadow(1px 2px 2px rgba(185, 185, 185, 1));}#publish-mermaid-5-1792402916727 :root{–mermaid-font-family:\”trebuchet ms\”,verdana,arial,sans-serif;}
执行启动命令: ./orchestrator -config orchestrator.conf.json
加载配置文件
连接元数据存储
开始拓扑发现
启动 Web 服务 (默认 3000 端口)
提供 REST API 和 Web 界面
该图展示了 Orchestrator 启动后的内部流程,从加载配置到启动 Web 服务,为后续的管理和监控做好准备。
五、Web 界面与集群管理:可视化操作与验证
Orchestrator 提供了一个功能丰富的 Web 界面,允许管理员以图形化的方式查看和管理 MySQL 集群。
5.1 访问 Web 界面
启动 Orchestrator 后,可以通过浏览器访问 http://<orchestrator_host>:3000 来打开 Web 界面。默认账号是 root,密码为空。
5.2 查看集群拓扑
Web 界面的核心是集群拓扑图,它以可视化的方式展示了所有 MySQL 节点及其主从关系。节点会根据其角色(主/从)和状态(在线/离线)以不同颜色显示,方便管理员快速了解集群健康状况。
5.3 手动触发故障转移
在 Web 界面中,管理员可以手动选择一个从节点并将其提升为新主节点,这对于测试或特定场景下的故障转移非常有用。这有助于验证集群的配置和 Orchestrator 的功能是否正常工作。
#publish-mermaid-6-1792402916821{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#publish-mermaid-6-1792402916821 .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#publish-mermaid-6-1792402916821 .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#publish-mermaid-6-1792402916821 .error-icon{fill:#552222;}#publish-mermaid-6-1792402916821 .error-text{fill:#552222;stroke:#552222;}#publish-mermaid-6-1792402916821 .edge-thickness-normal{stroke-width:1px;}#publish-mermaid-6-1792402916821 .edge-thickness-thick{stroke-width:3.5px;}#publish-mermaid-6-1792402916821 .edge-pattern-solid{stroke-dasharray:0;}#publish-mermaid-6-1792402916821 .edge-thickness-invisible{stroke-width:0;fill:none;}#publish-mermaid-6-1792402916821 .edge-pattern-dashed{stroke-dasharray:3;}#publish-mermaid-6-1792402916821 .edge-pattern-dotted{stroke-dasharray:2;}#publish-mermaid-6-1792402916821 .marker{fill:#333333;stroke:#333333;}#publish-mermaid-6-1792402916821 .marker.cross{stroke:#333333;}#publish-mermaid-6-1792402916821 svg{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;}#publish-mermaid-6-1792402916821 p{margin:0;}#publish-mermaid-6-1792402916821 .label{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;color:#333;}#publish-mermaid-6-1792402916821 .cluster-label text{fill:#333;}#publish-mermaid-6-1792402916821 .cluster-label span{color:#333;}#publish-mermaid-6-1792402916821 .cluster-label span p{background-color:transparent;}#publish-mermaid-6-1792402916821 .label text,#publish-mermaid-6-1792402916821 span{fill:#333;color:#333;}#publish-mermaid-6-1792402916821 .node rect,#publish-mermaid-6-1792402916821 .node circle,#publish-mermaid-6-1792402916821 .node ellipse,#publish-mermaid-6-1792402916821 .node polygon,#publish-mermaid-6-1792402916821 .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#publish-mermaid-6-1792402916821 .rough-node .label text,#publish-mermaid-6-1792402916821 .node .label text,#publish-mermaid-6-1792402916821 .image-shape .label,#publish-mermaid-6-1792402916821 .icon-shape .label{text-anchor:middle;}#publish-mermaid-6-1792402916821 .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#publish-mermaid-6-1792402916821 .rough-node .label,#publish-mermaid-6-1792402916821 .node .label,#publish-mermaid-6-1792402916821 .image-shape .label,#publish-mermaid-6-1792402916821 .icon-shape .label{text-align:center;}#publish-mermaid-6-1792402916821 .node.clickable{cursor:pointer;}#publish-mermaid-6-1792402916821 .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#publish-mermaid-6-1792402916821 .arrowheadPath{fill:#333333;}#publish-mermaid-6-1792402916821 .edgePath .path{stroke:#333333;stroke-width:1px;}#publish-mermaid-6-1792402916821 .flowchart-link{stroke:#333333;fill:none;}#publish-mermaid-6-1792402916821 .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#publish-mermaid-6-1792402916821 .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#publish-mermaid-6-1792402916821 .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#publish-mermaid-6-1792402916821 .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#publish-mermaid-6-1792402916821 .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#publish-mermaid-6-1792402916821 .cluster text{fill:#333;}#publish-mermaid-6-1792402916821 .cluster span{color:#333;}#publish-mermaid-6-1792402916821 div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#publish-mermaid-6-1792402916821 .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#publish-mermaid-6-1792402916821 rect.text{fill:none;stroke-width:0;}#publish-mermaid-6-1792402916821 .icon-shape,#publish-mermaid-6-1792402916821 .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#publish-mermaid-6-1792402916821 .icon-shape p,#publish-mermaid-6-1792402916821 .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#publish-mermaid-6-1792402916821 .icon-shape .label rect,#publish-mermaid-6-1792402916821 .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#publish-mermaid-6-1792402916821 .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#publish-mermaid-6-1792402916821 .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#publish-mermaid-6-1792402916821 .node .neo-node{stroke:#9370DB;}#publish-mermaid-6-1792402916821 [data-look=\”neo\”].node rect,#publish-mermaid-6-1792402916821 [data-look=\”neo\”].cluster rect,#publish-mermaid-6-1792402916821 [data-look=\”neo\”].node polygon{stroke:#9370DB;filter:drop-shadow(1px 2px 2px rgba(185, 185, 185, 1));}#publish-mermaid-6-1792402916821 [data-look=\”neo\”].swimlane.cluster rect{filter:none;}#publish-mermaid-6-1792402916821 [data-look=\”neo\”].node path{stroke:#9370DB;stroke-width:1px;}#publish-mermaid-6-1792402916821 [data-look=\”neo\”].node .outer-path{filter:drop-shadow(1px 2px 2px rgba(185, 185, 185, 1));}#publish-mermaid-6-1792402916821 [data-look=\”neo\”].node .neo-line path{stroke:#9370DB;filter:none;}#publish-mermaid-6-1792402916821 [data-look=\”neo\”].node circle{stroke:#9370DB;filter:drop-shadow(1px 2px 2px rgba(185, 185, 185, 1));}#publish-mermaid-6-1792402916821 [data-look=\”neo\”].node circle .state-start{fill:#000000;}#publish-mermaid-6-1792402916821 [data-look=\”neo\”].icon-shape .icon{fill:#9370DB;filter:drop-shadow(1px 2px 2px rgba(185, 185, 185, 1));}#publish-mermaid-6-1792402916821 [data-look=\”neo\”].icon-shape .icon-neo path{stroke:#9370DB;filter:drop-shadow(1px 2px 2px rgba(185, 185, 185, 1));}#publish-mermaid-6-1792402916821 :root{–mermaid-font-family:\”trebuchet ms\”,verdana,arial,sans-serif;}
User
Web_UI
Topology_View
Select_Slave_Node
Click_Promote_Button
Orchestrator_Backend
Execute_Failover
Update_Topology
此流程图模拟了用户通过 Web 界面手动触发故障转移的交互过程,从选择节点到后端执行,再到界面更新。
六、自动故障转移实战:模拟主节点故障
为了验证 Orchestrator 的自动故障转移功能,我们可以模拟主节点宕机场景,观察 Orchestrator 如何自动响应并完成切换。
6.1 模拟故障场景
可以通过 kill -9 <mysql_pid> 命令强制停止当前的主 MySQL 节点进程,或者直接关闭虚拟机/容器来模拟更真实的故障。
6.2 观察自动切换过程
在主节点宕机后,Orchestrator 会在其配置的检测间隔内发现故障。随后,它会自动执行故障转移流程,选择一个健康的从节点作为新主节点,并通知所有其他从节点切换复制源。整个过程通常在几十秒内完成。通过 Web 界面,可以实时观察到拓扑图中主节点角色的切换。
SlaveNew_MasterOrchestratorOld_MasterAppSlaveNew_MasterOrchestratorOld_MasterApp#publish-mermaid-7-1792402916865{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#publish-mermaid-7-1792402916865 .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#publish-mermaid-7-1792402916865 .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#publish-mermaid-7-1792402916865 .error-icon{fill:#552222;}#publish-mermaid-7-1792402916865 .error-text{fill:#552222;stroke:#552222;}#publish-mermaid-7-1792402916865 .edge-thickness-normal{stroke-width:1px;}#publish-mermaid-7-1792402916865 .edge-thickness-thick{stroke-width:3.5px;}#publish-mermaid-7-1792402916865 .edge-pattern-solid{stroke-dasharray:0;}#publish-mermaid-7-1792402916865 .edge-thickness-invisible{stroke-width:0;fill:none;}#publish-mermaid-7-1792402916865 .edge-pattern-dashed{stroke-dasharray:3;}#publish-mermaid-7-1792402916865 .edge-pattern-dotted{stroke-dasharray:2;}#publish-mermaid-7-1792402916865 .marker{fill:#333333;stroke:#333333;}#publish-mermaid-7-1792402916865 .marker.cross{stroke:#333333;}#publish-mermaid-7-1792402916865 svg{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;}#publish-mermaid-7-1792402916865 p{margin:0;}#publish-mermaid-7-1792402916865 .actor{stroke:#9370DB;fill:#ECECFF;stroke-width:1;}#publish-mermaid-7-1792402916865 rect.actor.outer-path[data-look=\”neo\”]{filter:drop-shadow(1px 2px 2px rgba(185, 185, 185, 1));}#publish-mermaid-7-1792402916865 rect.note[data-look=\”neo\”]{stroke:#aaaa33;fill:#fff5ad;filter:drop-shadow(1px 2px 2px rgba(185, 185, 185, 1));}#publish-mermaid-7-1792402916865 text.actor>tspan{fill:black;stroke:none;}#publish-mermaid-7-1792402916865 .actor-line{stroke:#9370DB;}#publish-mermaid-7-1792402916865 .innerArc{stroke-width:1.5;stroke-dasharray:none;}#publish-mermaid-7-1792402916865 .messageLine0{stroke-width:1.5;stroke-dasharray:none;stroke:#333;}#publish-mermaid-7-1792402916865 .messageLine1{stroke-width:1.5;stroke-dasharray:2,2;stroke:#333;}#publish-mermaid-7-1792402916865 [id$=\”-arrowhead\”] path{fill:#333;stroke:#333;}#publish-mermaid-7-1792402916865 .sequenceNumber{fill:white;}#publish-mermaid-7-1792402916865 [id$=\”-sequencenumber\”]{fill:#333;}#publish-mermaid-7-1792402916865 [id$=\”-crosshead\”] path{fill:#333;stroke:#333;}#publish-mermaid-7-1792402916865 .messageText{fill:#333;stroke:none;}#publish-mermaid-7-1792402916865 .labelBox{stroke:#9370DB;fill:#ECECFF;filter:none;}#publish-mermaid-7-1792402916865 .labelText,#publish-mermaid-7-1792402916865 .labelText>tspan{fill:black;stroke:none;}#publish-mermaid-7-1792402916865 .loopText,#publish-mermaid-7-1792402916865 .loopText>tspan{fill:black;stroke:none;}#publish-mermaid-7-1792402916865 .sectionTitle,#publish-mermaid-7-1792402916865 .sectionTitle>tspan{fill:black;stroke:none;}#publish-mermaid-7-1792402916865 .loopLine{stroke-width:2px;stroke-dasharray:2,2;stroke:#9370DB;fill:#9370DB;}#publish-mermaid-7-1792402916865 .note{stroke:#aaaa33;fill:#fff5ad;}#publish-mermaid-7-1792402916865 .noteText,#publish-mermaid-7-1792402916865 .noteText>tspan{fill:black;stroke:none;font-weight:normal;}#publish-mermaid-7-1792402916865 .activation0{fill:#f4f4f4;stroke:#666;}#publish-mermaid-7-1792402916865 .activation1{fill:#f4f4f4;stroke:#666;}#publish-mermaid-7-1792402916865 .activation2{fill:#f4f4f4;stroke:#666;}#publish-mermaid-7-1792402916865 .actorPopupMenu{position:absolute;}#publish-mermaid-7-1792402916865 .actorPopupMenuPanel{position:absolute;fill:#ECECFF;box-shadow:0px 8px 16px 0px rgba(0,0,0,0.2);filter:drop-shadow(3px 5px 2px rgb(0 0 0 / 0.4));}#publish-mermaid-7-1792402916865 .actor-man circle,#publish-mermaid-7-1792402916865 line{fill:#ECECFF;stroke-width:2px;}#publish-mermaid-7-1792402916865 g rect.rect{filter:drop-shadow(1px 2px 2px rgba(185, 185, 185, 1));stroke:#9370DB;}#publish-mermaid-7-1792402916865 .node .neo-node{stroke:#9370DB;}#publish-mermaid-7-1792402916865 [data-look=\”neo\”].node rect,#publish-mermaid-7-1792402916865 [data-look=\”neo\”].cluster rect,#publish-mermaid-7-1792402916865 [data-look=\”neo\”].node polygon{stroke:#9370DB;filter:drop-shadow(1px 2px 2px rgba(185, 185, 185, 1));}#publish-mermaid-7-1792402916865 [data-look=\”neo\”].swimlane.cluster rect{filter:none;}#publish-mermaid-7-1792402916865 [data-look=\”neo\”].node path{stroke:#9370DB;stroke-width:1px;}#publish-mermaid-7-1792402916865 [data-look=\”neo\”].node .outer-path{filter:drop-shadow(1px 2px 2px rgba(185, 185, 185, 1));}#publish-mermaid-7-1792402916865 [data-look=\”neo\”].node .neo-line path{stroke:#9370DB;filter:none;}#publish-mermaid-7-1792402916865 [data-look=\”neo\”].node circle{stroke:#9370DB;filter:drop-shadow(1px 2px 2px rgba(185, 185, 185, 1));}#publish-mermaid-7-1792402916865 [data-look=\”neo\”].node circle .state-start{fill:#000000;}#publish-mermaid-7-1792402916865 [data-look=\”neo\”].icon-shape .icon{fill:#9370DB;filter:drop-shadow(1px 2px 2px rgba(185, 185, 185, 1));}#publish-mermaid-7-1792402916865 [data-look=\”neo\”].icon-shape .icon-neo path{stroke:#9370DB;filter:drop-shadow(1px 2px 2px rgba(185, 185, 185, 1));}#publish-mermaid-7-1792402916865 :root{–mermaid-font-family:\”trebuchet ms\”,verdana,arial,sans-serif;}Old_Master 宕机数据库连接请求连接失败心跳检测失败提升为 New_Master确认切换复制源到 New_Master确认(通过 DNS/代理) 指向 New_Master数据库连接请求响应
该时序图详细描述了从应用程序连接失败,到 Orchestrator 检测、决策、执行,再到应用程序重新连接到新主节点的完整自动化故障转移事件流。
七、最佳实践与注意事项:确保稳定运行
为了最大化 Orchestrator 的效用并确保其自身的高可用,以下是一些重要的最佳实践和注意事项。
7.1 配置高可用 Orchestrator 本身
虽然 Orchestrator 本身是单点进程,但可以通过运行多个 Orchestrator 实例并配置它们指向同一个元数据存储来实现高可用。这些实例可以部署在不同的物理或虚拟机上,当一个实例故障时,其他实例可以接管工作。
7.2 监控与告警集成
应将 Orchestrator 的运行状态纳入整体监控系统。可以监控其 API 健康检查端点,并设置告警,以便在 Orchestrator 自身出现故障或频繁执行故障转移时及时收到通知。
7.3 定期演练
建议定期(如每季度)进行故障转移演练,模拟主节点故障,以验证 Orchestrator 的配置和集群的恢复能力,确保在真实故障发生时能够顺利应对。



