欢迎光临
我们一直在努力

FLINK故障重启策略

Flink 重启策略(Restart Strategy)

1. 重启策略概述

重启策略决定作业失败时是否可以重启以及重启的间隔。

重要说明:

  • 未定义重启策略时,使用集群默认策略(通过flink-conf.yaml配置)
  • 作业级配置会覆盖集群默认策略
  • 未启用检查点时,默认采用"不重启"策略
  • 启用检查点且未配置策略时,默认使用FixedDelayRestartStrategy(Integer.MAX_VALUE, 10000)

2. 重启策略类型

2.1 策略类型总览

策略类型配置值适用场景检查点要求
固定延迟策略 fixed-delay 临时性故障 必须启用
故障率策略 failure-rate 周期性/突发性故障 必须启用
不重启策略 none 批处理/无状态任务 可选
回退策略 fallback 集群默认策略 可选

2.2 策略选择决策树

#mermaid-svg-pNikvG4vnPJsWsUI{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-pNikvG4vnPJsWsUI .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-pNikvG4vnPJsWsUI .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-pNikvG4vnPJsWsUI .error-icon{fill:#552222;}#mermaid-svg-pNikvG4vnPJsWsUI .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-pNikvG4vnPJsWsUI .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-pNikvG4vnPJsWsUI .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-pNikvG4vnPJsWsUI .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-pNikvG4vnPJsWsUI .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-pNikvG4vnPJsWsUI .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-pNikvG4vnPJsWsUI .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-pNikvG4vnPJsWsUI .marker{fill:#333333;stroke:#333333;}#mermaid-svg-pNikvG4vnPJsWsUI .marker.cross{stroke:#333333;}#mermaid-svg-pNikvG4vnPJsWsUI svg{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-pNikvG4vnPJsWsUI p{margin:0;}#mermaid-svg-pNikvG4vnPJsWsUI .label{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;color:#333;}#mermaid-svg-pNikvG4vnPJsWsUI .cluster-label text{fill:#333;}#mermaid-svg-pNikvG4vnPJsWsUI .cluster-label span{color:#333;}#mermaid-svg-pNikvG4vnPJsWsUI .cluster-label span p{background-color:transparent;}#mermaid-svg-pNikvG4vnPJsWsUI .label text,#mermaid-svg-pNikvG4vnPJsWsUI span{fill:#333;color:#333;}#mermaid-svg-pNikvG4vnPJsWsUI .node rect,#mermaid-svg-pNikvG4vnPJsWsUI .node circle,#mermaid-svg-pNikvG4vnPJsWsUI .node ellipse,#mermaid-svg-pNikvG4vnPJsWsUI .node polygon,#mermaid-svg-pNikvG4vnPJsWsUI .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-pNikvG4vnPJsWsUI .rough-node .label text,#mermaid-svg-pNikvG4vnPJsWsUI .node .label text,#mermaid-svg-pNikvG4vnPJsWsUI .image-shape .label,#mermaid-svg-pNikvG4vnPJsWsUI .icon-shape .label{text-anchor:middle;}#mermaid-svg-pNikvG4vnPJsWsUI .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-pNikvG4vnPJsWsUI .rough-node .label,#mermaid-svg-pNikvG4vnPJsWsUI .node .label,#mermaid-svg-pNikvG4vnPJsWsUI .image-shape .label,#mermaid-svg-pNikvG4vnPJsWsUI .icon-shape .label{text-align:center;}#mermaid-svg-pNikvG4vnPJsWsUI .node.clickable{cursor:pointer;}#mermaid-svg-pNikvG4vnPJsWsUI .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-pNikvG4vnPJsWsUI .arrowheadPath{fill:#333333;}#mermaid-svg-pNikvG4vnPJsWsUI .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-pNikvG4vnPJsWsUI .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-pNikvG4vnPJsWsUI .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-pNikvG4vnPJsWsUI .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-pNikvG4vnPJsWsUI .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-pNikvG4vnPJsWsUI .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-pNikvG4vnPJsWsUI .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-pNikvG4vnPJsWsUI .cluster text{fill:#333;}#mermaid-svg-pNikvG4vnPJsWsUI .cluster span{color:#333;}#mermaid-svg-pNikvG4vnPJsWsUI div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-pNikvG4vnPJsWsUI .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-pNikvG4vnPJsWsUI rect.text{fill:none;stroke-width:0;}#mermaid-svg-pNikvG4vnPJsWsUI .icon-shape,#mermaid-svg-pNikvG4vnPJsWsUI .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-pNikvG4vnPJsWsUI .icon-shape p,#mermaid-svg-pNikvG4vnPJsWsUI .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-pNikvG4vnPJsWsUI .icon-shape rect,#mermaid-svg-pNikvG4vnPJsWsUI .image-shape rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-pNikvG4vnPJsWsUI .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-pNikvG4vnPJsWsUI .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-pNikvG4vnPJsWsUI :root{–mermaid-font-family:\”trebuchet ms\”,verdana,arial,sans-serif;}

是

否

临时性

周期性/突发性

是否需要状态恢复?

启用检查点

选择no-restart或fallback

故障类型?

fixed-delay

failure-rate

3. 固定延迟重启策略(Fixed Delay Restart Strategy)

3.1 核心原理

  • 固定间隔重启:每次失败后等待固定时间再尝试
  • 最大尝试次数:达到阈值后停止重启
  • 状态恢复:每次重启从最新检查点恢复

3.2 配置参数

3.2.1 代码配置

// 基础配置
env.setRestartStrategy(RestartStrategies.fixedDelayRestart(
3, // attempts
Time.seconds(30) // delay
));

// 指数退避配置
env.setRestartStrategy(RestartStrategies.fixedDelayRestart(
5, // attempts
Time.seconds(10), // initial-delay
Time.seconds(300), // max-delay
2.0 // multiplier
));

3.2.2 配置文件

restart-strategy: fixed–delay
restart-strategy.fixed-delay.attempts: 3
restart-strategy.fixed-delay.delay: 30 s
restart-strategy.fixed-delay.exponential-delay.initial-delay: 10 s
restart-strategy.fixed-delay.exponential-delay.max-delay: 5 min
restart-strategy.fixed-delay.exponential-delay.multiplier: 2.0

3.3 触发条件

条件类型具体表现日志特征
任务执行失败 用户代码异常、反序列化错误 TaskFailureException
检查点失败 超时、存储错误 Checkpoint expired
资源管理失败 TM失联、Slot分配失败 Heartbeat timed out

3.4 生产建议

  • 小状态作业:attempts=3-5, delay=30s, 启用指数退避
  • 大状态作业:attempts=5-10, delay=60s, 启用指数退避
  • 关键作业:attempts=5, delay=30s, 人工审批流程

4. 故障率重启策略(Failure Rate Restart Strategy)

4.1 核心原理

  • 时间窗口统计:在failure-rate-interval内统计失败次数
  • 阈值触发:超过max-failures-per-interval后触发重启
  • 冷却期:重启后清空窗口统计

4.2 配置参数

4.2.1 代码配置

env.setRestartStrategy(RestartStrategies.failureRateRestart(
3, // maxFailuresPerInterval
Time.minutes(5), // failureRateInterval
Time.seconds(30) // delay
));

4.2.2 配置文件

restart-strategy: failure–rate
restart-strategy.failure-rate.max-failures-per-interval: 3
restart-strategy.failure-rate.failure-rate-interval: 5 min
restart-strategy.failure-rate.delay: 30 s

4.3 触发场景

场景窗口大小最大失败次数重启间隔
外部系统周期性不稳定 5-10分钟 2-3次 30s
网络风暴 1-2分钟 3-5次 10s
资源竞争 3-5分钟 2-3次 60s

4.4 生产建议

  • 窗口大小应大于检查点间隔
  • 避免窗口过短(<1分钟)或阈值过低(<2次)
  • 监控failureRate指标

5. 不重启策略(No Restart Strategy)

5.1 适用场景

  • 批处理作业
  • 无状态任务
  • 需要人工干预的严重故障

5.2 配置方式

restart-strategy: none

5.3 检查点关联性

检查点状态默认行为风险提示
未启用 不重启 安全
已启用 不重启 状态丢失风险

6. 回退策略(Fallback Restart Strategy)

6.1 行为说明

  • 使用集群默认策略
  • 通常用于批处理作业

6.2 配置方式

restart-strategy: fallback

7. 生产环境最佳实践

7.1 检查点协同配置

// 检查点间隔应小于重启间隔
env.enableCheckpointing(20000); // 20秒

// 检查点超时时间应大于重启间隔
env.getCheckpointConfig().setCheckpointTimeout(300000); // 5分钟

// 允许并发检查点
env.getCheckpointConfig().setMaxConcurrentCheckpoints(2);

7.2 监控指标

指标说明告警阈值
numberOfRestarts 重启次数 > attempts
failureRate 失败率 > max-failures-per-interval
lastCheckpointDuration 检查点持续时间 > 检查点间隔

7.3 故障排查

  • 重启间隔不生效:检查检查点是否启用
  • 状态丢失:检查state.checkpoints.dir权限
  • 重启风暴:启用指数退避

8. 附录:配置优先级

  • 作业级配置(最高优先级):env.setRestartStrategy(...)
  • 集群级配置:# flink-conf.yaml
    restart-strategy: fixed–delay
  • 默认策略(最低优先级):
    • 启用检查点:FixedDelayRestartStrategy(Integer.MAX_VALUE, 10000)
    • 未启用检查点:NoRestartStrategy

  • 最后更新:2025-12-26 参考文档:Flink 1.12 官方文档

    赞(0)
    未经允许不得转载:171主机测评 » FLINK故障重启策略
    分享到: 更多 (0)

    评论 抢沙发

    • 昵称 (必填)
    • 邮箱 (必填)
    • 网址