《企业级K8s容器化部署方案：Jenkins+Harbor+Helm+Gitee+Jenkinsfile自动化实践》

企业级 K8s 容器化部署方案：Jenkins + Harbor + Helm/kubectl + Gitee + Jenkinsfile 自动化实践

本文档基于 jenkinsfileTest/cigroovy.jenkinsfile 及共享库 jenkinslib 编写，描述 Java 微服务从 Maven/Gradle 编译、Docker 镜像推送 Harbor，到 K8s 集群部署（kubectl 动态模板 / Helm Chart）、初始化与服务更新、版本回滚的完整 CI/CD 流程。

一、方案架构

1.1 组件说明

1.2 两条 CD 路径

构建阶段（CheckOut → Build → 镜像推送 Harbor）相同，差异在部署阶段。

1.3 流水线与共享库

二、发布模式（publishMode）

#mermaid-svg-eBjFu7mng7jKFcdT{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-eBjFu7mng7jKFcdT .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-eBjFu7mng7jKFcdT .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-eBjFu7mng7jKFcdT .error-icon{fill:#552222;}#mermaid-svg-eBjFu7mng7jKFcdT .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-eBjFu7mng7jKFcdT .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-eBjFu7mng7jKFcdT .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-eBjFu7mng7jKFcdT .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-eBjFu7mng7jKFcdT .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-eBjFu7mng7jKFcdT .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-eBjFu7mng7jKFcdT .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-eBjFu7mng7jKFcdT .marker{fill:#333333;stroke:#333333;}#mermaid-svg-eBjFu7mng7jKFcdT .marker.cross{stroke:#333333;}#mermaid-svg-eBjFu7mng7jKFcdT svg{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-eBjFu7mng7jKFcdT p{margin:0;}#mermaid-svg-eBjFu7mng7jKFcdT .label{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;color:#333;}#mermaid-svg-eBjFu7mng7jKFcdT .cluster-label text{fill:#333;}#mermaid-svg-eBjFu7mng7jKFcdT .cluster-label span{color:#333;}#mermaid-svg-eBjFu7mng7jKFcdT .cluster-label span p{background-color:transparent;}#mermaid-svg-eBjFu7mng7jKFcdT .label text,#mermaid-svg-eBjFu7mng7jKFcdT span{fill:#333;color:#333;}#mermaid-svg-eBjFu7mng7jKFcdT .node rect,#mermaid-svg-eBjFu7mng7jKFcdT .node circle,#mermaid-svg-eBjFu7mng7jKFcdT .node ellipse,#mermaid-svg-eBjFu7mng7jKFcdT .node polygon,#mermaid-svg-eBjFu7mng7jKFcdT .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-eBjFu7mng7jKFcdT .rough-node .label text,#mermaid-svg-eBjFu7mng7jKFcdT .node .label text,#mermaid-svg-eBjFu7mng7jKFcdT .image-shape .label,#mermaid-svg-eBjFu7mng7jKFcdT .icon-shape .label{text-anchor:middle;}#mermaid-svg-eBjFu7mng7jKFcdT .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-eBjFu7mng7jKFcdT .rough-node .label,#mermaid-svg-eBjFu7mng7jKFcdT .node .label,#mermaid-svg-eBjFu7mng7jKFcdT .image-shape .label,#mermaid-svg-eBjFu7mng7jKFcdT .icon-shape .label{text-align:center;}#mermaid-svg-eBjFu7mng7jKFcdT .node.clickable{cursor:pointer;}#mermaid-svg-eBjFu7mng7jKFcdT .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-eBjFu7mng7jKFcdT .arrowheadPath{fill:#333333;}#mermaid-svg-eBjFu7mng7jKFcdT .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-eBjFu7mng7jKFcdT .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-eBjFu7mng7jKFcdT .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-eBjFu7mng7jKFcdT .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-eBjFu7mng7jKFcdT .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-eBjFu7mng7jKFcdT .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-eBjFu7mng7jKFcdT .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-eBjFu7mng7jKFcdT .cluster text{fill:#333;}#mermaid-svg-eBjFu7mng7jKFcdT .cluster span{color:#333;}#mermaid-svg-eBjFu7mng7jKFcdT div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-eBjFu7mng7jKFcdT .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-eBjFu7mng7jKFcdT rect.text{fill:none;stroke-width:0;}#mermaid-svg-eBjFu7mng7jKFcdT .icon-shape,#mermaid-svg-eBjFu7mng7jKFcdT .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-eBjFu7mng7jKFcdT .icon-shape p,#mermaid-svg-eBjFu7mng7jKFcdT .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-eBjFu7mng7jKFcdT .icon-shape .label rect,#mermaid-svg-eBjFu7mng7jKFcdT .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-eBjFu7mng7jKFcdT .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-eBjFu7mng7jKFcdT .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-eBjFu7mng7jKFcdT :root{–mermaid-font-family:\”trebuchet ms\”,verdana,arial,sans-serif;}

回滚

其他

仅构建

自动/手动

kubectl

helm

publishMode

回滚阶段

CheckOut → Build

镜像构建并推送 Harbor

publishMode

结束

deployMethod

生成部署模板

跳过模板

发布到K8s

post 邮件

三、时序图

3.1 kubectl 自动发布

k8sdeploytemplateHarborbuildJenkins Podk8sdeploytemplateHarborbuildJenkins Pod#mermaid-svg-enYvI2GvxJmeu7KN{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-enYvI2GvxJmeu7KN .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-enYvI2GvxJmeu7KN .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-enYvI2GvxJmeu7KN .error-icon{fill:#552222;}#mermaid-svg-enYvI2GvxJmeu7KN .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-enYvI2GvxJmeu7KN .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-enYvI2GvxJmeu7KN .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-enYvI2GvxJmeu7KN .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-enYvI2GvxJmeu7KN .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-enYvI2GvxJmeu7KN .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-enYvI2GvxJmeu7KN .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-enYvI2GvxJmeu7KN .marker{fill:#333333;stroke:#333333;}#mermaid-svg-enYvI2GvxJmeu7KN .marker.cross{stroke:#333333;}#mermaid-svg-enYvI2GvxJmeu7KN svg{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-enYvI2GvxJmeu7KN p{margin:0;}#mermaid-svg-enYvI2GvxJmeu7KN .actor{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-enYvI2GvxJmeu7KN text.actor>tspan{fill:black;stroke:none;}#mermaid-svg-enYvI2GvxJmeu7KN .actor-line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);}#mermaid-svg-enYvI2GvxJmeu7KN .innerArc{stroke-width:1.5;stroke-dasharray:none;}#mermaid-svg-enYvI2GvxJmeu7KN .messageLine0{stroke-width:1.5;stroke-dasharray:none;stroke:#333;}#mermaid-svg-enYvI2GvxJmeu7KN .messageLine1{stroke-width:1.5;stroke-dasharray:2,2;stroke:#333;}#mermaid-svg-enYvI2GvxJmeu7KN #arrowhead path{fill:#333;stroke:#333;}#mermaid-svg-enYvI2GvxJmeu7KN .sequenceNumber{fill:white;}#mermaid-svg-enYvI2GvxJmeu7KN #sequencenumber{fill:#333;}#mermaid-svg-enYvI2GvxJmeu7KN #crosshead path{fill:#333;stroke:#333;}#mermaid-svg-enYvI2GvxJmeu7KN .messageText{fill:#333;stroke:none;}#mermaid-svg-enYvI2GvxJmeu7KN .labelBox{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-enYvI2GvxJmeu7KN .labelText,#mermaid-svg-enYvI2GvxJmeu7KN .labelText>tspan{fill:black;stroke:none;}#mermaid-svg-enYvI2GvxJmeu7KN .loopText,#mermaid-svg-enYvI2GvxJmeu7KN .loopText>tspan{fill:black;stroke:none;}#mermaid-svg-enYvI2GvxJmeu7KN .loopLine{stroke-width:2px;stroke-dasharray:2,2;stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);}#mermaid-svg-enYvI2GvxJmeu7KN .note{stroke:#aaaa33;fill:#fff5ad;}#mermaid-svg-enYvI2GvxJmeu7KN .noteText,#mermaid-svg-enYvI2GvxJmeu7KN .noteText>tspan{fill:black;stroke:none;}#mermaid-svg-enYvI2GvxJmeu7KN .activation0{fill:#f4f4f4;stroke:#666;}#mermaid-svg-enYvI2GvxJmeu7KN .activation1{fill:#f4f4f4;stroke:#666;}#mermaid-svg-enYvI2GvxJmeu7KN .activation2{fill:#f4f4f4;stroke:#666;}#mermaid-svg-enYvI2GvxJmeu7KN .actorPopupMenu{position:absolute;}#mermaid-svg-enYvI2GvxJmeu7KN .actorPopupMenuPanel{position:absolute;fill:#ECECFF;box-shadow:0px 8px 16px 0px rgba(0,0,0,0.2);filter:drop-shadow(3px 5px 2px rgb(0 0 0 / 0.4));}#mermaid-svg-enYvI2GvxJmeu7KN .actor-man line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-enYvI2GvxJmeu7KN .actor-man circle,#mermaid-svg-enYvI2GvxJmeu7KN line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;stroke-width:2px;}#mermaid-svg-enYvI2GvxJmeu7KN :root{–mermaid-font-family:\”trebuchet ms\”,verdana,arial,sans-serif;}compile(mvn/gradle)1BuildImage + push2generateWorkloadTemplate + Service(init时)3Deploy(kubectl apply + rollout)4

3.2 Helm 发布

• 跳过「生成部署模板」
• PackageAndPushChart → helm upgrade –install –atomic –wait
• 注意：Helm 模式下 Jenkins 侧 deployConfigType/Template 参数不生效，由仓库 Chart 决定

3.3 初始化部署 vs 服务更新

四、部署配置

4.1 部署配置类型（deployConfigType）

4.2 部署配置模板（deployConfigTemplate）

占位符：{{PRONAME}} {{FULL_IMAGE_NAME}} {{NAMESPACE}} {{REPLICAS}} {{CONTAINER_PORT}} 及 CPU/Memory、Apollo、SkyWalking 等。

五、环境准备

5.1 Jenkins K8s 云

Slave Pod 容器：mvn、docker（挂载 docker.sock）、kubectl、helm。

5.2 Harbor

镜像：{HARBOR_URL}/{PROJECT_GROUP}/{imageName}:{imageTag}_{Tenv}_{BUILD_TIME}
示例：harbor.jdicity.local/registry/order-service:v1.0_prod_20250618_143022

5.3 K8s 凭据（config.K8S_CLUSTER_CREDENTIALS）

可用 K8S_CREDENTIAL_ID 覆盖。

5.4 集群前置条件

• Namespace 已创建
• registry-secret imagePullSecrets
• SkyWalking 模板：OAP、Apollo Meta 集群内可达
• 应用暴露 /actuator/health/*（skywalking 模板探针）

5.5 Helm 额外要求

• 仓库根目录 helm/Chart.yaml
• Harbor Chart 仓库已启用

六、Jenkins 参数

6.1 基础参数

6.2 部署参数

6.3 SkyWalking 模板资源参数

containerCpuLimits、containerMemoryLimits、containerCpuRequests、containerMemoryRequests、APOLLO_*、skywalkingAgentImage、skywalkingCollector、JAVA_OPTS、APOLLO_OPTS

6.4 环境变量

七、流水线阶段

八、Helm 在 CI/CD 中的作用

kubectl vs Helm 选型：

• 动态 SkyWalking YAML、多工作负载类型 → kubectl
• 稳定 Chart、Release 级管理 → Helm

九、共享库 API

pipeline.groovy

同前后端：checkoutCode、runPublish、selectRollbackVersion

k8sdeploy.groovy

build.groovy

build.compile(buildType, buildshell) // mvn / gradle 统一入口

十、典型场景

十一、排错

十二、快速检查清单

• K8s 云 kubernetes 已配置，Slave Pod 正常
• 凭据 HARBOR_ID、环境 kubeconfig 已创建
• publishMode、deployMethod 已配置
• kubectl 路径：SkyWalking OAP、registry-secret 就绪
• helm 路径：helm/Chart.yaml 存在
• SMTP、emailUser 已配置

附录：DevOps 三条路径对照