Kubernetes 集群实战部署指南：从零搭建生产级K8s环境

📋 目录

• 第一章：准备工作
  • 1.1 环境规划
  • 1.2 系统要求
  • 1.3 关键决策
• 第二章：系统初始化
  • 2.1 主机名与hosts配置
  • 2.2 关闭防火墙和SELinux
  • 2.3 内核参数优化
  • 2.4 时间同步配置
• 第三章：安装容器运行时
• 第四章：部署K8s集群
• 第五章：网络与存储配置
• 第六章：Ingress与监控
• 第七章：常见问题与排查

第一章：准备工作

1.1 环境规划

节点配置（4节点示例）：

网络规划：

Pod网络（Calico）： 10.244.0.0/16
Service网络： 10.96.0.0/12
API Server端口： 6443
NodePort范围： 30000-32767

集群架构图：

#mermaid-svg-igNjbGaUm82qVspR{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-igNjbGaUm82qVspR .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-igNjbGaUm82qVspR .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-igNjbGaUm82qVspR .error-icon{fill:#552222;}#mermaid-svg-igNjbGaUm82qVspR .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-igNjbGaUm82qVspR .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-igNjbGaUm82qVspR .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-igNjbGaUm82qVspR .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-igNjbGaUm82qVspR .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-igNjbGaUm82qVspR .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-igNjbGaUm82qVspR .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-igNjbGaUm82qVspR .marker{fill:#333333;stroke:#333333;}#mermaid-svg-igNjbGaUm82qVspR .marker.cross{stroke:#333333;}#mermaid-svg-igNjbGaUm82qVspR svg{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-igNjbGaUm82qVspR p{margin:0;}#mermaid-svg-igNjbGaUm82qVspR .label{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;color:#333;}#mermaid-svg-igNjbGaUm82qVspR .cluster-label text{fill:#333;}#mermaid-svg-igNjbGaUm82qVspR .cluster-label span{color:#333;}#mermaid-svg-igNjbGaUm82qVspR .cluster-label span p{background-color:transparent;}#mermaid-svg-igNjbGaUm82qVspR .label text,#mermaid-svg-igNjbGaUm82qVspR span{fill:#333;color:#333;}#mermaid-svg-igNjbGaUm82qVspR .node rect,#mermaid-svg-igNjbGaUm82qVspR .node circle,#mermaid-svg-igNjbGaUm82qVspR .node ellipse,#mermaid-svg-igNjbGaUm82qVspR .node polygon,#mermaid-svg-igNjbGaUm82qVspR .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-igNjbGaUm82qVspR .rough-node .label text,#mermaid-svg-igNjbGaUm82qVspR .node .label text,#mermaid-svg-igNjbGaUm82qVspR .image-shape .label,#mermaid-svg-igNjbGaUm82qVspR .icon-shape .label{text-anchor:middle;}#mermaid-svg-igNjbGaUm82qVspR .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-igNjbGaUm82qVspR .rough-node .label,#mermaid-svg-igNjbGaUm82qVspR .node .label,#mermaid-svg-igNjbGaUm82qVspR .image-shape .label,#mermaid-svg-igNjbGaUm82qVspR .icon-shape .label{text-align:center;}#mermaid-svg-igNjbGaUm82qVspR .node.clickable{cursor:pointer;}#mermaid-svg-igNjbGaUm82qVspR .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-igNjbGaUm82qVspR .arrowheadPath{fill:#333333;}#mermaid-svg-igNjbGaUm82qVspR .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-igNjbGaUm82qVspR .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-igNjbGaUm82qVspR .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-igNjbGaUm82qVspR .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-igNjbGaUm82qVspR .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-igNjbGaUm82qVspR .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-igNjbGaUm82qVspR .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-igNjbGaUm82qVspR .cluster text{fill:#333;}#mermaid-svg-igNjbGaUm82qVspR .cluster span{color:#333;}#mermaid-svg-igNjbGaUm82qVspR div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-igNjbGaUm82qVspR .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-igNjbGaUm82qVspR rect.text{fill:none;stroke-width:0;}#mermaid-svg-igNjbGaUm82qVspR .icon-shape,#mermaid-svg-igNjbGaUm82qVspR .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-igNjbGaUm82qVspR .icon-shape p,#mermaid-svg-igNjbGaUm82qVspR .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-igNjbGaUm82qVspR .icon-shape rect,#mermaid-svg-igNjbGaUm82qVspR .image-shape rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-igNjbGaUm82qVspR .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-igNjbGaUm82qVspR .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-igNjbGaUm82qVspR :root{–mermaid-font-family:\”trebuchet ms\”,verdana,arial,sans-serif;}

存储层

网络层

工作节点

控制平面节点 (192.168.1.10)

管理

管理

管理

挂载

挂载

挂载

NodePort/Ingress

k8s-master2核4G

API Server :6443

etcd

kube-scheduler

kube-controller-manager

k8s-node1192.168.1.112核8G

k8s-node2192.168.1.122核8G

k8s-node3192.168.1.132核4GIngress入口

Calico VXLAN10.244.0.0/16

Service网络10.96.0.0/12

NFS动态存储192.168.1.10:/data/nfs

外部流量

1.2 系统要求

硬件要求：

• Master节点：≥2核CPU，≥2GB内存，≥20GB磁盘
• Worker节点：≥2核CPU，≥4GB内存，≥40GB磁盘
• 所有节点需要内网互通

软件要求：

• 操作系统：CentOS Stream 9 / Ubuntu 20.04+ / Debian 11+
• 内核版本：≥4.19（建议5.x）
• 容器运行时：containerd 1.6+
• 网络插件：Calico 3.27

云服务器配置：

• ✅ 安全组开放端口：6443（API Server）、10250（kubelet）、2379-2380（etcd）、4789（VXLAN）
• ✅ 关闭云防火墙或配置白名单
• ⚠️ 重要：云环境不支持IPIP协议，必须使用VXLAN模式

1.3 关键决策

决策1：网络插件选择

为什么云环境必须用VXLAN？

• 云厂商安全组不支持IPIP协议（协议号4）
• VXLAN使用UDP 4789端口，可以通过安全组规则放行
• 实测：阿里云/腾讯云/华为云均不支持IPIP

决策2：镜像源选择

由于网络原因，国内环境需要配置镜像加速：

# Kubernetes镜像源
registry.aliyuncs.com/google_containers

# containerd镜像加速
https://mirror.ccs.tencentyun.com
https://dockerproxy.com

决策3：存储方案

本文选择NFS动态存储，适合快速搭建测试环境。

第二章：系统初始化

⚠️ 以下操作需要在所有节点执行（除非特别说明）

2.1 主机名与hosts配置

设置主机名（每个节点执行不同命令）：

# Master节点
hostnamectl set-hostname k8s-master

# Worker节点1
hostnamectl set-hostname k8s-node1

# Worker节点2
hostnamectl set-hostname k8s-node2

# Worker节点3
hostnamectl set-hostname k8s-node3

配置hosts文件（所有节点）：

cat >> /etc/hosts << EOF
192.168.1.10 k8s-master
192.168.1.11 k8s-node1
192.168.1.12 k8s-node2
192.168.1.13 k8s-node3
EOF

验证：

# 测试主机名解析
ping -c 2 k8s-master
ping -c 2 k8s-node1

# 检查主机名
hostname

2.2 关闭防火墙和SELinux

关闭防火墙：

# 停止并禁用firewalld
systemctl stop firewalld
systemctl disable firewalld

# 验证状态
systemctl status firewalld

关闭SELinux：

# 临时关闭
setenforce 0

# 永久关闭
sed -i 's/^SELINUX=enforcing$/SELINUX=disabled/' /etc/selinux/config

# 验证
getenforce # 应显示 Permissive 或 Disabled

关闭swap：

# 临时关闭
swapoff -a

# 永久关闭（注释swap行）
sed -i '/ swap / s/^\\(.*\\)$/#\\1/g' /etc/fstab

# 验证
free -h # swap行应为0

2.3 内核参数优化

加载必需的内核模块：

cat > /etc/modules-load.d/k8s.conf << EOF
overlay
br_netfilter
EOF

# 立即加载
modprobe overlay
modprobe br_netfilter

# 验证
lsmod | grep -E "overlay|br_netfilter"

配置内核参数：

cat > /etc/sysctl.d/k8s.conf << EOF
# 启用IP转发
net.ipv4.ip_forward = 1

# 桥接流量经过iptables
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 1

# 禁用IPv6（可选）
net.ipv6.conf.all.disable_ipv6 = 0
net.ipv6.conf.default.disable_ipv6 = 0

# 优化网络性能
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_fin_timeout = 30
net.core.somaxconn = 32768
EOF

# 应用配置
sysctl –system

# 验证关键参数
sysctl net.ipv4.ip_forward
sysctl net.bridge.bridge-nf-call-iptables

2.4 时间同步配置

安装并启动chronyd：

# 安装时间同步工具
yum install -y chrony

# 启动服务
systemctl start chronyd
systemctl enable chronyd

# 验证时间同步状态
chronyc sources -v
timedatectl status

配置时区：

# 设置为中国时区
timedatectl set-timezone Asia/Shanghai

# 验证
date

验证所有节点时间一致：

# 在所有节点执行
date "+%Y-%m-%d %H:%M:%S"

# 时间差应小于1秒

✅ 第一阶段完成检查清单：

• 所有节点主机名已设置且可互相解析
• 防火墙、SELinux、swap已关闭
• 内核模块已加载（overlay、br_netfilter）
• 内核参数已生效（ip_forward=1）
• 时间同步正常，所有节点时间一致

第三章：安装容器运行时

Kubernetes从1.24版本开始移除了对Docker的内置支持，推荐使用containerd作为容器运行时。

3.1 安装containerd

配置YUM源（使用阿里云镜像）：

# 安装依赖
yum install -y yum-utils device-mapper-persistent-data lvm2

# 添加Docker CE仓库（包含containerd）
yum-config-manager –add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo

# 查看可用版本
yum list containerd.io –showduplicates | sort -r

安装containerd：

# 安装指定版本（推荐1.6.x）
yum install -y containerd.io-1.6.32

# 启动服务
systemctl start containerd
systemctl enable containerd

# 验证安装
containerd –version
systemctl status containerd

3.2 配置containerd

生成默认配置文件：

# 创建配置目录
mkdir -p /etc/containerd

# 生成默认配置
containerd config default > /etc/containerd/config.toml

修改配置文件（关键配置）：

# 1. 启用SystemdCgroup
sed -i 's/SystemdCgroup = false/SystemdCgroup = true/' /etc/containerd/config.toml

# 2. 配置镜像加速（使用腾讯云镜像）
sed -i '/\\[plugins."io.containerd.grpc.v1.cri".registry.mirrors\\]/a\\ [plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"]\\n endpoint = ["https://mirror.ccs.tencentyun.com", "https://dockerproxy.com"]' /etc/containerd/config.toml

# 3. 配置sandbox镜像（使用阿里云镜像）
sed -i 's|registry.k8s.io/pause:3.8|registry.aliyuncs.com/google_containers/pause:3.9|' /etc/containerd/config.toml

重启containerd：

systemctl restart containerd

# 验证配置
systemctl status containerd
ctr version

3.3 验证安装

测试镜像拉取：

# 拉取测试镜像
ctr image pull docker.io/library/busybox:latest

# 查看镜像列表
ctr image ls | grep busybox

# 清理测试镜像
ctr image rm docker.io/library/busybox:latest

检查配置：

# 验证SystemdCgroup配置
grep "SystemdCgroup = true" /etc/containerd/config.toml

# 验证镜像加速配置
grep -A 2 "registry.mirrors" /etc/containerd/config.toml

第四章：部署K8s集群

4.1 安装kubeadm、kubelet、kubectl

配置Kubernetes YUM源：

cat > /etc/yum.repos.d/kubernetes.repo << EOF
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

安装指定版本（所有节点）：

# 查看可用版本
yum list kubeadm –showduplicates | sort -r

# 安装1.28.2版本
yum install -y kubelet-1.28.2 kubeadm-1.28.2 kubectl-1.28.2

# 启动kubelet（此时会失败，正常现象）
systemctl enable kubelet
systemctl start kubelet

# 验证安装
kubeadm version
kubectl version –client

4.2 初始化Master节点

⚠️ 以下操作仅在Master节点执行

创建初始化配置文件：

cat > kubeadm-config.yaml << EOF
apiVersion: kubeadm.k8s.io/v1beta3
kind: ClusterConfiguration
kubernetesVersion: v1.28.2
controlPlaneEndpoint: "192.168.1.10:6443"
imageRepository: registry.aliyuncs.com/google_containers
networking:
podSubnet: "10.244.0.0/16"
serviceSubnet: "10.96.0.0/12"
—
apiVersion: kubeadm.k8s.io/v1beta3
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: "192.168.1.10"
bindPort: 6443
nodeRegistration:
criSocket: unix:///var/run/containerd/containerd.sock
taints:
– effect: NoSchedule
key: node-role.kubernetes.io/control-plane
—
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
cgroupDriver: systemd
EOF

执行初始化：

# 预拉取镜像（可选，加快初始化速度）
kubeadm config images pull –config kubeadm-config.yaml

# 初始化集群
kubeadm init –config kubeadm-config.yaml

# 初始化成功后会输出类似以下内容：
# Your Kubernetes control-plane has initialized successfully!
#
# To start using your cluster, you need to run the following as a regular user:
# mkdir -p $HOME/.kube
# sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
# sudo chown $(id -u):$(id -g) $HOME/.kube/config
#
# Then you can join any number of worker nodes by running the following on each as root:
# kubeadm join 192.168.1.10:6443 –token <token> \\
# –discovery-token-ca-cert-hash sha256:<hash>

⚠️ 重要：保存输出中的kubeadm join命令，后续Worker节点加入集群时需要使用。

配置kubectl：

# 配置kubectl访问集群
mkdir -p $HOME/.kube
cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
chown $(id -u):$(id -g) $HOME/.kube/config

# 验证集群状态
kubectl get nodes
kubectl get pods -A

此时Master节点状态为NotReady，因为还未安装网络插件。

4.3 Worker节点加入集群

⚠️ 以下操作在所有Worker节点执行

使用Master节点初始化时输出的kubeadm join命令：

# 示例命令（实际token和hash请使用你的输出）
kubeadm join 192.168.1.10:6443 –token abcdef.0123456789abcdef \\
–discovery-token-ca-cert-hash sha256:1234567890abcdef1234567890abcdef1234567890abcdef1234567890abcdef

如果token过期（24小时后失效）：

# 在Master节点生成新token
kubeadm token create –print-join-command

验证节点加入（在Master节点执行）：

# 查看节点状态（此时都是NotReady，正常）
kubectl get nodes

# 输出示例：
# NAME STATUS ROLES AGE VERSION
# k8s-master NotReady control-plane 5m v1.28.2
# k8s-node1 NotReady <none> 2m v1.28.2
# k8s-node2 NotReady <none> 2m v1.28.2
# k8s-node3 NotReady <none> 2m v1.28.2

4.4 部署流程图

#mermaid-svg-JaIbm51npcUHXlUS{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-JaIbm51npcUHXlUS .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-JaIbm51npcUHXlUS .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-JaIbm51npcUHXlUS .error-icon{fill:#552222;}#mermaid-svg-JaIbm51npcUHXlUS .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-JaIbm51npcUHXlUS .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-JaIbm51npcUHXlUS .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-JaIbm51npcUHXlUS .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-JaIbm51npcUHXlUS .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-JaIbm51npcUHXlUS .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-JaIbm51npcUHXlUS .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-JaIbm51npcUHXlUS .marker{fill:#333333;stroke:#333333;}#mermaid-svg-JaIbm51npcUHXlUS .marker.cross{stroke:#333333;}#mermaid-svg-JaIbm51npcUHXlUS svg{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-JaIbm51npcUHXlUS p{margin:0;}#mermaid-svg-JaIbm51npcUHXlUS .label{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;color:#333;}#mermaid-svg-JaIbm51npcUHXlUS .cluster-label text{fill:#333;}#mermaid-svg-JaIbm51npcUHXlUS .cluster-label span{color:#333;}#mermaid-svg-JaIbm51npcUHXlUS .cluster-label span p{background-color:transparent;}#mermaid-svg-JaIbm51npcUHXlUS .label text,#mermaid-svg-JaIbm51npcUHXlUS span{fill:#333;color:#333;}#mermaid-svg-JaIbm51npcUHXlUS .node rect,#mermaid-svg-JaIbm51npcUHXlUS .node circle,#mermaid-svg-JaIbm51npcUHXlUS .node ellipse,#mermaid-svg-JaIbm51npcUHXlUS .node polygon,#mermaid-svg-JaIbm51npcUHXlUS .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-JaIbm51npcUHXlUS .rough-node .label text,#mermaid-svg-JaIbm51npcUHXlUS .node .label text,#mermaid-svg-JaIbm51npcUHXlUS .image-shape .label,#mermaid-svg-JaIbm51npcUHXlUS .icon-shape .label{text-anchor:middle;}#mermaid-svg-JaIbm51npcUHXlUS .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-JaIbm51npcUHXlUS .rough-node .label,#mermaid-svg-JaIbm51npcUHXlUS .node .label,#mermaid-svg-JaIbm51npcUHXlUS .image-shape .label,#mermaid-svg-JaIbm51npcUHXlUS .icon-shape .label{text-align:center;}#mermaid-svg-JaIbm51npcUHXlUS .node.clickable{cursor:pointer;}#mermaid-svg-JaIbm51npcUHXlUS .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-JaIbm51npcUHXlUS .arrowheadPath{fill:#333333;}#mermaid-svg-JaIbm51npcUHXlUS .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-JaIbm51npcUHXlUS .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-JaIbm51npcUHXlUS .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-JaIbm51npcUHXlUS .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-JaIbm51npcUHXlUS .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-JaIbm51npcUHXlUS .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-JaIbm51npcUHXlUS .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-JaIbm51npcUHXlUS .cluster text{fill:#333;}#mermaid-svg-JaIbm51npcUHXlUS .cluster span{color:#333;}#mermaid-svg-JaIbm51npcUHXlUS div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-JaIbm51npcUHXlUS .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-JaIbm51npcUHXlUS rect.text{fill:none;stroke-width:0;}#mermaid-svg-JaIbm51npcUHXlUS .icon-shape,#mermaid-svg-JaIbm51npcUHXlUS .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-JaIbm51npcUHXlUS .icon-shape p,#mermaid-svg-JaIbm51npcUHXlUS .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-JaIbm51npcUHXlUS .icon-shape rect,#mermaid-svg-JaIbm51npcUHXlUS .image-shape rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-JaIbm51npcUHXlUS .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-JaIbm51npcUHXlUS .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-JaIbm51npcUHXlUS :root{–mermaid-font-family:\”trebuchet ms\”,verdana,arial,sans-serif;}

是

否

是

否

开始部署

安装kubeadm/kubelet/kubectl

是Master节点?

创建kubeadm-config.yaml

预拉取镜像

kubeadm init

配置kubectl

保存join命令

等待Master初始化完成

执行kubeadm join

检查节点状态

节点NotReady?

安装Calico网络插件

部署完成

✅ 第二阶段完成检查清单：

• containerd已安装并配置镜像加速
• kubeadm、kubelet、kubectl已安装（版本1.28.2）
• Master节点初始化成功
• kubectl可以访问集群
• 所有Worker节点已加入集群
• 所有节点状态为NotReady（等待网络插件）

下一步：第五章将部署Calico网络插件（VXLAN模式）和NFS存储，这是最关键的章节。

第五章：网络与存储配置

5.1 部署Calico网络插件（VXLAN模式）

⚠️ 关键：云环境必须使用VXLAN模式

5.1.1 为什么云环境不能用IPIP？

网络模式对比：

#mermaid-svg-68R9HXu2vNiU9KAz{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-68R9HXu2vNiU9KAz .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-68R9HXu2vNiU9KAz .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-68R9HXu2vNiU9KAz .error-icon{fill:#552222;}#mermaid-svg-68R9HXu2vNiU9KAz .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-68R9HXu2vNiU9KAz .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-68R9HXu2vNiU9KAz .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-68R9HXu2vNiU9KAz .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-68R9HXu2vNiU9KAz .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-68R9HXu2vNiU9KAz .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-68R9HXu2vNiU9KAz .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-68R9HXu2vNiU9KAz .marker{fill:#333333;stroke:#333333;}#mermaid-svg-68R9HXu2vNiU9KAz .marker.cross{stroke:#333333;}#mermaid-svg-68R9HXu2vNiU9KAz svg{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-68R9HXu2vNiU9KAz p{margin:0;}#mermaid-svg-68R9HXu2vNiU9KAz .label{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;color:#333;}#mermaid-svg-68R9HXu2vNiU9KAz .cluster-label text{fill:#333;}#mermaid-svg-68R9HXu2vNiU9KAz .cluster-label span{color:#333;}#mermaid-svg-68R9HXu2vNiU9KAz .cluster-label span p{background-color:transparent;}#mermaid-svg-68R9HXu2vNiU9KAz .label text,#mermaid-svg-68R9HXu2vNiU9KAz span{fill:#333;color:#333;}#mermaid-svg-68R9HXu2vNiU9KAz .node rect,#mermaid-svg-68R9HXu2vNiU9KAz .node circle,#mermaid-svg-68R9HXu2vNiU9KAz .node ellipse,#mermaid-svg-68R9HXu2vNiU9KAz .node polygon,#mermaid-svg-68R9HXu2vNiU9KAz .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-68R9HXu2vNiU9KAz .rough-node .label text,#mermaid-svg-68R9HXu2vNiU9KAz .node .label text,#mermaid-svg-68R9HXu2vNiU9KAz .image-shape .label,#mermaid-svg-68R9HXu2vNiU9KAz .icon-shape .label{text-anchor:middle;}#mermaid-svg-68R9HXu2vNiU9KAz .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-68R9HXu2vNiU9KAz .rough-node .label,#mermaid-svg-68R9HXu2vNiU9KAz .node .label,#mermaid-svg-68R9HXu2vNiU9KAz .image-shape .label,#mermaid-svg-68R9HXu2vNiU9KAz .icon-shape .label{text-align:center;}#mermaid-svg-68R9HXu2vNiU9KAz .node.clickable{cursor:pointer;}#mermaid-svg-68R9HXu2vNiU9KAz .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-68R9HXu2vNiU9KAz .arrowheadPath{fill:#333333;}#mermaid-svg-68R9HXu2vNiU9KAz .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-68R9HXu2vNiU9KAz .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-68R9HXu2vNiU9KAz .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-68R9HXu2vNiU9KAz .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-68R9HXu2vNiU9KAz .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-68R9HXu2vNiU9KAz .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-68R9HXu2vNiU9KAz .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-68R9HXu2vNiU9KAz .cluster text{fill:#333;}#mermaid-svg-68R9HXu2vNiU9KAz .cluster span{color:#333;}#mermaid-svg-68R9HXu2vNiU9KAz div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-68R9HXu2vNiU9KAz .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-68R9HXu2vNiU9KAz rect.text{fill:none;stroke-width:0;}#mermaid-svg-68R9HXu2vNiU9KAz .icon-shape,#mermaid-svg-68R9HXu2vNiU9KAz .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-68R9HXu2vNiU9KAz .icon-shape p,#mermaid-svg-68R9HXu2vNiU9KAz .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-68R9HXu2vNiU9KAz .icon-shape rect,#mermaid-svg-68R9HXu2vNiU9KAz .image-shape rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-68R9HXu2vNiU9KAz .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-68R9HXu2vNiU9KAz .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-68R9HXu2vNiU9KAz :root{–mermaid-font-family:\”trebuchet ms\”,verdana,arial,sans-serif;}

VXLAN模式（云环境推荐）

VXLAN封装UDP 4789

云安全组放行✅

Pod B10.244.2.20

vxlan.calico接口

物理网卡

目标节点

IPIP模式（物理机可用）

IPIP封装协议号4

云安全组拦截❌

Pod A10.244.1.10

tunl0接口

物理网卡

目标节点

核心区别：

• IPIP：使用IP协议号4封装，云厂商安全组不支持
• VXLAN：使用UDP 4789端口，可以通过安全组规则放行

5.1.2 下载并修改Calico配置

# 下载Calico 3.27版本配置文件
curl -O https://raw.githubusercontent.com/projectcalico/calico/v3.27.0/manifests/calico.yaml

# 备份原文件
cp calico.yaml calico.yaml.bak

修改配置文件（3处关键修改）：

# 1. 修改Pod网络CIDR（与kubeadm-config.yaml保持一致）
sed -i 's|# – name: CALICO_IPV4POOL_CIDR|- name: CALICO_IPV4POOL_CIDR|' calico.yaml
sed -i 's|# value: "192.168.0.0/16"| value: "10.244.0.0/16"|' calico.yaml

# 2. 修改网络模式为VXLAN（默认是IPIP）
sed -i 's|# – name: CALICO_IPV4POOL_IPIP|- name: CALICO_IPV4POOL_IPIP|' calico.yaml
sed -i 's|# value: "Always"| value: "Never"|' calico.yaml

# 添加VXLAN配置
sed -i '/CALICO_IPV4POOL_IPIP/a\\ – name: CALICO_IPV4POOL_VXLAN\\n value: "Always"' calico.yaml

# 3. 配置网卡自动检测（可选，多网卡环境需要）
sed -i '/CALICO_IPV4POOL_VXLAN/a\\ – name: IP_AUTODETECTION_METHOD\\n value: "interface=eth0"' calico.yaml

手动验证配置（推荐）：

# 检查关键配置
grep -A 1 "CALICO_IPV4POOL_CIDR" calico.yaml
grep -A 1 "CALICO_IPV4POOL_IPIP" calico.yaml
grep -A 1 "CALICO_IPV4POOL_VXLAN" calico.yaml

期望输出：

– name: CALICO_IPV4POOL_CIDR
value: "10.244.0.0/16"
– name: CALICO_IPV4POOL_IPIP
value: "Never"
– name: CALICO_IPV4POOL_VXLAN
value: "Always"

5.1.3 部署Calico

# 应用配置
kubectl apply -f calico.yaml

# 等待Pod启动（约1-2分钟）
kubectl get pods -n kube-system -l k8s-app=calico-node -w

验证部署：

# 1. 检查Calico Pod状态
kubectl get pods -n kube-system -l k8s-app=calico-node

# 期望输出：所有Pod都是Running
# NAME READY STATUS RESTARTS AGE
# calico-node-xxxxx 1/1 Running 0 2m
# calico-node-yyyyy 1/1 Running 0 2m
# calico-node-zzzzz 1/1 Running 0 2m

# 2. 检查节点状态（应该变为Ready）
kubectl get nodes

# 期望输出：所有节点都是Ready
# NAME STATUS ROLES AGE VERSION
# k8s-master Ready control-plane 10m v1.28.2
# k8s-node1 Ready <none> 7m v1.28.2
# k8s-node2 Ready <none> 7m v1.28.2
# k8s-node3 Ready <none> 7m v1.28.2

# 3. 验证VXLAN接口（在任意节点执行）
ip link show vxlan.calico

# 期望输出：
# 5: vxlan.calico: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UNKNOWN mode DEFAULT group default
# link/ether 66:xx:xx:xx:xx:xx brd ff:ff:ff:ff:ff:ff

# 4. 检查路由表
ip route | grep vxlan.calico

5.1.4 网络连通性测试

创建测试Pod：

# 创建3个nginx Pod
kubectl create deployment nginx-test –image=nginx:1.25 –replicas=3

# 等待Pod启动
kubectl get pods -o wide

# 期望输出：Pod分布在不同节点
# NAME READY STATUS RESTARTS AGE IP NODE
# nginx-test-xxxxxx-xxxxx 1/1 Running 0 30s 10.244.36.10 k8s-node1
# nginx-test-xxxxxx-yyyyy 1/1 Running 0 30s 10.244.169.20 k8s-node2
# nginx-test-xxxxxx-zzzzz 1/1 Running 0 30s 10.244.104.30 k8s-node3

测试跨节点通信：

# 获取Pod IP
POD1_IP=$(kubectl get pod -l app=nginx-test -o jsonpath='{.items[0].status.podIP}')
POD2_IP=$(kubectl get pod -l app=nginx-test -o jsonpath='{.items[1].status.podIP}')

# 从Pod1访问Pod2
kubectl exec -it $(kubectl get pod -l app=nginx-test -o jsonpath='{.items[0].metadata.name}') — curl -s http://$POD2_IP

# 期望输出：nginx欢迎页面HTML

Pod网络通信流程：

Pod B

10.244.169.20

k8s-node2

vxlan.calico

node2

eth0

192.168.1.12

eth0

192.168.1.11

vxlan.calico

node1

Pod A

10.244.36.10

k8s-node1

Pod B

10.244.169.20

k8s-node2

vxlan.calico

node2

eth0

192.168.1.12

eth0

192.168.1.11

vxlan.calico

node1

Pod A

10.244.36.10

k8s-node1

#mermaid-svg-5NpROUJZ5xoDUVa1{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-5NpROUJZ5xoDUVa1 .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-5NpROUJZ5xoDUVa1 .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-5NpROUJZ5xoDUVa1 .error-icon{fill:#552222;}#mermaid-svg-5NpROUJZ5xoDUVa1 .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-5NpROUJZ5xoDUVa1 .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-5NpROUJZ5xoDUVa1 .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-5NpROUJZ5xoDUVa1 .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-5NpROUJZ5xoDUVa1 .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-5NpROUJZ5xoDUVa1 .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-5NpROUJZ5xoDUVa1 .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-5NpROUJZ5xoDUVa1 .marker{fill:#333333;stroke:#333333;}#mermaid-svg-5NpROUJZ5xoDUVa1 .marker.cross{stroke:#333333;}#mermaid-svg-5NpROUJZ5xoDUVa1 svg{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-5NpROUJZ5xoDUVa1 p{margin:0;}#mermaid-svg-5NpROUJZ5xoDUVa1 .actor{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-5NpROUJZ5xoDUVa1 text.actor>tspan{fill:black;stroke:none;}#mermaid-svg-5NpROUJZ5xoDUVa1 .actor-line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);}#mermaid-svg-5NpROUJZ5xoDUVa1 .innerArc{stroke-width:1.5;stroke-dasharray:none;}#mermaid-svg-5NpROUJZ5xoDUVa1 .messageLine0{stroke-width:1.5;stroke-dasharray:none;stroke:#333;}#mermaid-svg-5NpROUJZ5xoDUVa1 .messageLine1{stroke-width:1.5;stroke-dasharray:2,2;stroke:#333;}#mermaid-svg-5NpROUJZ5xoDUVa1 #arrowhead path{fill:#333;stroke:#333;}#mermaid-svg-5NpROUJZ5xoDUVa1 .sequenceNumber{fill:white;}#mermaid-svg-5NpROUJZ5xoDUVa1 #sequencenumber{fill:#333;}#mermaid-svg-5NpROUJZ5xoDUVa1 #crosshead path{fill:#333;stroke:#333;}#mermaid-svg-5NpROUJZ5xoDUVa1 .messageText{fill:#333;stroke:none;}#mermaid-svg-5NpROUJZ5xoDUVa1 .labelBox{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-5NpROUJZ5xoDUVa1 .labelText,#mermaid-svg-5NpROUJZ5xoDUVa1 .labelText>tspan{fill:black;stroke:none;}#mermaid-svg-5NpROUJZ5xoDUVa1 .loopText,#mermaid-svg-5NpROUJZ5xoDUVa1 .loopText>tspan{fill:black;stroke:none;}#mermaid-svg-5NpROUJZ5xoDUVa1 .loopLine{stroke-width:2px;stroke-dasharray:2,2;stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);}#mermaid-svg-5NpROUJZ5xoDUVa1 .note{stroke:#aaaa33;fill:#fff5ad;}#mermaid-svg-5NpROUJZ5xoDUVa1 .noteText,#mermaid-svg-5NpROUJZ5xoDUVa1 .noteText>tspan{fill:black;stroke:none;}#mermaid-svg-5NpROUJZ5xoDUVa1 .activation0{fill:#f4f4f4;stroke:#666;}#mermaid-svg-5NpROUJZ5xoDUVa1 .activation1{fill:#f4f4f4;stroke:#666;}#mermaid-svg-5NpROUJZ5xoDUVa1 .activation2{fill:#f4f4f4;stroke:#666;}#mermaid-svg-5NpROUJZ5xoDUVa1 .actorPopupMenu{position:absolute;}#mermaid-svg-5NpROUJZ5xoDUVa1 .actorPopupMenuPanel{position:absolute;fill:#ECECFF;box-shadow:0px 8px 16px 0px rgba(0,0,0,0.2);filter:drop-shadow(3px 5px 2px rgb(0 0 0 / 0.4));}#mermaid-svg-5NpROUJZ5xoDUVa1 .actor-man line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-5NpROUJZ5xoDUVa1 .actor-man circle,#mermaid-svg-5NpROUJZ5xoDUVa1 line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;stroke-width:2px;}#mermaid-svg-5NpROUJZ5xoDUVa1 :root{–mermaid-font-family:\”trebuchet ms\”,verdana,arial,sans-serif;}

跨节点通信通过VXLAN隧道实现

发送数据包

目标: 10.244.169.20

VXLAN封装

外层: 192.168.1.11 → 192.168.1.12

内层: 10.244.36.10 → 10.244.169.20

UDP 4789端口传输

VXLAN解封装

转发到目标Pod

5.2 配置NFS动态存储

5.2.1 部署NFS服务器

⚠️ 以下操作在Master节点执行（作为NFS服务器）

# 安装NFS服务
yum install -y nfs-utils rpcbind

# 创建共享目录
mkdir -p /data/nfs/k8s
chmod 777 /data/nfs/k8s

# 配置NFS导出
cat >> /etc/exports << EOF
/data/nfs/k8s *(rw,sync,no_root_squash,no_subtree_check)
EOF

# 启动服务
systemctl start rpcbind nfs-server
systemctl enable rpcbind nfs-server

# 重新加载配置
exportfs -r

# 验证NFS导出
showmount -e localhost

在所有Worker节点安装NFS客户端：

# 安装nfs-utils
yum install -y nfs-utils

# 测试挂载
mkdir -p /mnt/test
mount -t nfs 192.168.1.10:/data/nfs/k8s /mnt/test
df -h | grep nfs
umount /mnt/test

5.2.2 部署NFS动态供应器

# 创建命名空间
kubectl create namespace nfs-provisioner

# 部署nfs-subdir-external-provisioner
kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/nfs-subdir-external-provisioner/master/deploy/rbac.yaml

# 下载deployment配置
curl -O https://raw.githubusercontent.com/kubernetes-sigs/nfs-subdir-external-provisioner/master/deploy/deployment.yaml

# 修改NFS服务器地址和路径
sed -i 's|10.3.243.101|192.168.1.10|' deployment.yaml
sed -i 's|/ifs/kubernetes|/data/nfs/k8s|' deployment.yaml

# 应用配置
kubectl apply -f deployment.yaml

# 验证Pod状态
kubectl get pods -n nfs-provisioner

创建StorageClass：

cat > nfs-storage-class.yaml << EOF
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: nfs-storage
provisioner: k8s-sigs.io/nfs-subdir-external-provisioner
parameters:
archiveOnDelete: "false"
reclaimPolicy: Delete
volumeBindingMode: Immediate
EOF

kubectl apply -f nfs-storage-class.yaml

# 设置为默认StorageClass
kubectl patch storageclass nfs-storage -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'

# 验证
kubectl get storageclass

5.2.3 测试动态存储

# 创建测试PVC
cat > test-pvc.yaml << EOF
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: test-pvc
spec:
accessModes:
– ReadWriteMany
resources:
requests:
storage: 1Gi
storageClassName: nfs-storage
EOF

kubectl apply -f test-pvc.yaml

# 验证PVC状态（应该自动绑定）
kubectl get pvc test-pvc

# 期望输出：
# NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
# test-pvc Bound pvc-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx 1Gi RWX nfs-storage 10s

# 检查NFS目录（在Master节点）
ls -l /data/nfs/k8s/

# 清理测试资源
kubectl delete pvc test-pvc

存储动态供应流程：

#mermaid-svg-pukJF3wsJJPAbLVL{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-pukJF3wsJJPAbLVL .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-pukJF3wsJJPAbLVL .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-pukJF3wsJJPAbLVL .error-icon{fill:#552222;}#mermaid-svg-pukJF3wsJJPAbLVL .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-pukJF3wsJJPAbLVL .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-pukJF3wsJJPAbLVL .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-pukJF3wsJJPAbLVL .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-pukJF3wsJJPAbLVL .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-pukJF3wsJJPAbLVL .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-pukJF3wsJJPAbLVL .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-pukJF3wsJJPAbLVL .marker{fill:#333333;stroke:#333333;}#mermaid-svg-pukJF3wsJJPAbLVL .marker.cross{stroke:#333333;}#mermaid-svg-pukJF3wsJJPAbLVL svg{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-pukJF3wsJJPAbLVL p{margin:0;}#mermaid-svg-pukJF3wsJJPAbLVL .label{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;color:#333;}#mermaid-svg-pukJF3wsJJPAbLVL .cluster-label text{fill:#333;}#mermaid-svg-pukJF3wsJJPAbLVL .cluster-label span{color:#333;}#mermaid-svg-pukJF3wsJJPAbLVL .cluster-label span p{background-color:transparent;}#mermaid-svg-pukJF3wsJJPAbLVL .label text,#mermaid-svg-pukJF3wsJJPAbLVL span{fill:#333;color:#333;}#mermaid-svg-pukJF3wsJJPAbLVL .node rect,#mermaid-svg-pukJF3wsJJPAbLVL .node circle,#mermaid-svg-pukJF3wsJJPAbLVL .node ellipse,#mermaid-svg-pukJF3wsJJPAbLVL .node polygon,#mermaid-svg-pukJF3wsJJPAbLVL .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-pukJF3wsJJPAbLVL .rough-node .label text,#mermaid-svg-pukJF3wsJJPAbLVL .node .label text,#mermaid-svg-pukJF3wsJJPAbLVL .image-shape .label,#mermaid-svg-pukJF3wsJJPAbLVL .icon-shape .label{text-anchor:middle;}#mermaid-svg-pukJF3wsJJPAbLVL .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-pukJF3wsJJPAbLVL .rough-node .label,#mermaid-svg-pukJF3wsJJPAbLVL .node .label,#mermaid-svg-pukJF3wsJJPAbLVL .image-shape .label,#mermaid-svg-pukJF3wsJJPAbLVL .icon-shape .label{text-align:center;}#mermaid-svg-pukJF3wsJJPAbLVL .node.clickable{cursor:pointer;}#mermaid-svg-pukJF3wsJJPAbLVL .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-pukJF3wsJJPAbLVL .arrowheadPath{fill:#333333;}#mermaid-svg-pukJF3wsJJPAbLVL .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-pukJF3wsJJPAbLVL .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-pukJF3wsJJPAbLVL .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-pukJF3wsJJPAbLVL .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-pukJF3wsJJPAbLVL .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-pukJF3wsJJPAbLVL .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-pukJF3wsJJPAbLVL .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-pukJF3wsJJPAbLVL .cluster text{fill:#333;}#mermaid-svg-pukJF3wsJJPAbLVL .cluster span{color:#333;}#mermaid-svg-pukJF3wsJJPAbLVL div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-pukJF3wsJJPAbLVL .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-pukJF3wsJJPAbLVL rect.text{fill:none;stroke-width:0;}#mermaid-svg-pukJF3wsJJPAbLVL .icon-shape,#mermaid-svg-pukJF3wsJJPAbLVL .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-pukJF3wsJJPAbLVL .icon-shape p,#mermaid-svg-pukJF3wsJJPAbLVL .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-pukJF3wsJJPAbLVL .icon-shape rect,#mermaid-svg-pukJF3wsJJPAbLVL .image-shape rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-pukJF3wsJJPAbLVL .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-pukJF3wsJJPAbLVL .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-pukJF3wsJJPAbLVL :root{–mermaid-font-family:\”trebuchet ms\”,verdana,arial,sans-serif;}

用户创建PVC

StorageClass

NFS Provisioner

创建PV

在NFS服务器创建目录

PVC绑定PV

Pod挂载PVC

✅ 第三阶段完成检查清单：

• Calico网络插件已部署（VXLAN模式）
• 所有节点状态为Ready
• vxlan.calico接口存在（不是tunl0）
• 跨节点Pod通信正常
• NFS服务器已配置并可访问
• NFS动态供应器已部署
• StorageClass已创建并设为默认
• 测试PVC可以自动绑定

下一步：第六章将部署Ingress-Nginx和Prometheus监控系统。

第六章：Ingress与监控

6.1 部署Ingress-Nginx（hostPort模式）

Ingress-Nginx是K8s集群的流量入口，负责将外部HTTP/HTTPS请求路由到集群内的Service。

6.1.1 为什么使用hostPort模式？

部署模式对比：

本文选择hostPort模式，将Ingress Controller固定在k8s-node3节点，直接监听80/443端口。

6.1.2 部署Ingress-Nginx

# 下载官方配置文件
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.9.4/deploy/static/provider/cloud/deploy.yaml

# 等待Pod启动
kubectl get pods -n ingress-nginx -w

修改为hostPort模式：

# 编辑Deployment
kubectl edit deployment ingress-nginx-controller -n ingress-nginx

# 在containers部分添加hostPort配置：
# ports:
# – containerPort: 80
# hostPort: 80
# name: http
# protocol: TCP
# – containerPort: 443
# hostPort: 443
# name: https
# protocol: TCP

# 添加节点选择器（固定到node3）：
# nodeSelector:
# kubernetes.io/hostname: k8s-node3

验证部署：

# 检查Pod状态
kubectl get pods -n ingress-nginx -o wide

# 期望输出：Pod运行在k8s-node3节点
# NAME READY STATUS NODE
# ingress-nginx-controller-xxxxxxxxxx-xxxxx 1/1 Running k8s-node3

# 测试访问（在k8s-node3节点执行）
curl http://localhost

# 期望输出：404 Not Found（正常，因为还没有配置Ingress规则）

6.1.3 创建测试Ingress

# 创建测试应用
kubectl create deployment web –image=nginx:1.25
kubectl expose deployment web –port=80

# 创建Ingress规则
cat > test-ingress.yaml << EOF
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: test-ingress
spec:
ingressClassName: nginx
rules:
– host: test.example.com
http:
paths:
– path: /
pathType: Prefix
backend:
service:
name: web
port:
number: 80
EOF

kubectl apply -f test-ingress.yaml

# 验证Ingress
kubectl get ingress test-ingress

测试访问：

# 在本地配置hosts（或使用curl -H）
curl -H "Host: test.example.com" http://192.168.1.13

# 期望输出：nginx欢迎页面

6.2 部署Prometheus监控

6.2.1 安装Helm

# 下载Helm安装脚本
curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash

# 验证安装
helm version

6.2.2 部署Prometheus Stack

# 添加Prometheus社区Helm仓库
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update

# 创建命名空间
kubectl create namespace monitoring

# 创建values配置文件
cat > prometheus-values.yaml << EOF
prometheus:
prometheusSpec:
storageSpec:
volumeClaimTemplate:
spec:
storageClassName: nfs-storage
accessModes: ["ReadWriteOnce"]
resources:
requests:
storage: 10Gi

grafana:
adminPassword: "admin123"
persistence:
enabled: true
storageClassName: nfs-storage
size: 5Gi
service:
type: NodePort
nodePort: 30300

alertmanager:
alertmanagerSpec:
storage:
volumeClaimTemplate:
spec:
storageClassName: nfs-storage
accessModes: ["ReadWriteOnce"]
resources:
requests:
storage: 5Gi
EOF

# 安装Prometheus Stack
helm install prometheus prometheus-community/kube-prometheus-stack \\
-n monitoring \\
-f prometheus-values.yaml

# 等待Pod启动（约2-3分钟）
kubectl get pods -n monitoring -w

6.2.3 访问Grafana

# 获取Grafana访问地址
kubectl get svc -n monitoring prometheus-grafana

# 期望输出：
# NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
# prometheus-grafana NodePort 10.96.xxx.xxx <none> 80:30300/TCP 5m

# 访问Grafana
# URL: http://192.168.1.13:30300
# 用户名: admin
# 密码: admin123

验证监控数据：

第七章：常见问题与排查

7.1 镜像拉取失败

问题现象：

Failed to pull image "registry.k8s.io/pause:3.9": rpc error: code = Unknown desc = failed to pull and unpack image

解决方案：

# 1. 检查containerd配置
grep "registry.aliyuncs.com" /etc/containerd/config.toml

# 2. 手动拉取测试
ctr image pull registry.aliyuncs.com/google_containers/pause:3.9

# 3. 重启containerd
systemctl restart containerd

7.2 跨节点通信失败

问题现象：

# Pod无法访问其他节点的Pod
curl: (7) Failed to connect to 10.244.x.x port 80: No route to host

排查步骤：

# 1. 检查网络模式（必须是VXLAN）
ip link show vxlan.calico # 应该存在
ip link show tunl0 # 不应该存在

# 2. 检查Calico配置
kubectl get ippool default-ipv4-ippool -o yaml | grep vxlan

# 期望输出：
# vxlanMode: Always
# ipipMode: Never

# 3. 检查安全组规则
# 确保UDP 4789端口已开放

# 4. 重启Calico（如果配置错误）
kubectl delete pod -n kube-system -l k8s-app=calico-node

7.3 快速诊断命令

集群状态检查：

# 节点状态
kubectl get nodes -o wide

# Pod状态
kubectl get pods -A -o wide | grep -v Running

# 组件状态
kubectl get cs

# 事件查看
kubectl get events -A –sort-by='.lastTimestamp' | tail -20

网络诊断：

# 检查VXLAN接口
ip link show vxlan.calico

# 检查路由表
ip route | grep vxlan

# 测试Pod网络
kubectl run test –image=busybox:1.28 –restart=Never –rm -it — ping <pod-ip>

日志查看：

# Calico日志
kubectl logs -n kube-system -l k8s-app=calico-node –tail=50

# kubelet日志
journalctl -u kubelet -n 50 –no-pager

# containerd日志
journalctl -u containerd -n 50 –no-pager

🎉 总结

恭喜你！现在你已经成功搭建了一套完整的Kubernetes集群，包括：

✅ 4节点集群：1个Master + 3个Worker节点 ✅ 网络插件：Calico VXLAN模式（云环境必备） ✅ 动态存储：NFS自动供应PV ✅ 流量入口：Ingress-Nginx hostPort模式 ✅ 监控体系：Prometheus + Grafana

核心要点回顾

参考资源

• Kubernetes官方文档：https://kubernetes.io/docs/
• Calico官方文档：https://docs.tigera.io/calico/latest/
• Prometheus官方文档：https://prometheus.io/docs/

如有问题，欢迎在评论区交流讨论！