一：目标
安装复合当前k8s版本的ingress controller，实现ingress的高可用架构、
ingress controller的的pod应该部署在专用服务器上，所以一般都使用节点硬亲和性。
我们使用1.9.5好了。
二：安装过程
1：下载源码包
https://codeload.github.com/kubernetes/ingress-nginx/tar.gz/refs/tags/controller-v1.9.5
ingress-nginx-controller-v1.9.5.tar.gz
2：上传到主节点
1：解压包
下载源码包之后，把源码包，给他解压。完事之后，拿到deploy文件就行了。
然后，解压，然后把deploy文件拷贝出来，我们要使用的就是deploy文件。
2：拿到核心文件
-rw-rw-r–. 1 root root 16107 Dec 21 2023 deploy.yaml
-rw-rw-r–. 1 root root 321 Dec 21 2023 kustomization.yaml
[root@k8s-node1 cloud]# pwd
/root/ingress-nginx-controller-v1.9.5/deploy/static/provider/cloud
我们用的就是这个文件
3：原始文件内容
apiVersion: v1
kind: Namespace
metadata:
labels:
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
name: ingress-nginx
—
apiVersion: v1
automountServiceAccountToken: true
kind: ServiceAccount
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.9.5
name: ingress-nginx
namespace: ingress-nginx
—
apiVersion: v1
kind: ServiceAccount
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.9.5
name: ingress-nginx-admission
namespace: ingress-nginx
—
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.9.5
name: ingress-nginx
namespace: ingress-nginx
rules:
– apiGroups:
– ""
resources:
– namespaces
verbs:
– get
– apiGroups:
– ""
resources:
– configmaps
– pods
– secrets
– endpoints
verbs:
– get
– list
– watch
– apiGroups:
– ""
resources:
– services
verbs:
– get
– list
– watch
– apiGroups:
– networking.k8s.io
resources:
– ingresses
verbs:
– get
– list
– watch
– apiGroups:
– networking.k8s.io
resources:
– ingresses/status
verbs:
– update
– apiGroups:
– networking.k8s.io
resources:
– ingressclasses
verbs:
– get
– list
– watch
– apiGroups:
– coordination.k8s.io
resourceNames:
– ingress-nginx-leader
resources:
– leases
verbs:
– get
– update
– apiGroups:
– coordination.k8s.io
resources:
– leases
verbs:
– create
– apiGroups:
– ""
resources:
– events
verbs:
– create
– patch
– apiGroups:
– discovery.k8s.io
resources:
– endpointslices
verbs:
– list
– watch
– get
—
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.9.5
name: ingress-nginx-admission
namespace: ingress-nginx
rules:
– apiGroups:
– ""
resources:
– secrets
verbs:
– get
– create
—
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
labels:
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.9.5
name: ingress-nginx
rules:
– apiGroups:
– ""
resources:
– configmaps
– endpoints
– nodes
– pods
– secrets
– namespaces
verbs:
– list
– watch
– apiGroups:
– coordination.k8s.io
resources:
– leases
verbs:
– list
– watch
– apiGroups:
– ""
resources:
– nodes
verbs:
– get
– apiGroups:
– ""
resources:
– services
verbs:
– get
– list
– watch
– apiGroups:
– networking.k8s.io
resources:
– ingresses
verbs:
– get
– list
– watch
– apiGroups:
– ""
resources:
– events
verbs:
– create
– patch
– apiGroups:
– networking.k8s.io
resources:
– ingresses/status
verbs:
– update
– apiGroups:
– networking.k8s.io
resources:
– ingressclasses
verbs:
– get
– list
– watch
– apiGroups:
– discovery.k8s.io
resources:
– endpointslices
verbs:
– list
– watch
– get
—
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.9.5
name: ingress-nginx-admission
rules:
– apiGroups:
– admissionregistration.k8s.io
resources:
– validatingwebhookconfigurations
verbs:
– get
– update
—
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.9.5
name: ingress-nginx
namespace: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: ingress-nginx
subjects:
– kind: ServiceAccount
name: ingress-nginx
namespace: ingress-nginx
—
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.9.5
name: ingress-nginx-admission
namespace: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: ingress-nginx-admission
subjects:
– kind: ServiceAccount
name: ingress-nginx-admission
namespace: ingress-nginx
—
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
labels:
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.9.5
name: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: ingress-nginx
subjects:
– kind: ServiceAccount
name: ingress-nginx
namespace: ingress-nginx
—
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.9.5
name: ingress-nginx-admission
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: ingress-nginx-admission
subjects:
– kind: ServiceAccount
name: ingress-nginx-admission
namespace: ingress-nginx
—
apiVersion: v1
data:
allow-snippet-annotations: "false"
kind: ConfigMap
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.9.5
name: ingress-nginx-controller
namespace: ingress-nginx
—
apiVersion: v1
kind: Service
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.9.5
name: ingress-nginx-controller
namespace: ingress-nginx
spec:
externalTrafficPolicy: Local
ipFamilies:
– IPv4
ipFamilyPolicy: SingleStack
ports:
– appProtocol: http
name: http
port: 80
protocol: TCP
targetPort: http
– appProtocol: https
name: https
port: 443
protocol: TCP
targetPort: https
selector:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
type: LoadBalancer
—
apiVersion: v1
kind: Service
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.9.5
name: ingress-nginx-controller-admission
namespace: ingress-nginx
spec:
ports:
– appProtocol: https
name: https-webhook
port: 443
targetPort: webhook
selector:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
type: ClusterIP
—
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.9.5
name: ingress-nginx-controller
namespace: ingress-nginx
spec:
minReadySeconds: 0
revisionHistoryLimit: 10
selector:
matchLabels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
strategy:
rollingUpdate:
maxUnavailable: 1
type: RollingUpdate
template:
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.9.5
spec:
containers:
– args:
– /nginx-ingress-controller
– –publish-service=$(POD_NAMESPACE)/ingress-nginx-controller
– –election-id=ingress-nginx-leader
– –controller-class=k8s.io/ingress-nginx
– –ingress-class=nginx
– –configmap=$(POD_NAMESPACE)/ingress-nginx-controller
– –validating-webhook=:8443
– –validating-webhook-certificate=/usr/local/certificates/cert
– –validating-webhook-key=/usr/local/certificates/key
env:
– name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
– name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
– name: LD_PRELOAD
value: /usr/local/lib/libmimalloc.so
image: registry.k8s.io/ingress-nginx/controller:v1.9.5@sha256:b3aba22b1da80e7acfc52b115cae1d4c687172cbf2b742d5b502419c25ff340e
imagePullPolicy: IfNotPresent
lifecycle:
preStop:
exec:
command:
– /wait-shutdown
livenessProbe:
failureThreshold: 5
httpGet:
path: /healthz
port: 10254
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
name: controller
ports:
– containerPort: 80
name: http
protocol: TCP
– containerPort: 443
name: https
protocol: TCP
– containerPort: 8443
name: webhook
protocol: TCP
readinessProbe:
failureThreshold: 3
httpGet:
path: /healthz
port: 10254
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
resources:
requests:
cpu: 100m
memory: 90Mi
securityContext:
allowPrivilegeEscalation: false
capabilities:
add:
– NET_BIND_SERVICE
drop:
– ALL
readOnlyRootFilesystem: false
runAsNonRoot: true
runAsUser: 101
seccompProfile:
type: RuntimeDefault
volumeMounts:
– mountPath: /usr/local/certificates/
name: webhook-cert
readOnly: true
dnsPolicy: ClusterFirst
nodeSelector:
kubernetes.io/os: linux
serviceAccountName: ingress-nginx
terminationGracePeriodSeconds: 300
volumes:
– name: webhook-cert
secret:
secretName: ingress-nginx-admission
—
apiVersion: batch/v1
kind: Job
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.9.5
name: ingress-nginx-admission-create
namespace: ingress-nginx
spec:
template:
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.9.5
name: ingress-nginx-admission-create
spec:
containers:
– args:
– create
– –host=ingress-nginx-controller-admission,ingress-nginx-controller-admission.$(POD_NAMESPACE).svc
– –namespace=$(POD_NAMESPACE)
– –secret-name=ingress-nginx-admission
env:
– name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
image: registry.k8s.io/ingress-nginx/kube-webhook-certgen:v20231011-8b53cabe0@sha256:a7943503b45d552785aa3b5e457f169a5661fb94d82b8a3373bcd9ebaf9aac80
imagePullPolicy: IfNotPresent
name: create
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop:
– ALL
readOnlyRootFilesystem: true
runAsNonRoot: true
runAsUser: 65532
seccompProfile:
type: RuntimeDefault
nodeSelector:
kubernetes.io/os: linux
restartPolicy: OnFailure
serviceAccountName: ingress-nginx-admission
—
apiVersion: batch/v1
kind: Job
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.9.5
name: ingress-nginx-admission-patch
namespace: ingress-nginx
spec:
template:
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.9.5
name: ingress-nginx-admission-patch
spec:
containers:
– args:
– patch
– –webhook-name=ingress-nginx-admission
– –namespace=$(POD_NAMESPACE)
– –patch-mutating=false
– –secret-name=ingress-nginx-admission
– –patch-failure-policy=Fail
env:
– name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
image: registry.k8s.io/ingress-nginx/kube-webhook-certgen:v20231011-8b53cabe0@sha256:a7943503b45d552785aa3b5e457f169a5661fb94d82b8a3373bcd9ebaf9aac80
imagePullPolicy: IfNotPresent
name: patch
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop:
– ALL
readOnlyRootFilesystem: true
runAsNonRoot: true
runAsUser: 65532
seccompProfile:
type: RuntimeDefault
nodeSelector:
kubernetes.io/os: linux
restartPolicy: OnFailure
serviceAccountName: ingress-nginx-admission
—
apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.9.5
name: nginx
spec:
controller: k8s.io/ingress-nginx
—
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingWebhookConfiguration
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.9.5
name: ingress-nginx-admission
webhooks:
– admissionReviewVersions:
– v1
clientConfig:
service:
name: ingress-nginx-controller-admission
namespace: ingress-nginx
path: /networking/v1/ingresses
failurePolicy: Fail
matchPolicy: Equivalent
name: validate.nginx.ingress.kubernetes.io
rules:
– apiGroups:
– networking.k8s.io
apiVersions:
– v1
operations:
– CREATE
– UPDATE
resources:
– ingresses
sideEffects: None
4：修改文件内容
1：将deployment控制器修改为DaemonSet资源。
Ingress Controller 是集群流量入口，不是普通应用：
- Deployment 是弹性应用模式，副本随机调度，会造成流量集中、跨节点转发。
- DaemonSet 保证每台节点都运行一个 Ingress，让流量直接进入当前节点，性能、高可用、扩展性最适合生产网关。
2：添加hostnetwork
hostnetwork 为仅主机网络模式。使用这种网络模式可以不采用 service 或 ingress 做流量转发暴露服务。
3：修改镜像地址为阿里云镜像
将 controller 镜像替换为：
registry.cn-hangzhou.aliyuncs.com/google_containers/nginx-ingress-controller:v1.9.5
将 kube-webhook-certgen 镜像替换为：
registry.cn-hangzhou.aliyuncs.com/google_containers/kube-webhook-certgen:v20230312-helm-chart-4.5.2-28-g66a760794
4：添加亲和性
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
– matchExpressions:
– key: kubernetes.io/ingress
operator: In
values:
– nginx
5：禁用节点选择器
用亲和性就好了，没必要搞这个。
5：镜像拉取测试一下
替换之后的地址，镜像拉取没有问题。
[root@k8s-node1 ingress]# nerdctl pull registry.cn-hangzhou.aliyuncs.com/google_containers/nginx-ingress-controller:v1.7.1
registry.cn-hangzhou.aliyuncs.com/google_containers/nginx-ingress-controller:v1.7.1: resolved |++++++++++++++++++++++++++++++++++++++|
index-sha256:788c6dc8dcc0b17330789048c47cdf41d2ec21427d3adb61543a7de7ee85512c: done |++++++++++++++++++++++++++++++++++++++|
manifest-sha256:7f5af537d80289e0f500bfd9e4da4ef5f22bbcaca1d8da7016f0539608797442: done |++++++++++++++++++++++++++++++++++++++|
config-sha256:2db0b57c87122bf9dab14f527656bb8a1665f4b6a4e56f2e7feac71502a8f7c3: done |++++++++++++++++++++++++++++++++++++++|
layer-sha256:6ce556c836df801f8122b520249a10731799d98671ded8b8bba06b82d1ca5b96: done |++++++++++++++++++++++++++++++++++++++|
layer-sha256:63b65145d645c1250c391b2d16ebe53b3747c295ca8ba2fcb6b0cf064a4dc21c: done |++++++++++++++++++++++++++++++++++++++|
layer-sha256:c61852a1d454c869c329f4871030ac288c13f981e75a05956c6a5602559208b7: done |++++++++++++++++++++++++++++++++++++++|
layer-sha256:de1befb33c3510358367c65acdfaebc09e5260b2e65b5a0f231404c7173c0011: done |++++++++++++++++++++++++++++++++++++++|
layer-sha256:36aeb843eff85df19800670a4a25bd52bb362e9afd74ae3a7eeab03cad1031b9: done |++++++++++++++++++++++++++++++++++++++|
layer-sha256:3af847ca67b5cff8a4d741509e1ff6f95986958ca43b4d84e30637d62f588d5d: done |++++++++++++++++++++++++++++++++++++++|
layer-sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1: done |++++++++++++++++++++++++++++++++++++++|
layer-sha256:f6792653db2c6bcd68b40231529bef0142a6ac0635116efc8148e78e3d5e24de: done |++++++++++++++++++++++++++++++++++++++|
layer-sha256:ca7797e1dfc51d7b7d8896672b68579cdb6bc2d78fb96287930af899f8baf7a4: done |++++++++++++++++++++++++++++++++++++++|
layer-sha256:dd05e355bb765b78a9556099a8a8809f71bb61486dd73373da6aa25bab132f3f: done |++++++++++++++++++++++++++++++++++++++|
layer-sha256:6df75ccbca0b51983ec8e10bf35e99ceb6826367cf34278bc0b585d601032c56: done |++++++++++++++++++++++++++++++++++++++|
layer-sha256:6eeb15bc73f773c499494c636466d143bba80da4906608495166b83e6ed516c1: done |++++++++++++++++++++++++++++++++++++++|
layer-sha256:b2dec14bbfcbdf6770c59e7f9463bcdc0d95086147ce41f50799b2c4b4387195: done |++++++++++++++++++++++++++++++++++++++|
layer-sha256:794377b137abc61ceda512ff18a13118fb4a981dbd1b2bba458e115bd9acc858: done |++++++++++++++++++++++++++++++++++++++|
elapsed: 9.2 s total: 111.2 (12.1 MiB/s)
[root@k8s-node1 ingress]#
3：应用脚本
修改之后的文件：
apiVersion: v1
kind: Namespace
metadata:
labels:
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
name: ingress-nginx
—
apiVersion: v1
automountServiceAccountToken: true
kind: ServiceAccount
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.9.5
name: ingress-nginx
namespace: ingress-nginx
—
apiVersion: v1
kind: ServiceAccount
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.9.5
name: ingress-nginx-admission
namespace: ingress-nginx
—
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.9.5
name: ingress-nginx
namespace: ingress-nginx
rules:
– apiGroups:
– ""
resources:
– namespaces
verbs:
– get
– apiGroups:
– ""
resources:
– configmaps
– pods
– secrets
– endpoints
verbs:
– get
– list
– watch
– apiGroups:
– ""
resources:
– services
verbs:
– get
– list
– watch
– apiGroups:
– networking.k8s.io
resources:
– ingresses
verbs:
– get
– list
– watch
– apiGroups:
– networking.k8s.io
resources:
– ingresses/status
verbs:
– update
– apiGroups:
– networking.k8s.io
resources:
– ingressclasses
verbs:
– get
– list
– watch
– apiGroups:
– coordination.k8s.io
resourceNames:
– ingress-nginx-leader
resources:
– leases
verbs:
– get
– update
– apiGroups:
– coordination.k8s.io
resources:
– leases
verbs:
– create
– apiGroups:
– ""
resources:
– events
verbs:
– create
– patch
– apiGroups:
– discovery.k8s.io
resources:
– endpointslices
verbs:
– list
– watch
– get
—
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.9.5
name: ingress-nginx-admission
namespace: ingress-nginx
rules:
– apiGroups:
– ""
resources:
– secrets
verbs:
– get
– create
—
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
labels:
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.9.5
name: ingress-nginx
rules:
– apiGroups:
– ""
resources:
– configmaps
– endpoints
– nodes
– pods
– secrets
– namespaces
verbs:
– list
– watch
– apiGroups:
– coordination.k8s.io
resources:
– leases
verbs:
– list
– watch
– apiGroups:
– ""
resources:
– nodes
verbs:
– get
– apiGroups:
– ""
resources:
– services
verbs:
– get
– list
– watch
– apiGroups:
– networking.k8s.io
resources:
– ingresses
verbs:
– get
– list
– watch
– apiGroups:
– ""
resources:
– events
verbs:
– create
– patch
– apiGroups:
– networking.k8s.io
resources:
– ingresses/status
verbs:
– update
– apiGroups:
– networking.k8s.io
resources:
– ingressclasses
verbs:
– get
– list
– watch
– apiGroups:
– discovery.k8s.io
resources:
– endpointslices
verbs:
– list
– watch
– get
—
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.9.5
name: ingress-nginx-admission
rules:
– apiGroups:
– admissionregistration.k8s.io
resources:
– validatingwebhookconfigurations
verbs:
– get
– update
—
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.9.5
name: ingress-nginx
namespace: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: ingress-nginx
subjects:
– kind: ServiceAccount
name: ingress-nginx
namespace: ingress-nginx
—
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.9.5
name: ingress-nginx-admission
namespace: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: ingress-nginx-admission
subjects:
– kind: ServiceAccount
name: ingress-nginx-admission
namespace: ingress-nginx
—
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
labels:
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.9.5
name: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: ingress-nginx
subjects:
– kind: ServiceAccount
name: ingress-nginx
namespace: ingress-nginx
—
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.9.5
name: ingress-nginx-admission
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: ingress-nginx-admission
subjects:
– kind: ServiceAccount
name: ingress-nginx-admission
namespace: ingress-nginx
—
apiVersion: v1
data:
allow-snippet-annotations: "false"
kind: ConfigMap
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.9.5
name: ingress-nginx-controller
namespace: ingress-nginx
—
apiVersion: v1
kind: Service
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.9.5
name: ingress-nginx-controller
namespace: ingress-nginx
spec:
externalTrafficPolicy: Local
ipFamilies:
– IPv4
ipFamilyPolicy: SingleStack
ports:
– appProtocol: http
name: http
port: 80
protocol: TCP
targetPort: http
– appProtocol: https
name: https
port: 443
protocol: TCP
targetPort: https
selector:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
type: LoadBalancer
—
apiVersion: v1
kind: Service
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.9.5
name: ingress-nginx-controller-admission
namespace: ingress-nginx
spec:
ports:
– appProtocol: https
name: https-webhook
port: 443
targetPort: webhook
selector:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
type: ClusterIP
—
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.9.5
name: ingress-nginx-controller
namespace: ingress-nginx
spec:
minReadySeconds: 0
revisionHistoryLimit: 10
selector:
matchLabels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
strategy:
rollingUpdate:
maxUnavailable: 1
type: RollingUpdate
template:
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.9.5
spec:
hostNetwork: true
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
– matchExpressions:
– key: kubernetes.io/ingress
operator: In
values:
– nginx
containers:
– args:
– /nginx-ingress-controller
– –publish-service=$(POD_NAMESPACE)/ingress-nginx-controller
– –election-id=ingress-nginx-leader
– –controller-class=k8s.io/ingress-nginx
– –ingress-class=nginx
– –configmap=$(POD_NAMESPACE)/ingress-nginx-controller
– –validating-webhook=:8443
– –validating-webhook-certificate=/usr/local/certificates/cert
– –validating-webhook-key=/usr/local/certificates/key
env:
– name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
– name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
– name: LD_PRELOAD
value: /usr/local/lib/libmimalloc.so
image: registry.cn-hangzhou.aliyuncs.com/google_containers/nginx-ingress-controller:v1.9.5
imagePullPolicy: IfNotPresent
lifecycle:
preStop:
exec:
command:
– /wait-shutdown
livenessProbe:
failureThreshold: 5
httpGet:
path: /healthz
port: 10254
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
name: controller
ports:
– containerPort: 80
name: http
protocol: TCP
– containerPort: 443
name: https
protocol: TCP
– containerPort: 8443
name: webhook
protocol: TCP
readinessProbe:
failureThreshold: 3
httpGet:
path: /healthz
port: 10254
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
resources:
requests:
cpu: 100m
memory: 90Mi
securityContext:
allowPrivilegeEscalation: false
capabilities:
add:
– NET_BIND_SERVICE
drop:
– ALL
readOnlyRootFilesystem: false
runAsNonRoot: true
runAsUser: 101
seccompProfile:
type: RuntimeDefault
volumeMounts:
– mountPath: /usr/local/certificates/
name: webhook-cert
readOnly: true
dnsPolicy: ClusterFirst
# nodeSelector:
# kubernetes.io/os: linux
serviceAccountName: ingress-nginx
terminationGracePeriodSeconds: 300
volumes:
– name: webhook-cert
secret:
secretName: ingress-nginx-admission
—
apiVersion: batch/v1
kind: Job
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.9.5
name: ingress-nginx-admission-create
namespace: ingress-nginx
spec:
template:
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.9.5
name: ingress-nginx-admission-create
spec:
containers:
– args:
– create
– –host=ingress-nginx-controller-admission,ingress-nginx-controller-admission.$(POD_NAMESPACE).svc
– –namespace=$(POD_NAMESPACE)
– –secret-name=ingress-nginx-admission
env:
– name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
image: registry.cn-hangzhou.aliyuncs.com/google_containers/kube-webhook-certgen:v20230312-helm-chart-4.5.2-28-g66a760794
imagePullPolicy: IfNotPresent
name: create
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop:
– ALL
readOnlyRootFilesystem: true
runAsNonRoot: true
runAsUser: 65532
seccompProfile:
type: RuntimeDefault
nodeSelector:
kubernetes.io/os: linux
restartPolicy: OnFailure
serviceAccountName: ingress-nginx-admission
—
apiVersion: batch/v1
kind: Job
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.9.5
name: ingress-nginx-admission-patch
namespace: ingress-nginx
spec:
template:
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.9.5
name: ingress-nginx-admission-patch
spec:
containers:
– args:
– patch
– –webhook-name=ingress-nginx-admission
– –namespace=$(POD_NAMESPACE)
– –patch-mutating=false
– –secret-name=ingress-nginx-admission
– –patch-failure-policy=Fail
env:
– name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
image: registry.cn-hangzhou.aliyuncs.com/google_containers/kube-webhook-certgen:v20230312-helm-chart-4.5.2-28-g66a760794
imagePullPolicy: IfNotPresent
name: patch
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop:
– ALL
readOnlyRootFilesystem: true
runAsNonRoot: true
runAsUser: 65532
seccompProfile:
type: RuntimeDefault
nodeSelector:
kubernetes.io/os: linux
restartPolicy: OnFailure
serviceAccountName: ingress-nginx-admission
—
apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.9.5
name: nginx
spec:
controller: k8s.io/ingress-nginx
—
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingWebhookConfiguration
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.9.5
name: ingress-nginx-admission
webhooks:
– admissionReviewVersions:
– v1
clientConfig:
service:
name: ingress-nginx-controller-admission
namespace: ingress-nginx
path: /networking/v1/ingresses
failurePolicy: Fail
matchPolicy: Equivalent
name: validate.nginx.ingress.kubernetes.io
rules:
– apiGroups:
– networking.k8s.io
apiVersions:
– v1
operations:
– CREATE
– UPDATE
resources:
– ingresses
sideEffects: None
注意看下边的效果：
apply之后是pedding之后，因为没有标签不满足亲和性，所以是pedding状态。但是一旦打了标签之后，立刻就开始部署了。
[root@k8s-node1 ingress]# kubectl get pod -o wide -A |grep ingress
ingress-nginx ingress-nginx-admission-create-tfncr 0/1 Completed 0 7m49s 10.244.122.108 k8s-node4 <none> <none>
ingress-nginx ingress-nginx-admission-patch-78cpn 0/1 Completed 0 7m49s 10.244.122.109 k8s-node4 <none> <none>
ingress-nginx ingress-nginx-controller-7469fb79f-7vrjw 0/1 Pending 0 9m49s <none> <none> <none> <none>
ingress-nginx ingress-nginx-controller-c954cf69-spssp 0/1 Terminating 12 (90m ago) 47d 10.244.169.146 k8s-node2 <none> <none>
[root@k8s-node1 ingress]# kubectl label nodes k8s-node2 kubernetes.io/ingress=nginx
node/k8s-node2 labeled
[root@k8s-node1 ingress]# kubectl get pod -o wide -A |grep ingress
ingress-nginx ingress-nginx-admission-create-tfncr 0/1 Completed 0 8m22s 10.244.122.108 k8s-node4 <none> <none>
ingress-nginx ingress-nginx-admission-patch-78cpn 0/1 Completed 0 8m22s 10.244.122.109 k8s-node4 <none> <none>
ingress-nginx ingress-nginx-controller-7469fb79f-7vrjw 0/1 ContainerCreating 0 10m 192.168.67.129 k8s-node2 <none> <none>
[root@k8s-node1 ingress]# kubectl get pod -o wide -A |grep ingress
ingress-nginx ingress-nginx-admission-create-tfncr 0/1 Completed 0 8m46s 10.244.122.108 k8s-node4 <none> <none>
ingress-nginx ingress-nginx-admission-patch-78cpn 0/1 Completed 0 8m46s 10.244.122.109 k8s-node4 <none> <none>
ingress-nginx ingress-nginx-controller-7469fb79f-7vrjw 1/1 Running 0 10m 192.168.67.129 k8s-node2 <none> <none>
[root@k8s-node1 ingress]#
4：查看效果
5：修改为多个副本
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.9.5
name: ingress-nginx-controller
namespace: ingress-nginx
spec:
# 修改为多个副本
replicas: 3
minReadySeconds: 0
revisionHistoryLimit: 10
selector:
matchLabels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
strategy:
rollingUpdate:
maxUnavailable: 1
type: RollingUpdate
[root@k8s-node1 pod]# kubectl get pods -A -o wide | grep ingress
ingress-nginx ingress-nginx-controller-7469fb79f-7vrjw 1/1 Running 0 31m 192.168.67.129 k8s-node2 <none> <none>
ingress-nginx ingress-nginx-controller-7469fb79f-mtj54 1/1 Running 5 (115s ago) 3m33s 192.168.67.131 k8s-node4 <none> <none>
ingress-nginx ingress-nginx-controller-7469fb79f-sjtcp 1/1 Running 0 10m 192.168.67.130 k8s-node3 <none> <none>
[root@k8s-node1 pod]#
以上三个，通过宿主机的ip地址进行访问的时候，还是没有问题的。非常棒！
所以，现在又给我们提供了Ingress 两种的方式。ingress的Pod直接暴露出来，或者Ingress 暴露出来Service进行统一入口。
6：注意
如果一个node80端口已经被一个Pod占用（仅仅宿主机模式），那么部署的时候Pedding根本过不去。会一直报错。最终是error，必须挪走一个Pod
四：hostnetwork知识内容补充
1：声明Pod资源文件
apiVersion: v1
kind: Pod
metadata:
name: nginx-hostnetwork
spec:
hostNetwork: true
containers:
– name: nginx-web
image: nginx:latest
imagePullPolicy: IfNotPresent
lifecycle:
postStart:
exec:
command:
– "/bin/sh"
– "-c"
– "echo 'hello hook' > /usr/share/nginx/html/index.html"
preStop:
exec:
command: ["/bin/sh", "-c", "nginx -s stop"]
2：验证结果
[root@k8s-node1 pod]# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
my-nginx-654f997cd5-jtf5q 1/1 Running 1 (31m ago) 3d6h 10.244.107.253 k8s-node3 <none> <none>
my-nginx-654f997cd5-qbsrz 1/1 Running 1 (31m ago) 3d6h 10.244.122.107 k8s-node4 <none> <none>
nginx-hostnetwork 1/1 Running 0 11s 192.168.67.131 k8s-node4 <none> <none>
[root@k8s-node1 pod]#
仅仅主机网络的话，相当于直接可以通过主机ip地址+监听端口好访问到容器。
访问结果：
3：查看宿主机接口监听情况
[root@k8s-node4 ~]# netstat -tulnp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 127.0.0.1:10248 0.0.0.0:* LISTEN 2214/kubelet
tcp 0 0 127.0.0.1:10249 0.0.0.0:* LISTEN 2506/kube-proxy
tcp 0 0 127.0.0.1:9098 0.0.0.0:* LISTEN 2559/calico-typha
tcp 0 0 127.0.0.1:9099 0.0.0.0:* LISTEN 2873/calico-node
tcp 0 0 0.0.0.0:111 0.0.0.0:* LISTEN 733/rpcbind
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 8825/nginx: master
tcp 0 0 0.0.0.0:179 0.0.0.0:* LISTEN 3087/bird
tcp 0 0 127.0.0.1:33299 0.0.0.0:* LISTEN 1110/containerd
tcp 0 0 192.168.122.1:53 0.0.0.0:* LISTEN 1525/dnsmasq
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 1088/sshd
tcp 0 0 127.0.0.1:631 0.0.0.0:* LISTEN 1092/cupsd
tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN 1450/master
tcp 0 0 127.0.0.1:6010 0.0.0.0:* LISTEN 10260/sshd: root@pt
tcp6 0 0 :::5473 :::* LISTEN 2559/calico-typha
tcp6 0 0 :::10250 :::* LISTEN 2214/kubelet
tcp6 0 0 :::111 :::* LISTEN 733/rpcbind
tcp6 0 0 :::80 :::* LISTEN 8825/nginx: master
tcp6 0 0 :::10256 :::* LISTEN 2506/kube-proxy
tcp6 0 0 :::22 :::* LISTEN 1088/sshd
tcp6 0 0 ::1:631 :::* LISTEN 1092/cupsd
tcp6 0 0 ::1:25 :::* LISTEN 1450/master
tcp6 0 0 ::1:6010 :::* LISTEN 10260/sshd: root@pt
udp 0 0 0.0.0.0:60131 0.0.0.0:* 716/avahi-daemon: r
udp 0 0 0.0.0.0:871 0.0.0.0:* 733/rpcbind
udp 0 0 0.0.0.0:5353 0.0.0.0:* 716/avahi-daemon: r
udp 0 0 192.168.122.1:53 0.0.0.0:* 1525/dnsmasq
udp 0 0 0.0.0.0:67 0.0.0.0:* 1525/dnsmasq
udp 0 0 0.0.0.0:111 0.0.0.0:* 733/rpcbind
udp 0 0 127.0.0.1:323 0.0.0.0:* 731/chronyd
udp6 0 0 :::871 :::* 733/rpcbind
udp6 0 0 :::111 :::* 733/rpcbind
udp6 0 0 ::1:323 :::* 731/chronyd
[root@k8s-node4 ~]#
这样的话，就可以不通过service直接访问你的Pod。
