K8S集群搭建过程中节点无法加入的问题

集群环境

Master 节点：

hostname = mgt
release = CentOS Linux release 7.9.2009
kubeadm = v1.22.1
kubectl = v1.22.1
docker  = build 7d71120/1.13.1

work节点

hostname = node01
release  = Ubuntu 20.04.3 LTS
kubeadm = v1.22.1
kubectl = v1.22.1
docker  = build 20.10.7-0ubuntu1~20.04.1

问题描述

在Master节点init完成后，使用kubeadm join命令加入集群失败，报错如下

root@node01:~# kubeadm join 192.168.1.xx:6443 --token zj10hx.yw70w9dchdtkej17 \
>         --discovery-token-ca-cert-hash sha256:4c0a7f13c55b7598a801a0838a7364034719940097d790e701081430213f472f
[preflight] Running pre-flight checks
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Starting the kubelet
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...
[kubelet-check] Initial timeout of 40s passed.

Unfortunately, an error has occurred:
    timed out waiting for the condition

This error is likely caused by:
    - The kubelet is not running
    - The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)

检查发现kubelet不在running状态，使用journalctl -u kubelet查到错误日志如下

Aug 26 00:00:21 node01 kubelet[57999]: E0826 00:00:21.826495   57999 server.go:294] "Failed to run kubelet" err="failed to run Kubelet: misconfiguration: kubelet cgroup driver: \"systemd\" is different from docker cgroup driver: \"cgroupfs\""
Aug 26 00:00:21 node01 systemd[1]: kubelet.service: Main process exited, code=exited, status=1/FAILURE

处理过程

查找kubenetes官方文档，在官方文档中提到，由于 kubeadm 把 kubelet 视为一个系统服务来管理，所以对基于 kubeadm 的安装，我们推荐使用 systemd 驱动，不推荐 cgroupfs 驱动。

https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/configure-cgroup-driver/

再结合报错信息，那么就应该要把docker的cgrop driver更换成为systemd

编写/etc/docker/daemon.json 文件，没有的话就新建一个，写入如下内容：

{
  "exec-opts": ["native.cgroupdriver=systemd"]
}

然后重启docker服务service docker restart,使用docker info |grep Cgroup 确认Cgroup Driver已经变成 systemd

随后，让node01重新加入master节点：

root@node01:~# kubeadm join 192.168.1.20:6443 --token zj10hx.yw70w9dchdtkej17 \
>         --discovery-token-ca-cert-hash sha256:4c0a7f13c55b7598a801a0838a7364034719940097d790e701081430213f472f
[preflight] Running pre-flight checks
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Starting the kubelet
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...

This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.

Run 'kubectl get nodes' on the control-plane to see this node join the cluster.

root@node01:~#

在Master节点上检查状态

[root@mgt ~]# kubectl get node
NAME     STATUS   ROLES                  AGE   VERSION
mgt      Ready    control-plane,master   16h   v1.22.1
node01   Ready    <none>                 97m   v1.22.1
[root@mgt ~]#

STATUS为Ready，成功加入集群