How to upgrade Kubernetes Cluster created with kubeadm

Last updated on November 22, 2023

Kubernetes has become the de facto standard for container orchestration, providing robust and scalable solutions for managing containerized applications. As the Kubernetes ecosystem evolves, it is crucial to keep your cluster up to date with the latest features, bug fixes, and security patches. In this blog post, we will explore the process of upgrading your Kubernetes cluster using Kubeadm, a popular tool for managing cluster lifecycle operations. We will walk you through the steps outlined in the official Kubernetes documentation’s “Kubeadm Upgrade” guide (version 1.27) to ensure a smooth and successful upgrade.

Step 1: Prerequisites:

Before diving into the upgrade process, it’s essential to review and ensure that your cluster meets the necessary prerequisites. This step includes verifying the compatibility of your current Kubernetes version, checking system requirements, and backing up any critical data or configuration files to safeguard against potential issues during the upgrade process.

For example, to check the current Kubernetes version, run the following command:

kubectl version

Step 2: Planning the Upgrade:

In this step, we’ll discuss the importance of planning your upgrade strategy. It involves understanding the upgrade paths, release notes, and any potential breaking changes. By carefully analyzing these aspects, you can make informed decisions about the target version and any additional considerations specific to your cluster’s configuration.

For example, to view the release notes for a specific version, you can use the following command:

kubeadm upgrade plan

Step 3: Performing the Upgrade:

With the groundwork laid, it’s time to proceed with the actual upgrade process. This step will guide you through the necessary commands and procedures involved in upgrading the control plane components and worker nodes. Let’s take a look at a few examples:

a. Upgrade Kubeadm itself:

 apt-mark unhold kubeadm && \
 apt-get update && apt-get install -y kubeadm=1.27.x-00 && \
 apt-mark hold kubeadm

kubectl drain <controlplane-node-name> --ignore-daemonsets

b. Upgrade the control plane:

sudo kubeadm upgrade apply v1.27.x

Uncordon the control plane node:

kubectl uncordon <node name>
kubectl uncordon master

c. Upgrade kubelet and kubectl:

kubectl drain <node-to-drain> --ignore-daemonsets

apt-mark unhold kubelet kubectl && \
apt-get update && apt-get install -y kubelet=1.27.x-00 kubectl=1.27.x-00 && \
apt-mark hold kubelet kubectl

Restart the kubelet:

sudo systemctl daemon-reload
sudo systemctl restart kubelet

Uncordon the node

Bring the node back online by marking it schedulable:

kubectl uncordon <node-to-uncordon>

Step 4: Verifying the Upgrade:

Once the upgrade process is complete, it is vital to validate that the cluster is functioning correctly and that all components are running as expected. We will explore various techniques and commands to ensure that your upgraded Kubernetes cluster is in a healthy state.

For example, to check the status of the control plane components, run the following command:

kubectl get nodes

kubectl get pods --all-namespaces

Step 5: Post-Upgrade Cleanup and Maintenance:

After a successful upgrade, there are a few additional steps to consider to ensure the optimal performance and stability of your cluster. We will discuss best practices for post-upgrade cleanup, including removing deprecated resources, updating configuration files, and performing any necessary maintenance tasks.

For example, to remove deprecated API resources, you can use the following command:

kubectl get <resource> -o name | xargs -n1 -I{} kubectl annotate {} kubectl.kubernetes.io/last-applied-configuration-

Step 6: Upgrade worker nodes:

The upgrade procedure on worker nodes should be executed one node at a time or few nodes at a time

kubectl get nodes

apt-mark unhold kubeadm && \
apt-get update && apt-get install -y kubeadm=1.27.0-00 && \
apt-mark hold kubeadm

kubectl drain <node-to-drain> --ignore-daemonsets

kubeadm upgrade node

apt-mark unhold kubelet kubectl && \
apt-get update && apt-get install -y kubelet=1.27.0-00 kubectl=1.27.0-00 && \
apt-mark hold kubelet kubectl

systemctl daemon-reload
systemctl restart kubelet

kubectl uncordon worker-node-01

kubectl get nodes

NAME         STATUS   ROLES          AGE    VERSION

node-2       Ready    <none>         219d   v1.27.0
master       Ready    control-plane  295d   v1.27.0
node-1       Ready    <none>         294d   v1.27.0
node-3       Ready    <none>         152d   v1.27.0

Upgrading your Kubernetes cluster is a crucial task that should be approached with careful planning and execution. By following the steps outlined in this blog post, based on the official Kubernetes documentation’s “Kubeadm Upgrade” guide, you can confidently navigate the upgrade process and keep your cluster up to date with the latest Kubernetes features and enhancements. Remember to always backup your data and configuration files before upgrading and thoroughly test your cluster after the upgrade to ensure a smooth transition. Stay on top of the evolving Kubernetes ecosystem and empower your organization to leverage the full potential of container orchestration.

references
https://v1-26.docs.kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade/