This article is a guide to setup a Kubernetes cluster on a remote server running Ubuntu. The guide also includes the Kubernetes dashboard setup and access.

Firewall setup (Master Node)

Required ports for Master Node can be found here.

Firewall setup (Worker Nodes)

Required ports for Worker Nodes can be found here.

Disable swap (Master and Workers)

The kubelet is not designed to use SWAP memory therefore, it needs to be disabled.

sudo swapoff -a && sudo sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab

Install containerd (Master and Workers)

Kubernetes can use containerd directly as a container runtime, meaning Docker is not needed in this middle-man role.

cat <<EOF | sudo tee /etc/modules-load.d/containerd.conf
overlay
br_netfilter
EOF

sudo modprobe overlay && sudo modprobe br_netfilter

cat <<EOF | sudo tee /etc/sysctl.d/99-kubernetes-cri.conf
net.bridge.bridge-nf-call-iptables  = 1
net.ipv4.ip_forward                 = 1
net.bridge.bridge-nf-call-ip6tables = 1
EOF

sudo sysctl --system

sudo apt-get update

sudo apt-get install -y containerd

sudo mkdir -p /etc/containerd

sudo containerd config default | sudo tee /etc/containerd/config.toml

At the end of this section in /etc/containerd/config.toml change the value for SystemCgroup from false to true. More info here and here.

...
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
    ...
    SystemCgroup = true
    ...
...

sudo systemctl restart containerd

Install kubeadm toolbox (Master and Workers)

More info here.

sudo apt-get update

sudo apt-get install -y apt-transport-https ca-certificates curl gpg

curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.33/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg

sudo chmod 644 /etc/apt/keyrings/kubernetes-apt-keyring.gpg

echo 'deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.33/deb/ /' | sudo tee /etc/apt/sources.list.d/kubernetes.list

sudo apt-get update

sudo apt-get install -y kubelet kubeadm kubectl

sudo apt-mark hold kubelet kubeadm kubectl

Initialize kubeadm (Master Node)

More info here.

sudo kubeadm init --pod-network-cidr=10.244.0.0/16

sudo kubeadm init --pod-network-cidr=10.244.0.0/16 --control-plane-endpoint=<endpoint-ip-or-dns>:<port>

sudo kubeadm init --pod-network-cidr=10.244.0.0/16 --apiserver-advertise-address=<endpoint-ip-or-dns> --apiserver-cert-extra-sans=<endpoint-ip-or-dns>

--pod-network-cidr=10.244.0.0/16 is important to work with flannel.

mkdir -p $HOME/.kube

sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config

sudo chown $(id -u):$(id -g) $HOME/.kube/config

Apply kube-flannel.yml right after.

kubectl apply -f https://github.com/flannel-io/flannel/releases/latest/download/kube-flannel.yml

Install NGINX Ingress Controller (Master Node)

More info here.

kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.8.1/deploy/static/provider/baremetal/deploy.yaml

spec:
    template:
        spec:
            hostNetwork: true

Connect with the cluster (for Windows)

kubectl config view --flatten

curl.exe -LO "https://dl.k8s.io/release/v1.27.3/bin/windows/amd64/kubectl.exe"

apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: ....
    server: https://........
    name: kubernetes
contexts:
- context:
    cluster: kubernetes
    user: kubernetes-admin
    name: kubernetes-admin@kubernetes
current-context: kubernetes-admin@kubernetes
kind: Config
preferences: {}
users:
- name: kubernetes-admin
    user:
    client-certificate-data: ...
    client-key-data: ...

kubectl cluster-info

Control plane node isolation (Master Node)

This will allow pods on Master Node. This step is required to allow Kubernetes dashobard pods initialized on Master Node.

kubectl taint nodes --all node-role.kubernetes.io/control-plane-

Kubernetes dashboard (Master Node)

More info here.

kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.7.0/aio/deploy/recommended.yaml

apiVersion: v1
kind: ServiceAccount
metadata:
    name: admin-user
    namespace: kubernetes-dashboard
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
    name: admin-user
roleRef:
    apiGroup: rbac.authorization.k8s.io
    kind: ClusterRole
    name: cluster-admin
subjects:
    - kind: ServiceAccount
    name: admin-user
    namespace: kubernetes-dashboard

kubectl apply -f %UserProfile%\.kube\kubernetes-dashboard.yaml

kubectl -n kubernetes-dashboard create token admin-user --duration=48h

kubectl proxy

http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/#/login

Optional configurations

Enable nfs kernel on ubundu to be able to mount volumes.

sudo apt install nfs-kernel-server

Setup metrics server for basic metrics (might need to download and set --kubelet-insecure-tls on args to work).

kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml

Rename a node

• Remove node from cluster (if already joined).
• $kubeadm reset
• Set the --hostname-override arg on file /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
• $systemctl daemon-reload
• $kubeadm reset
• Re-join the node

Error on kubeadm join

If you get jwt error, get a new token on the master node using the following command and try again joining the node.

kubeadm token create --print-join-command

Remove kubernetes installation

kubeadm reset

sudo apt-get purge kubeadm kubectl kubelet kubernetes-cni kube*

sudo apt-get autoremove

sudo rm -rf ~/.kube