If you already have k3s cluster installed, you still need to solve storage class issue. Lets assume, you have 2 nodes cluster, and your workload was rescheduled to another node. To move the pod, and give it an access to its belongings, you need some sort of distributed storage system. One of the options is Longhorn. Among many advantages, here are my highlights:

• lightweight and easy to install. It makes a perfect match for k3s
• snapshots
• backups that can be sent to s3
• arm compatibility
• dynamic volume provisioning
• allows volume expansion
• read-write many
• metrics accesibie for prometheus
• clear and friendly UI
• community backed up by rancher
• resilence and disaster recovery. By default, each volume has two replicas (assuming 3+ nodes cluster)

Installing longhorn is quite easy. But first, we need to check couple of prerequisites. Required software for each of the nodes is:

• running iscsid
• installed open-iscsi
• NFS4 for RWX volumes

You can check it with below command:

# For AMD64 platform
curl -sSfL -o longhornctl https://github.com/longhorn/cli/releases/download/v1.7.2/longhornctl-linux-amd64
# For ARM platform
curl -sSfL -o longhornctl https://github.com/longhorn/cli/releases/download/v1.7.2/longhornctl-linux-arm64

./longhornctl check preflight

And even fix what is missing:

./longhornctl install preflight

Installing longhorn on your cluster is even easier. First, add longhorn repository:

helm repo add longhorn https://charts.longhorn.io
helm repo update

And install it in longhorn namespace:

helm install longhorn longhorn/longhorn --namespace longhorn-system --create-namespace --version 1.7.2

You can watch the progress:

watch kubectl get pods -n longhorn-system

I always find typing -n {{namespace}} after each command tedious. I highly recommend kns tool for switching kubectl context: https://github.com/blendle/kns . For those working with multiple clusters on daily basis, ktx would be nice addition too.

After installing longhorn, we need to make it default storage class for our cluster. Lets inspect:

kubectl get sc -o custom-columns="NAME:.metadata.name,DEFAULT:.metadata.annotations.storageclass\.kubernetes\.io/is-default-class"
NAME              DEFAULT
local-path        true
longhorn          true
longhorn-static   <none>

We need to set default to false for local-path. Here is the command generated by ChatGPT:

patch kubernetes storage class local-path, set storageclass.kubernetes.io/is-default-class to false

ChatGPT said:

To patch the local-path StorageClass and set storageclass.kubernetes.io/is-default-class to false, you can use the following kubectl command:

kubectl patch storageclass local-path -p '{"metadata": {"annotations": {"storageclass.kubernetes.io/is-default-class": "false"}}}'

Thank you AI

kubectl get sc -o custom-columns="NAME:.metadata.name,DEFAULT:.metadata.annotations.storageclass\.kubernetes\.io/is-default-class"
NAME              DEFAULT
local-path        false
longhorn          true
longhorn-static   <none>