k3s setup

Cover Image for k3s setup
Tomas
Tomas

With the advent of lxc, docker and supporting technologies like cgroups and OverlayFS, various workloads began adopting container technologies. This shift has opened opportunities for container orchestration. After getting open sourced by Google, Kubernetes became dominant platform for container orchestration, application deployment, scaling, and management. Docker swarm is another open source container orchestrator with similar features.
K3s- certified highly available open source Kubernetes distribution is "5 less than k8s":

  • Cloud providers and storage plugins removed.
  • Uses sqlite3 storage engine.
  • Packaged as a single binary.
  • Comes with simple but powerful features.
  • Minimal OS dependencies.

Setting up k3s

Libvirt virtual machines are setup with qemu:///system hypervisor using Fedora base image for master and worker nodes.

Dependencies

Since 31st release Fedora has switched to cgroups v2 and nftables since 32nd release. Both technologies are not supported in kubernetes and k3s yet.

To enable cgroups v1, add systemd.unified_cgroup_hierarchy=0 to kernel boot args using grubby tool:

# grubby --args="systemd.unified_cgroup_hierarchy=0" --update-kernel=ALL
# reboot

To change the default firewalld backend, set FirewallBackend= value from nftables to iptables for /etc/firewalld/firewalld.conf:

# sed -i 's/FirewallBackend=.*/FirewallBackend=iptables/g' /etc/firewalld/firewalld.conf
# systemctl restart firewalld

On selinux enabled machines, k3s will need selinux policy, it can be installed using:

# dnf install -y https://rpm.rancher.io/k3s-selinux-0.1.1-rc1.el7.noarch.rpm

k3s master setup

Easiest way to setup k3s is to use install script from https://get.k3s.io: 

# curl -sfL https://get.k3s.io | sh -
# k3s kubectl get node
NAME      STATUS   ROLES    AGE     VERSION
master    Ready    master   15m     v1.18.2+k3s1

k3s api server is listening on TCP port 6443, firewall-cmd is used to allow connections from the worker node pool on 192.168.31.0/24 subnet:

# firewall-cmd --permanent --zone=public --add-rich-rule='
    rule family="ipv4"
    source address="192.168.31.0/24"
    port protocol="tcp" port="6443" accept' 
# systemctl restart firewalld

k3s node setup

Node installation follows similar process with additional environment variables to pass in k3s master api url and secret node token

# SECRET_TOKEN=`ssh master.k3s.local -f 'sudo cat /var/lib/rancher/k3s/server/node-token'`
# curl -sfL https://get.k3s.io | K3S_URL=https://master.k3s.local:6443 K3S_TOKEN=$SECRET_TOKEN sh -

Once k3s worker node setup is complete, it should start appearing on master:

# k3s kubectl get node
NAME      STATUS   ROLES    AGE   VERSION
worker1   Ready    <none>   54m   v1.18.2+k3s1
master    Ready    master   58m   v1.18.2+k3s1

back