With the advent of lxc, docker and supporting technologies like cgroups and OverlayFS, various workloads began adopting container technologies. This shift has opened opportunities for container orchestration. After getting open sourced by Google, Kubernetes became dominant platform for container orchestration, application deployment, scaling, and management. Docker swarm is another open source container orchestrator with similar features.
K3s- certified highly available open source Kubernetes distribution is "5 less than k8s":
- Cloud providers and storage plugins removed.
- Uses sqlite3 storage engine.
- Packaged as a single binary.
- Comes with simple but powerful features.
- Minimal OS dependencies.
Setting up k3s
Since 31st release Fedora has switched to cgroups v2 and nftables since 32nd release. Both technologies are not supported in kubernetes and k3s yet.
To enable cgroups v1, add systemd.unified_cgroup_hierarchy=0 to kernel boot args using grubby tool:
# grubby --args="systemd.unified_cgroup_hierarchy=0" --update-kernel=ALL # reboot
To change the default firewalld backend, set FirewallBackend= value from nftables to iptables for /etc/firewalld/firewalld.conf:
# sed -i 's/FirewallBackend=.*/FirewallBackend=iptables/g' /etc/firewalld/firewalld.conf # systemctl restart firewalld
On selinux enabled machines, k3s will need selinux policy, it can be installed using:
# dnf install -y https://rpm.rancher.io/k3s-selinux-0.1.1-rc1.el7.noarch.rpm
k3s master setup
# curl -sfL https://get.k3s.io | sh - # k3s kubectl get node NAME STATUS ROLES AGE VERSION master Ready master 15m v1.18.2+k3s1
k3s api server is listening on TCP port 6443, firewall-cmd is used to allow connections from the worker node pool on 192.168.31.0/24 subnet:
# firewall-cmd --permanent --zone=public --add-rich-rule=' rule family="ipv4" source address="192.168.31.0/24" port protocol="tcp" port="6443" accept' # systemctl restart firewalld
k3s node setup
Node installation follows similar process with additional environment variables to pass in k3s master api url and secret node token
# SECRET_TOKEN=`ssh master.k3s.local -f 'sudo cat /var/lib/rancher/k3s/server/node-token'` # curl -sfL https://get.k3s.io | K3S_URL=https://master.k3s.local:6443 K3S_TOKEN=$SECRET_TOKEN sh -
Once k3s worker node setup is complete, it should start appearing on master:
# k3s kubectl get node NAME STATUS ROLES AGE VERSION worker1 Ready <none> 54m v1.18.2+k3s1 master Ready master 58m v1.18.2+k3s1