Docker registry

Cover Image for Docker registry

Container registries are used for hosting containerized application images. It's highly desirable to run container registry locally for speed, cost and privacy reasons. K3s kubernetes cluster setup earlier is used to host docker container registry, with fedora 32 machine used for management.

Setting up manager.k3s.local

manager.k3s.local is setup using fedora 32 base image, /etc/rancher/k3s/k3s.yaml copied from master.k3s.local to manager.k3s.local:~/.kube/config. Server value in config is changed from to https://master.k3s.local:6443:

$ mkdir ~/.kube
$ scp -oStrictHostKeyChecking=no master:/etc/rancher/k3s/k3s.yaml ~/.kube/config
k3s.yaml                                     100% 1052   789.2KB/s   00:00
$ sed -i 's/server:.*/server: https:\/\/master.k3s.local:6443/g' ~/.kube/config
$ grep server: ~/.kube/config
    server: https://master.k3s.local:6443

After which, k3s is downloaded as /usr/local/bin/kubectl on manager.k3s.local:

$ sudo wget -q -O /usr/local/bin/kubectl
$ sudo chmod +x /usr/local/bin/kubectl
$ kubectl get node
INFO[0000] Preparing data dir /home/ansible/.rancher/k3s/data/31f16867327c45f309297f2846b7f471edba2b97c394cddb49d4e70ff7d41bcd
master   Ready    master   16m   v1.18.3+k3s1

Setting up helm

Helm is kubernetes package manager. It allows deploying various applications from charts which describe how given application is to be setup in kubernetes. Since v3 helm installation has been simplified namely be removing dependency on tiller. Helm can be installed using:

$ curl -sfL | sh -
Preparing to install helm into /usr/local/bin
helm installed into /usr/local/bin/helm

Once helm is installed, add stable chart repository:

$ helm repo add stable
"stable" has been added to your repositories

Installing docker registry

Docker registry installed using helm to kube-system namespace with data persistence class set to local-path:

$ helm install registry stable/docker-registry \
> --namespace kube-system \
> --set=persistence.enabled=true,persistence.storageClass=local-path
NAME: registry
LAST DEPLOYED: Fri Jun 19 02:12:38 2020
NAMESPACE: kube-system
STATUS: deployed
1. Get the application URL by running these commands:
export POD_NAME=$(kubectl get pods --namespace kube-system -l "app=docker-registry,release=registry" -o jsonpath="{.items[0]}")
echo "Visit to use your application"
kubectl -n kube-system port-forward $POD_NAME 8080:5000

Setting up ssl certs

To enable TLS on traefik load balancer, create self signed SSL key and cert:

$ openssl req -x509 -nodes -days 365 -newkey rsa:2048 \
> -keyout apps.local.key \
> -out apps.local.crt \
> -subj "/CN=*.apps.local"
Generating a RSA private key
writing new private key to 'apps.local.key'

Add newly created SSL key and cert to kube-system namespace:

$ kubectl create secret tls traefik-tls-cert \
> --namespace=kube-system \
> --key=apps.local.key \
> --cert=apps.local.crt
secret/traefik-tls-cert created

Remove the key from the filesystem and add certificate to system ca-trust bundle:

$ rm apps.local.key
$ sudo mv apps.local.crt /etc/pki/ca-trust/source/anchors/apps.local.crt
$ sudo update-ca-trust

Configuring traefik ingress controller

To access registry-docker-registry kubernetes service, ingress controller is configured to route http/https requests using traefik-tls-cert and matching hostname docker-registry.apps.local port 5000

$ cat /tmp/ingress.yaml
apiVersion: extensions/v1beta1
kind: Ingress
  name: docker-registry
  namespace: kube-system
  annotations: traefik
  - host: docker-registry.apps.local
      - backend:
          serviceName: registry-docker-registry
          servicePort: 5000
   - secretName: traefik-tls-cert
$ kubectl apply -f /tmp/ingress.yaml
ingress.extensions/docker-registry created

Test out new service using curl:

$ curl -iL https://docker-registry.apps.local/v2/ -iL
HTTP/2 200
content-type: application/json; charset=utf-8
date: Fri, 19 Jun 2020 02:14:01 GMT
docker-distribution-api-version: registry/2.0
vary: Accept-Encoding
x-content-type-options: nosniff
content-length: 2

Install docker

Docker package can be used for managing docker images locally, use dnf to install docker on fedora:

$ sudo dnf install docker -y
Last metadata expiration check: 0:19:04 ago on Fri 19 Jun 2020 01:55:06 AM UTC.
Dependencies resolved.
 Package            Arch    Version                             Repo      Size
 moby-engine        x86_64  19.03.8-2.ce.gitafacb8b.fc32        updates   51 M
Installing dependencies:
 container-selinux  noarch  2:2.135.0-1.fc32                    updates   47 k
 containerd         x86_64  1.3.3-1.fc32                        updates   32 M
 libbsd             x86_64  0.10.0-2.fc32                       fedora   106 k
 libnet             x86_64  1.1.6-19.fc32                       fedora    64 k
 protobuf-c         x86_64  1.3.2-2.fc32                        fedora    35 k
 runc               x86_64     fedora   2.7 M
Installing weak dependencies:
 criu               x86_64  3.14-1.fc32                         updates  507 k

Transaction Summary
Install  8 Packages

Total size: 86 M
Installed size: 314 M
Downloading Packages:
[SKIPPED] container-selinux-2.135.0-1.fc32.noarch.rpm: Already downloaded
[SKIPPED] containerd-1.3.3-1.fc32.x86_64.rpm: Already downloaded
[SKIPPED] criu-3.14-1.fc32.x86_64.rpm: Already downloaded
[SKIPPED] moby-engine-19.03.8-2.ce.gitafacb8b.fc32.x86_64.rpm: Already downloaded
[SKIPPED] libbsd-0.10.0-2.fc32.x86_64.rpm: Already downloaded
[SKIPPED] libnet-1.1.6-19.fc32.x86_64.rpm: Already downloaded
[SKIPPED] protobuf-c-1.3.2-2.fc32.x86_64.rpm: Already downloaded
[SKIPPED] Already downloaded
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Preparing        :                                                       1/1
  Running scriptlet: container-selinux-2:2.135.0-1.fc32.noarch             1/8
  Installing       : container-selinux-2:2.135.0-1.fc32.noarch             1/8
  Running scriptlet: container-selinux-2:2.135.0-1.fc32.noarch             1/8
  Installing       : protobuf-c-1.3.2-2.fc32.x86_64                        2/8
  Installing       : libnet-1.1.6-19.fc32.x86_64                           3/8
  Installing       : libbsd-0.10.0-2.fc32.x86_64                           4/8
  Installing       : criu-3.14-1.fc32.x86_64                               5/8
  Installing       :           6/8
  Installing       : containerd-1.3.3-1.fc32.x86_64                        7/8
  Running scriptlet: containerd-1.3.3-1.fc32.x86_64                        7/8
  Running scriptlet: moby-engine-19.03.8-2.ce.gitafacb8b.fc32.x86_64       8/8
  Installing       : moby-engine-19.03.8-2.ce.gitafacb8b.fc32.x86_64       8/8
  Running scriptlet: moby-engine-19.03.8-2.ce.gitafacb8b.fc32.x86_64       8/8
Created symlink /etc/systemd/system/ → /usr/lib/systemd/system/docker.socket.

  Running scriptlet: container-selinux-2:2.135.0-1.fc32.noarch             8/8
  Running scriptlet: moby-engine-19.03.8-2.ce.gitafacb8b.fc32.x86_64       8/8
  Verifying        : container-selinux-2:2.135.0-1.fc32.noarch             1/8
  Verifying        : containerd-1.3.3-1.fc32.x86_64                        2/8
  Verifying        : criu-3.14-1.fc32.x86_64                               3/8
  Verifying        : moby-engine-19.03.8-2.ce.gitafacb8b.fc32.x86_64       4/8
  Verifying        : libbsd-0.10.0-2.fc32.x86_64                           5/8
  Verifying        : libnet-1.1.6-19.fc32.x86_64                           6/8
  Verifying        : protobuf-c-1.3.2-2.fc32.x86_64                        7/8
  Verifying        :           8/8


$ sudo systemctl start docker

Using local registry

Docker image can be created locally or downloaded from docker hub:

$ sudo docker pull alpine
Using default tag: latest
latest: Pulling from library/alpine
df20fa9351a1: Pull complete
Digest: sha256:185518070891758909c9f839cf4ca393ee977ac378609f700f60a771a2dfe321
Status: Downloaded newer image for alpine:latest

Once image is on the local system, tag it with registry prefix:

$ sudo docker tag alpine docker-registry.apps.local/demo-app

Tagged image can now be pushed up to the registry:

$ sudo docker push docker-registry.apps.local/demo-app
The push refers to repository [docker-registry.apps.local/demo-app]
50644c29ef5a: Pushed
latest: digest: sha256:a15790640a6690aa1730c38cf0a440e2aa44aaca9b0e8931a9f2b0d7cc90fd65 size: 528

To download the image from the new registry, the local images needs to be deleted first:

$ sudo docker rmi alpine docker-registry.apps.local/demo-app
sudo Untagged: alpine:latest
Untagged: alpine@sha256:185518070891758909c9f839cf4ca393ee977ac378609f700f60a771a2dfe321
dUntagged: docker-registry.apps.local/demo-app:latest
Untagged: docker-registry.apps.local/demo-app@sha256:a15790640a6690aa1730c38cf0a440e2aa44aaca9b0e8931a9f2b0d7cc90fd65
Deleted: sha256:a24bb4013296f61e89ba57005a7b3e52274d8edd3ae2077d04395f806b63d83e
Deleted: sha256:50644c29ef5a27c9a40c393a73ece2479de78325cae7d762ef3cdc19bf42dd0a
$ sudo docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE

Docker can now pull image from the docker-registry.apps.local:

$ sudo docker pull docker-registry.apps.local/demo-app
Using default tag: latest
latest: Pulling from demo-app
df20fa9351a1: Pull complete
Digest: sha256:a15790640a6690aa1730c38cf0a440e2aa44aaca9b0e8931a9f2b0d7cc90fd65
Status: Downloaded newer image for docker-registry.apps.local/demo-app:latest
$ sudo docker images
REPOSITORY                            TAG                 IMAGE ID            CREATED             SIZE
docker-registry.apps.local/demo-app   latest              a24bb4013296        2 weeks ago         5.57MB