Skip to content

Docker Registry

This document shows how to deploy an on-prem private Docker Registry on Kubernetes, using Helm.

Prerequisites

  • Kubernetes cluster accessible with kubectl CLI
  • Helm CLI
  • CFSSL CLI (at least cfssl and cfssljson).
  • htpasswd CLI
  • Nginx Ingress Controller installed in the cluster
  • Some LB (e.g. HAProxy) is set up to balance registry.<BASE_DOMAIN> to cluster Nginx Ingress Controller load balancer.

Generate TLS certs

Create a working directory $HOME/docker-registry:

mkdir $HOME/docker-registry
cd $HOME/docker-registry

Get the Docker Registry Helm repository:

helm repo add twuni https://twuni.github.io/docker-registry.helm
helm repo update twuni

Create a script registry-tls.sh that will generate TLS CA, certificate and corresponding secrets:

touch registry-tls.sh

Put the following content:

#!/bin/bash
set -e

#############
## setup environment
NAMESPACE=${NAMESPACE:-docker-registry}
RELEASE=${RELEASE:-registry}
DOMAIN=${DOMAIN:-apps.k8s.example.com}
## stop if variable is unset beyond this point
set -u
## known expected patterns for SAN
CERT_SANS="*.${RELEASE}.${DOMAIN},*.${DOMAIN}"

#############
## generate default CA config
cfssl print-defaults config > ca-config.json
## generate a CA
echo '{"CN":"'${RELEASE}.${DOMAIN}.ca'","key":{"algo":"ecdsa","size":256}}' | \
  cfssl gencert -initca - | \
  cfssljson -bare ca -
## generate certificate
echo '{"CN":"'${RELEASE}.${DOMAIN}'","key":{"algo":"ecdsa","size":256}}' | \
  cfssl gencert -config=ca-config.json -ca=ca.pem -ca-key=ca-key.pem -profile www -hostname="${CERT_SANS}" - |\
  cfssljson -bare ${RELEASE}

#############
## load certificates into K8s
kubectl delete secret ${RELEASE}-tls -n ${NAMESPACE}
kubectl delete secret ${RELEASE}-tls-ca -n ${NAMESPACE}
kubectl -n ${NAMESPACE} create secret tls ${RELEASE}-tls \
  --cert=${RELEASE}.pem \
  --key=${RELEASE}-key.pem
kubectl -n ${NAMESPACE} create secret generic ${RELEASE}-tls-ca \
  --from-file=${RELEASE}.${DOMAIN}.crt=ca.pem

Run the script, provide a valid base DOMAIN e.g. apps.k8s.example.com:

chmod u+x registry-tls.sh
export DOMAIN="<YOUR_DOMAIN>"
./registry-tls.sh

Configure and Deploy Helm Chart

Generate default user and password for registry, provide valid USERNAME and PASSWORD:

htpasswd -Bbn ${USERNAME} ${PASSWORD} > .htpasswd

Install Helm Chart:

helm upgrade --install registry twuni/docker-registry -n docker-registry --create-namespace --set secrets.htpasswd=$(cat .htpasswd)

Configure Ingress

Create Ingress file, provide valid TLS hosts and rule host:

# $HOME/docker-registry/ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: registry-ingress
  namespace: docker-registry
  annotations:
    nginx.ingress.kubernetes.io/proxy-body-size: "0"
    nginx.ingress.kubernetes.io/proxy-read-timeout: "600"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "600"
spec:
  ingressClassName: nginx
  tls:
  - hosts:
    - registry.apps.k8s.example.com # change me
    secretName: registry-tls
  rules:
  - host: registry.apps.k8s.example.com # change me
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: registry-docker-registry
            port:
              number: 5000

Apply ingress configuration:

kubectl apply -f ingress.yaml

Optional: Registry access from external K8s cluster

On each worker node of your K8s cluster that will need access to this private registry, put the content of your ca.pem in a file /usr/local/share/ca-certificates/private-docker-registry.crt and restart your container runtime e.g. on Ubuntu with docker runtime:

sudo su -
cat <<EOF > /usr/local/share/ca-certificates/private-docker-registry.crt
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
EOF
sudo update-ca-certificates
sudo systemctl restart docker