GitLab

Install GitLab on Kubernetes using your existing ingress-nginx and cert-manager. Promethus coming soon.

Helm

Install Helm:

wget https://get.helm.sh/helm-v3.13.3-linux-amd64.tar.gz
tar -xvf helm-v3.13.3-linux-amd64.tar.gz
sudo mv linux-amd64/helm /usr/local/bin/helm
rm -rf linux-amd64
rm helm-v3.6.3-linux-amd64.tar.gz

Metrics-server

Metrics server is used by pod autoscalers. GitLab uses this to scale up or down the number of running pods based on utilization.

wget https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml

Edit components.yaml and add the following line:

--kubelet-insecure-tls

Then apply the yaml:

kubectl apply -f components.yaml

NFS-client-subdir-provisioner

Add the nfs-subdir-external-provisioner helm chart for persistent storage:

helm repo add nfs-subdir-external-provisioner https://kubernetes-sigs.github.io/nfs-subdir-external-provisioner

Install into the kube-system namespace:

helm install nfs-client-provisioner \
    -n kube-system \
    --set nfs.server=192.168.1.201 \
    --set nfs.path=/mnt/tank/kubernetes \
    --set storageClass.defaultClass=true \
    --set storageClass.provisionerName=nfs-client-provisioner \
    nfs-subdir-external-provisioner/nfs-subdir-external-provisioner

Configure the nfs-client-provisioner as the default storage class

cat << EOF > sc.yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  annotations:
    storageclass.kubernetes.io/is-default-class: "true"
  labels:
    app: nfs-subdir-external-provisioner
  name: nfs-client
parameters:
  archiveOnDelete: "true"
provisioner: nfs-client-provisioner
reclaimPolicy: Delete
volumeBindingMode: Immediate
EOF

kubectl apply -f sc.yaml

Install the nfs-client package for your OS. For Ubuntu this is nfs-common.

apt install nfs-common

Redis

Install the bitnami helm chart:

helm repo add bitnami https://charts.bitnami.com/bitnami

Create the values file:

cat << EOF > redis-values.yaml
global:
  storageClass: nfs-client
architecture: standalone
auth:
  enabled: true
  password: redispass
metrics:
  enabled: true
EOF

Install Redis:

helm --create-namespace -n redis install redis bitnami/redis -f redis-values.yaml

MetalLB

Add the MetalLB helm chart:

helm repo add metallb https://metallb.github.io/metallb

Create a values file:

cat << EOF > metallb-values.yaml
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: ip-pool
  namespace: metallb
spec:
  addresses:
  - 192.168.1.197/32
---
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
  name: l2-config
  namespace: metallb
spec:
  ipAddressPools:
  - ip-pool
EOF

Then install into the metallb-system namespace

helm --create-namespace -n metallb-system install metallb metallb/metallb -f metallb-values.yaml

Ingress-nginx

Add the ingress-nginx helm chart:

helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx

Create a values file:

cat << EOF > ingress-nginx-values.yaml
controller:
  metrics:
    enabled: true
  service:
    annotations:
      prometheus.io/scrape: "true"
      prometheus.io/port: "10254"
tcp:
  22: "gitlab/gitlab-gitlab-shell:22"
EOF

The tcp line tells ingress-nginx we’re going to do some tcp load balancing for port 22. This is for gitlab-shell so we can use git+ssh.

Install into the ingress-nginx namespace:

helm --create-namespace -n ingress-nginx install ingress-nginx ingress-nginx/ingress-nginx -f ingress-nginx-values.yaml

Cert-manager

Add the jetstack helm chart:

helm repo add jetstack https://charts.jetstack.io

Create a values file:

cat << EOF > cert-manager-values.yaml
installCRDs: true
EOF

Install into the cert-manager namespace:

helm --create-namespace -n cert-manager install cert-manager jetstack/cert-manager -f cert-manager-values.yaml

Create your issuers, in my case I’m using ClusterIssuer with Lets Encrypt and DigitalOcean DNS.

Note

When using a DNS solver cert-manager can get wildcard certs. This is useful for gitlab pages, ie: *.pages.pwned.com

cat << EOF > cluster-issuer.yaml
kind: ClusterIssuer
metadata:
  name: letsencrypt-staging
spec:
  acme:
    email: chris@pwned.com
    server: https://acme-staging-v02.api.letsencrypt.org/directory
    privateKeySecretRef:
      name: letsencrypt-staging
    solvers:
    - dns01:
        digitalocean:
          tokenSecretRef:
            name: digitalocean-dns
            key: access-token
---
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-production
spec:
  acme:
    email: chris@pwned.com
    server: https://acme-v02.api.letsencrypt.org/directory
    privateKeySecretRef:
      name: letsencrypt-production
    solvers:
    - dns01:
        digitalocean:
          tokenSecretRef:
            name: digitalocean-dns
            key: access-token
EOF

And apply to the cluster:

kubectl create -f cluster-issuer.yaml

GitLab

Add the gitlab helm chart:

helm repo add gitlab https://charts.gitlab.io/

Create a values file:

cat << EOF > gitlab-values.yaml
postgresql:
  install: false
prometheus:
  install: false
redis:
  install: false
certmanager:
  install: false
nginx-ingress:
  enabled: false
gitlab-runner:
  install: false
gitlab:
  gitlab-pages:
    ingress:
      tls:
        secretName: gitlab-pages-tls
    resources:
      requests:
        cpu: 100m
        memory: 200M
  kas:
    ingress:
      tls:
        secretName: gitlab-kas-tls
  sidekiq:
    resources:
      requests:
        cpu: 50m
        memory: 650M
  webservice:
    ingress:
      tls:
        secretName: gitlab-web-tls
    resources:
      requests:
        cpu: 250m
        memory: 1.5G
global:
  hosts:
    domain: pwned.com
  kas:
    enabled: true
  pages:
    accessControl: true
    enabled: true
    host: pages.pwned.com
  ingress:
    annotations:
      cert-manager.io/cluster-issuer: letsencrypt-production
    class: nginx
    configureCertmanager: false
  redis:
    auth:
      enabled: true
    host: redis-master.redis.svc.cluster.local
  psql:
    host: pgdb1-rw.postgres.svc.cluster.local
    password:
      secret: gitlab-database-password
      key: dbpass
    database: gitlab
    username: gluser
registry:
  ingress:
    tls:
      secretName: gitlab-registry-tls
minio:
  ingress:
    tls:
      secretName: gitlab-minio-tls
EOF

Install gitlab into the gitlab namespace:

helm --create-namespace -n gitlab install gitlab gitlab/gitlab -f gitlab-values.yaml