Gitlab CI/CD pipeline

CI/CD seems to be all the rage these days, and you use a CI/CD pipeline to do it so I will build one in the Nerdhole. To do so, I need:

  • Gitlab - The piece of software that kicks off builds, tests, and installations.
  • Docker - A containerised application platform.

Like many of these things, the software lives on the Internet, and all I need to do is download it and install it. I have found the following instructions:

I think this installation needs a bit more oomph than my basic VM installation so I have created a new class of VM named "Heavy" with more memory and more CPU capacity.

The gitlab servers will be in the group gitlab_servers and that will be the target for the playbook. I may also have to add a few runner nodes, but that comes later.

Having worked on this for a while, I have to say that the Gitlab documentation is excellent!

Notes to self: - Make everything idempotent. - After this is running, check to see if we can just generate the final gitlab.rb and have it work.

Installing the software

This is how to install and configure Docker and Gitlab. As it turns out, the default package includes an NGINX and various other services that get set up automatically, so... that is very user friendly.

Installing Gitlab

I have to say the Gitlab people are doing a good job documenting their doings. Very refreshing.

URLs: - Omnibus GitLab Admin guide - Git blame for /etc/gitlab/gitlab.rb

Download the repo script and run it

Gitlab uses a shell script to set up its repos that you have to download from the Internet and run as root... Nice! So let's do that. It's only a VM after all...

So we get that from: https://packages.gitlab.com/install/repositories/gitlab/gitlab-ee/script.deb.sh"

I'm having the gitlab server download and execute it automatically using the shell directive.

Installing the software

With the repo enabled, a simple install gives us the following:

It looks like GitLab has not been configured yet; skipping the upgrade script.
         .                        ..
        :c:                      ,cc'
       :ccc:                    'cccc.
      :ccccc,                   cccccc
     .ccccccc                  :cccccc:
     cccccccc:                .cccccccc'
    :ccccccccc;..............'cccccccccc'
   cccccccccccccccccccccccccccccccccccccc'
  :ooolccccccccccccccccccccccccccccccllooo
  ooooooollccccccccccccccccccccccclooooooo
  ;ooooooooollcccccccccccccccccloooooooool
   oooooooooooolccccccccccccloooooooooooo.
    .ooooooooooooolcccccclloooooooooooo;
       cooooooooooooolllooooooooooooo.
          loooooooodxkkxddoooooooo.
            .ooodxkkkkkkkkxdooo;
               .kkkkkkkkkkkk:
                  ;kkkkkkx
                     :d

Thank you for installing GitLab!
GitLab was unable to detect a valid hostname for your instance.
Please configure a URL for your GitLab instance by setting `external_url`
configuration in /etc/gitlab/gitlab.rb file.
Then, you can start your GitLab instance by running the following command:
  sudo gitlab-ctl reconfigure

For a comprehensive list of configuration options please see the Omnibus GitLab readme
https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/README.md

Help us improve the installation experience, let us know how we did with a 1 minute survey:
https://gitlab.fra1.qualtrics.com/jfe/form/SV_6kVqZANThUQ1bZb?installation=omnibus&release=18-7


  Verifying        : gitlab-ee-18.7.0-ee.0.el9.x86_64                       1/1 

Installed:
  gitlab-ee-18.7.0-ee.0.el9.x86_64                                              

Complete!

Very artistic, and I'm sure sysadmins worldwide will be thrilled to see it.

The default /etc/gitlab/gitlab.rb file has only one uncommented line: external_url. The rest is basically templates and documentation. I approve! This will make it easy to generate my own config file.

I have now opened up the HTTP and HTTPS ports on the gitlab server, and I am getting a login prompt.

Enabling SSL

The installation instructions specify that you need to enable https only after you have done the initial install. You need to add a few lines to gitlab.rb and run the reconfigure script again. The steps are:

  • Remove the initial external_url line from gitlab.rb
  • Add the https external URL and disable let's encrypt.
  • Use openssl to create a new self-signed certificate.
  • Reconfigure gitlab.

And amazingly it works great! Am now connected to https!

After that, we need to copy the SSL certificate gitlab.nerdhole.me.uk.crt to various places so that Docker, Gitlab, and the OS itself may have private conversations:

  • /etc/docker/certs.d/gitlab.nerdhole.me.uk:5000/ca.crt - For Docker.
  • /etc/gitlab-runner/certs/gitlab.gitlab.nerdhole.me.crt - For the Gitlab runner
  • /etc/pki/ca-trust/source/anchors/gitlab.nerdhole.me.uk.crt - For the OS, things like curl.

This is taken care of in a few Ansible stanzas.

Enabling the Gitlab container registry

We want to do containers and for that we need a container registry and most likely a few runner nodes.

  • Add the registry_external_url line underneath the external_url line.
  • Reconfigure gitlab.

Ran it from Ansible and it seems to have worked.

Log in on your shiny new Gitlab server

This is a manual operation.

We will now point the web browser at the New Gitlab server and log in:

  • user: root
  • password: is in /etc/gitlab/initial_root_password

We may want to reset the root password. We put off making users, groups, and processes for now. Do change the password to something that doesn't disappear off the system in 24 hours...

Installing the Gitlab runner

The "Unknown issuer" message meant that the Gitlab certificate should be copied to the /etc/gitlab-runner/certs/ca.crt file. I am now getting a "Wrong token" error, which should be solved by putting in the needed token. You stupid moron! That copy command was specified loud and clear in the installation instructions! Really.

This is a manual part of the procedure. We need to create a record for a runner and then register that runner on the local runhost.

  • Log in as "root" in the Gitlab webgui.
  • Go to "CI/CD" and there choose "Runners"
  • Add a new one.
    • For now do not add any tags and let it run untagged jobs.
    • Use gitlab-runner in the runner description (mostly ornamental)
    • This will show you a long password-like registration token which you will need in the next step.
  • On the host, define the runner into Docker:
    • As root on the gitlab host, execute: docker run --rm -it -v /etc/gitlab-runner:/etc/gitlab-runner gitlab/gitlab-runner list
    • Enter the URL of the Gitlab server: http://gitlab.nerdhole.me.uk
    • Enter the registration token you got from Gitlab
    • At "Enter an executor:" enter "docker"
    • At "Enter the default Docker image" enter the default (ruby:3.3)
    • The runner is now defined.
  • Fine-tune the configuration:
    • Edit the file /etc/gitlab-runner/config.toml and add the block below to the gitlab-runner's stanza.
    volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/cache"]
    pull_policy = "if-not-present"

(Get rid of the original volumes line).

  • Start the runner making sure that Docker will always start it:
docker run -d \
    --restart always \
    --name gitlab-runner \
    -v /etc/gitlab-runner:/etc/gitlab-runner \
    -v /var/run/docker.sock:/var/run/docker.sock \
    gitlab/gitlab-runner:latest

And now we have a running Gitlab Runner, ready to accept jobs for us.

Disable Version Check and Service Ping

This is a daemon that keeps various parts of Gitlab up to date. I don't particularly care. I'll skip this step.

Disable runner version management

This is a process that kaaps currently-loaded runners up to date. I don't mind it doing that.

Configure NTP

My NTP is fine, and pool.ntp.org is available to all. So all is well.

Enabling the Package Metadata Database

This is a license and vulnerability scanner, which I don't want.

Enabling the Gitlab container registry

There are instructions on gitlab.com on how to set up Gitlab to allow machines in your network to push and pull containers.

Using Docker

Gitlab CI/CD can work with Docker - and I think also with OpenShift but let's not get too fancy.