Fuga Cloud Academy Tutorials How to setup GitLab CI/CD on Fuga Cloud

How to setup GitLab CI/CD on Fuga Cloud

Fuga Cloud

23/05/2018

For some features in GitLab you will have to implement an email service like MailGun, which is beyond the scope of this tutorial.

In this tutorial you will learn how to create a CI/CD pipeline that deploys to the Fuga Object Store using GitLab and GitLab runners. GitLab comes with built-in Continuous Integration, Continuous Deployment, and Continuous Delivery support to build, test, and deploy your application.

When a commit is made, a CI/CD pipeline will be started. GitLab will notify the runner that a new job is available. This runner will run all the tests defined for the project and will (re)deploy the application when the tests are successful. The flow looks like this:

Requirements

Followed the Fuga Getting Started guide.

Creating a new security group

To access GitLab by a browser outside of our local network, we need to open the ports 80 (HTTP) and 443 (HTTPS). This chapter will explain how to open these ports.

Log in to your Fuga Cloud dashboard if you’re not already logged in.
In your Fuga Dashboard go to Networking -> Security Groups or use this link.
Click on + Create Security Group to create a new security group.
Enter a name for the security group, for example webservers.
Click on Create Security Group to save the new group.
Click on the Manage Rules button from the newly created security group.
Click on + Add Rule in the top left corner of the screen.
From the Rule drop-down list select HTTP and click Add.
Repeat steps 6 and 7 for HTTPS.

Creating a Fuga Object Store container

This chapter will explain how to create a Fuga Object Store container which will be used for the deployment.

Log in to your Fuga Cloud dashboard if you’re not already logged in.
In your Fuga Dashboard go to Storage -> Object Store or use this link.
Create a new container using the orange + Container button.
Give it a fitting name and check the Public checkbox, so that we can access it without authentication later on. Please note that making your container Public allows everyone to access your files, you should change this after the tutorial.
Click on the + Create button to add the container.

Installing GitLab on a Fuga instance

In this section we’ll cover installing the Enterprise Editon (EE) of GitLab and setting up the root account. However there is also an open-source version of GitLab, the Community Edition (CE). If you want to use that version instead, change the commands with ee in them to ce. For example apt install gitlab-ee to apt install gitlab-ce.

Fire up an instance on Fuga that has enough performance for your needs. For this tutorial we will use a c2.large with 4 cores and 7.5GB of RAM. Take a look at the GitLab Requirements. Ubuntu 18.04 is used in this tutorial.
Click on Compute -> Instances or use this link.
In the row of the GitLab instance, click on the down arrow right of the Create Snapshot button and click on Edit Security groups
Our new security group should be on the left side of the popup under All Security Groups.
Click on the + button and the security group should move to the right.
Click Save to save the changes we made to our security group.
Log in to your instance using the following example command in your Terminal:
```
ssh ubuntu@IP_ADDRESS
```
For other examples of SSH follow the Getting Started guide (⁵⁄₅).
Start by updating your repositories by using the command:
```
sudo apt update
```
Install required packages for GitLab with the following command:
```
sudo apt install -y curl openssh-server ca-certificates
```

Add the GitLab repository:

curl https://packages.gitlab.com/install/repositories/gitlab/gitlab-ee/script.deb.sh | sudo bash

Now it’s time to install GitLab, make sure to change the EXTERNAL_URL to your domain or the floating IP from your instance. This would for example be: http://gitlab.example.com or if using an IP: http://185.XX.XX.XX.
```
sudo EXTERNAL_URL="http://gitlab.example.com" apt install gitlab-ee
```
When GitLab is done installing, browse to the EXTERNAL_URL with your browser and setup your account. For example http://gitlab.example.com. The username of the first account, is called root.

Installing a GitLab Runner that uses Docker

After installing GitLab on an instance, the next step is installing and registering a GitLab runner to GitLab. This section will cover the installation and registration of a GitLab runner.

Fire up another instance on Fuga that has enough performance for your GitLab runner. For the GitLab runner we will use a c2.small with Ubuntu 18.04.
Log in to your GitLab runner instance using the following example command:
```
$ ssh ubuntu@FLOATING_IP
```
For other examples of connecting with SSH see Getting Started guide (⁵⁄₅)
Start by updating your repositories by using the command:
```
$ sudo apt update
```

Install Docker with the following command:

$ sudo apt-get install \
apt-transport-https \
ca-certificates \
curl \
gnupg-agent \
software-properties-common

Add Docker’s official GPG key:

$ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

Verify that you now have the key with the fingerprint 9DC8 5822 9FC7 DD38 854A E2D8 8D81 803C 0EBF CD88, by searching for the last 8 characters of the fingerprint.
```
$ sudo apt-key fingerprint 0EBFCD88
```

Set up the stable repository:

$ sudo add-apt-repository \
"deb [arch=amd64] https://download.docker.com/linux/ubuntu \
$(lsb_release -cs) \
stable"

Update your repositories:
```
$ sudo apt-get update
```

Install the latest version of Docker CE and containerd

$ sudo apt-get install docker-ce docker-ce-cli containerd.io

Add the GitLab Runner repository:

$ curl -L https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.deb.sh | sudo bash

Now install the GitLab runner package:
```
$ sudo apt install gitlab-runner
```
Go to your GitLab URL with your browser and login with root and your chosen password.
Click on Configure GitLab.
Go to Overview -> Runners in the left menu bar. Keep this page open because we need the URL and registration token in the following steps.
Start the registration of the GitLab runner in the GitLab runner instance:
```
sudo gitlab-runner register
```
Now you have to enter some details about the runner, some of which you can copy/paste from YOUR_DOMAIN/admin/runners or YOUR_IP/admin/runners.
When asked for your GitLab URL, enter the URL of your GitLab. For example http://gitlab.example.com or http://185.xx.xx.xx.
When asked for a GitLab-ci token, enter the token on the page of step 9.
When asked for a description of your GitLab runner, enter a fitting description or leave empty.
When asked for tags for your GitLab runner, enter tags that you deem fitting or leave empty.
When asked to lock your GitLab runner, keep it default; true.
When asked for an Executor for your GitLab runner enter docker.
When asked for a default Docker image, enter nginx. You can choose anything you like but for this tutorial the nginx image is used.
The runner should have been registered successfully. Check your GitLab runners on your GitLab dashboard and it should look like this:

Fuga Object Store tokens

To connect to your Fuga Object Store, access- and secret tokens are required. If you already have these you can skip this section.

In Release 2 you can add an EC2/S3 user in two ways:

Using the dashboard
Using the OpenStack CLI

Using the Fuga dashboard to create an EC2/S3 user

In the dashboard, go to Account -> Access or use this link.
If you don’t have one yet, create OpenStack Credentials by pressing the big orange button.
Make sure to securely store your password, you will not be able to access the password again.
Now you can press the + Add EC2/S3 credentials link. This creates EC2/S3 credentials for you.

Using the OpenStack CLI to create an EC2/S3 user

To create and list API keys for the Object Store we need the Openstack CLI. If you haven’t installed it yet checkout our tutorials.
After following the tutorial for your operating system it’s time to list the credentials. We need the credentials to access our Object Store container in later steps.
To list all the credentials, enter the command:
```
$ openstack ec2 credentials list
```
If the list of credentials is empty, a new pair of credentials can be generated with:
```
$ openstack ec2 credentials create
```
Either copy the Access- and Secret key to the clipboard or save them temporarily

Setting up CI/CD

It’s now time to setup the CI/CD pipeline within GitLab. The tokens of the previous section will be used here.

To create a new project on your GitLab, click the plus icon located on the top of the screen next to the search bar and click New project.
Give it a name and click on Create project.
Open the new project and go to Settings -> CI/CD, which is located in the left menu bar.
Expand the Environment variables menu and enter your Access Key ID as ACCESS_TOKEN and your Secret Access Key as SECRET_TOKEN. Also enter your project ID as PROJECT_ID. The values for these keys should be the tokens from the Fuga Object Store tokens section. It should look like the picture below:
Click on Save variables.
Now it’s time to add an SSH key to your GitLab root account, so we can actually push and pull to our GitLab. In the top right corner of the screen click on your profile picture with the down arrow and select Settings.
Go to SSH Keys in the left menu bar.
Click on generate one or paste in your existing key, depending on if you already have a key or not. You can read the copy/paste your key by:
```
cat .ssh/id_rsa.pub
```
Click Add key.
Clone your new project to your system using the git link located on the project page:
```
git clone git@gitlab.example.com:root/your-project.git
```
Open your new git folder inside your terminal:
```
cd your-project
```
Create a new file that will contain our CI/CD settings:
```
touch .gitlab-ci.yml
```

Open the file in your favorite text editor and copy paste the following code inside:

  image: nginx
  stages:
    - test
    - deploy

  test:
    stage: test
    script:
        - echo "This is the test stage."

  deploy:
    stage: deploy
    script:
        - apt update
        - apt install -y s3cmd
        - s3cmd --quiet --no-check-certificate --access_key "$ACCESS_TOKEN" --secret_key "$SECRET_TOKEN" --host object.api.ams.fuga.cloud --host-bucket object.api.ams.fuga.cloud/$PROJECT_ID/%\(bucket\) --exclude ".git/*" put -r  ./ s3://YOUR_CONTAINER

Make sure to replace YOUR_CONTAINER with the name of your container made in the ‘Creating a Fuga Object Store container’ section. The last command will upload all the files from your git repository to your Fuga Object Store container.

Save the file and we’re now ready to test our CI/CD pipeline.

Testing the CI/CD pipeline

We will now test the CI/CD pipeline by creating a basic HTML file. If the pipeline works correctly we will see a live update.

Next to your .gitlab-ci.yml file, create a new file called index.html:
```
touch index.html
```

Open index.html with your favorite text editor and paste in the follow example HTML:

<html>
  <head>
    <title>CI/CD test</title>
  </head>
  <body>
    <h1>Hello, CI/CD!</h1>
  </body>
</html>

Add the .gitlab-ci.yml and index.html to your git staged changes:
```
git add .
```

Create a new git commit with these changes:

git commit -m "Initial commit with .gitlab-ci and test html page."

Push the changes to the GitLab:
```
git push
```
Go to your GitLab project and in the left menu bar click on CI/CD.
A list of pipelines should appear like below:
There should be two green check marks marking both of our CI/CD stages as successful.
Open a new tab in your webbrowser and go to https://object.api.ams.fuga.cloud/swift/v1/YOUR_PROJECT_ID/YOUR_CONTAINER/index.html. To find your Domain ID, go to your Account Details page. Replace YOUR_CONTAINER with the name of your own container.
The HTML that you committed to your Gitlab should be displaying now.
You now have a working GitLab CI/CD base to improve further on! Don’t forget to change your container to private again if you don’t want to give everyone access to your files.

Was this article helpful?

How to Autoscale your GitLab runners on OpenStack

Autoscaling your GitLab runners on Fuga has many advantages. For example, you can only have runners active during office hours or just when you need them. The unused runners will be removed automatically. Since Fuga Cloud is pay for what you use, the total costs for your GitLab CI/CD will go down. In this tutorial, I will explain how to add autoscaling GitLab runners to your GitLab installation using GitLab CI and Docker Machine.