One of the most significant limitations of AWS Fargate is the inability to run containers in privileged mode. This limitation means Docker-in-Docker (DinD), which enables the building and running of container images inside of containers, does not work with the AWS Fargate Custom Executor driver for GitLab Runner. The good news is that users don’t have to be blocked by this limitation and may use a cloud-native approach to build containers, effectively leveraging a seamless integration with AWS CodeBuild in the CI/CD pipeline.

We provide in-depth instructions on how to autoscale GitLab CI on AWS Fargate in GitLab Runner’s documentation. In this blog post, we explain how to instrument CI containers and source repositories to trigger AWS CodeBuild and use it to build container images.

Architecture overview

 How distinct CI workloads run on Fargate.

The picture above illustrates distinct GitLab CI workloads running on Fargate. The container identified by ci-coordinator (001) is running a typical CI job which does not build containers, so it does not require additional configuration or dependencies. The second container, ci-coordinator (002), illustrates the problem to be tackled in this post: The CI container includes the AWS CLI in order to send content to an Amazon S3 Bucket, trigger the AWS CodeBuild job, and fetch logs.

Prerequisites

Once these prerequisites are configured, you can dive into the six-step process to configure CI containers and source repositories to trigger AWS CodeBuild and use it to build container images.

• The AWS Fargate Custom Executor driver for GitLab Runner must be set-up appropriately.
• Ensure the AWS IAM user permissions include the ability to create and configure S3 and CodeBuild resources.
• AWS IAM user or service role with permissions to upload files to S3, start CodeBuild jobs, and read CloudWatch Logs.
• AWS IAM user with permissions to create and configure IAM Policies and Users.

Step 1: Create an AWS S3 bucket

1. In the top menu of AWS Management Console click Services.
2. In the Storage section, select S3.
3. Click Create bucket.
4. Choose a descriptive name (ci-container-build-bucket will be used as example) and select your preferred region.
5. Leave all other fields with default values and click Create bucket.
6. In the Buckets list, click the name of the bucket you created.
7. Click Create folder.
8. Give it the gitlab-runner-builds name.
9. Click Save.

Step 2: Create an AWS CodeBuild Project

1. Using the AWS Console, click Services in the top menu
2. Select CodeBuild in the Developer Tools section
3. Click Create build project
4. In Project Name enter ci-container-build-project
5. In Source provider select Amazon S3
6. In Bucket select the ci-container-build-bucket created in step one
7. In S3 object key or S3 folder enter gitlab-runner-builds/build.zip
8. In Environment image, select Managed image
9. For Operating system select your preferred OS from the available options
10. For Runtime(s), choose Standard.
11. For Image, select aws/codebuild/standard:4.0
12. For Image version, select Always use the latest image for this runtime version
13. For Environment type select Linux
14. Check the Privileged flag
15. For the Service role select New service role and note the sugggested Role name
16. For Build specifications select Use a buildspec file
17. Scroll down to the bottom of the page and click “Create build project”

#aws #cloud computing #fargate custom executor