Docker containers may be deployed using one of the several cloud platforms, Amazon Elastic Container Service (ECS) being. AWS CodePipeline is a DevOps service for Continuous Integration, Continuous Delivery and Continuous Deployment of applications hosted on various AWS platforms.
Amazon Elastic Container Service (ECS) is an AWS managed service for containerized applications for Docker containers.
Amazon Fargate is a serverless launch type for Amazon Elastic Container Service (ECS).
For the example application deployed to ECS an AWS CodePipelineconsists of a source code repository such as a GitHub repo, AWS CodeBuild for Build and a AWS ECS (Fargate) service for Staging.
The benefit of using an AWS CodePipeline for an AWS ECS service is that the ECS service continues to run while a new Docker image is built and deployed.
A Docker container deployment may need to be updated or modified due to changes in the Docker image source code or code build. Any modifications in the source code for a Docker image would require that a Docker image be rebuilt and the Docker service be redeployed. Without a mechanism to integrate, build, deliver and deploy source code while an AWS ECS deployment is running, it would involve stopping the ECS service tasks and as a result, incurring downtime for an ECS service. High availability of an ECS service being a priority, stopping tasks to redeploy an application is not a suitable option.
AWS CodePipeline is a DevOps service for Continuous Integration, Continuous Delivery and Continuous Deployment of applications hosted on the various AWS platforms, including Amazon ECS and Fargate as a deployment platform. An ECS service may be updated or modified without stopping the ECS service tasks. AWS CodePipeline provides high availability of an ECS service in a dynamic environment in which source code changes for a Docker image are common. A CodePipeline consists of three phases: Source code integration, Source code build, and Deployment, as shown in Figure 1.
Figure 1. CodePipeline Phases
For source code we shall use a Github repository. For source code build we shall use a AWS CodeBuild project. For deployment we shall use an ECS service of launch type Fargate.
Creating and deploying a CodePipeline application to ECS Fargate involves the following procedure:
The only prerequisite is an AWS account. The application deployed by a CodePipeline on ECS Fargate is a Docker application. Any Docker image that has source code repo could be used and we have used Docker image dvohra/node-server
. Here is the GitHub source code repository for the Docker image dvohra/node-server.
If a new GitHub source code repository were to be used, it must include a Dockerfile from which to build Docker image. The Dockerfile for the dvohra/node-server
image is based on the Docker image node:4.6
. Dockerfile instructions copy a server.js
file, which is used to create a Node server, to the current directory, expose port 8080 for the Node server to listen on, and run a node command on the server.js script. The server.js file creates a Node server and handles an HTTP request/response.
A build spec is a YAML syntax file with build commands and settings used by a CodeBuild project to run a build. The build spec file must be called “buildspec.yml” and must be copied to the root of the source code repository. A buildspec.yml file consists of key/value pairs to describe the various phases of a build. The build phases
are represented with the phases sequence, which is a required mapping in a buildspec.yml
. The version
is the other required mapping in a buildspec.yml
. The buildspec.yml file is listed on the GitHub repo.
For deploying Container based applications such as those deployed to ECS, AWS CodePipeline requires an Image definitions file in JSON format. The Image definitions file is called imagedefinitions.json
by default but could be given another name. The image definitions file describes the container application and consists of two attributes: name and imageURI
. The name
specifies the Docker container name and the container must be running prior to running the CodePipeline. The imageURI
specifies the Docker image to be run in the Docker container. The Docker image would typically be the same as the Docker image already running in an ECS container. The image could be different and the variation would typically be of the image tag. The imagedefinitions.json used for the Node server application deployed to ECS Fargate is listed on the GitHub.
A task definition in an ECS application describes the container/s in the ECS deployment. In this section, we shall create task definition for a Node Server container to be deployed on ECS Fargate. Open this URL and log in if not already logged in.. Click on Get started to access the ECS Console. Click on Task Definitions in the navigation margin. In the Task Definitions click on Create new Task Definition. In the Create new Task Definition select launch type compatibility as Fargate. Click on Next step. Next, configure task and container definitions. In the Add container dialog specify a Container name (node-server) and specify Image as dvohra/node-server. The task definition is shown in Figure 2.
Figure 2. Task Definition
Before creating a service we need to configure connectivity to the Internet in the Subnets to be used when configuring the service. The Route Table lists the routes. We need to add a route with a default Internet gateway. Add a route with Destination as 0.0.0.0/0, which is an IP address. Select Target as an internet gateway.
Next, create an ECS container service in the default cluster as shown in Figure 3.
Figure 3. Cluster with 1 Service
With Fargate launch type an Elastic Network Interface (ENI) is provisioned for each task. Copy the Public IP of the task, which is the same as the Public IP of the Elastic Network Interface, from either the task Details page Network section or the ENI Console. Open the URL :8080 in a browser to invoke the Node Service application. The Node Server returns a message as shown in Figure 4.
Figure 4. Node Server Response
The CodePipeline to build the Source code for the Node server application and deploy a Docker image to an ECS service requires that the CodeBuild project generate “Output Artifacts”. The Output artifacts are stored in an S3 bucket, which is selected when creating a CodePipeline. Create a new S3 Bucket in the S3 Console, or alternatively select a S3 bucket that may be created from an earlier run of CodePipeline.
Next, create a CodeBuild project that is to be used to build the source code into a Docker image. The source code for a Docker image dvohra/node-server
and the buildspec.yml
file used to build the source code into the Docker image was discussed earlier. After a Docker image is built, it is uploaded to Docker hub by CodeBuild. To create a CodeBuild project, open this URL in a web browser. Select Build projects and click on Create project as shown in Figure 5.
Figure 5. Build projects>Create project
The Create project wizard gets started. In Configure project specify a Project name (node-server). In Source>Source Provider select GitHub. For Repository select Use a repository in my account and Choose a repository as the _dvohra/docker-node-server _ repo. Keep the default setting for Git clone depth as 1.
Select the options for Webhook, Insecure SSL and Build Badge. Selecting Webhook makes the code get rebuilt every time a code update is made in the GitHub repo. The Insecure SSL option makes the code build SSL warnings when connecting to project source. The Build Badge option makes the project’s status visible and embeddable. In Environment: How to build select Environment image as Use an image managed by AWS CodeBuild. Select Operating System as Ubuntu. For Runtime, select Docker as shown in Figure 6.
Figure 6. Selecting Runtime as Docker
For Runtime version, select aws:codebuild/docker:17.09.0, which represents the Docker version 17.9. If a later version is available select the later version. The Privileged option gets selected automatically for Docker runtime as it is required to build a Docker image. For Build specification select Use the buildspec.yml in the source code root directory as shown in Figure 7. The Buildspec name is buildspec.yml by default.
Figure 7. Configuring Environment: How to build
For Certificate select Do not install any certificate. A certificate is not required by CodePipeline, but for a more secure CodeBuild a self-signed certificate could be installed from S3. Next, configure the output artifacts in the Artifacts. Output artifacts are required by CodePipeline Select Type as Amazon S3. Specify a S3 bucket folder name to use. Set the Path as “/”, which creates the folder at the bucket root. Select Namespace type as Node. Select a Bucket name as the Bucket configured earlier as shown in Figure 8. Select Cache as No cache.
Figure 8. Configuring S3 Bucket for Artifacts
In Service role select the option Create a service role in your account if the CodeBuild project is being created for the first time. If the CodeBuild project was created before, select Choose an existing service role from your account. Select the option Allow AWS CodeBuild to modify this service role so it can be used with this build project as shown in Figure 9. For VPC select No VPC. Click on Continue.
Figure 9. Configuring Service Role and VPC settings
In Review, review the Source and Build environment. Scroll down and click on Create to create the CodeBuild project. A CodeBuild project gets created and listed in Build projects as shown in Figure 10.
Figure 10. CodeBuild Project
The service role created by default by CodeBuild does not include some of the required permissions. The service role needs to be modified by adding an inline policy that adds permissions s3:GetObject
and s3:PutObject
. The inline policy to add is listed:
{
"Version":"2012-10-17",
"Statement":[
{
"Effect":"Allow",
"Action":["s3:PutObject","s3:GetObject"],
"Resource":"arn:aws:s3:::*"
}
]
}
Test the CodeBuild project before creating and configuring the CodeBuild in a CodePipeline so that if any errors exist they may be fixed. Click on Start build to start the build as shown in Figure 11.
Figure 11. Start build
The Start new build wizard gets started. Click on Start build. The CodeBuild project gets started and the code starts to get built. When the CodeBuild project gets completed, the Phase details indicate the same as shown in Figure 12.
Figure 12. Phase details and Build logs indicate that the CodeBuild has completed
The Docker image dvohra/node-server generated and updated by the CodeBuild project to Docker Hub is shown in Figure 13.
Figure 13. Docker Image generated and uploaded by CodeBuild on Docker Hub
Having created projects for each of the CodePipeline stages; GitHub code repository for Source, CodeBuild for Build and ECS Fargate service for Staging, next we shall create a CodePipeline. Open CodePipeline Console with URL and click on Get started as shown in Figure 14.
Figure 14. CodePipeline>Get started
The Create pipeline wizard gets started as shown in Figure 15. First, specify a Pipeline name (node-server-fargate) and click on Next step.
Figure 15. Specifying Pipeline Name
Next, configure the Source location. For Source provider select GitHub as shown in Figure 16.
Figure 16. Selecting Source provider
Next, connect to the GitHub with Connect to GitHub as shown in Figure 17.
Figure 17. Connect to GitHub
Select the GitHub Repository as shown in Figure 18.
Figure 18. Selecting GitHub Repo
Select the repo Branch as shown in Figure 19. Click on Next step.
Figure 19. Source location>Next step
Next, configure the Build. Select Build provider as AWS CodeBuild as shown in Figure 20.
Figure 20. Selecting Build provider as AWS CodeBuild
The subsequent section displayed is based on the Build provider selected. For AWS CodeBuild, a section to provide details about the CodeBuild gets displayed. For Configure your project select Select an existing project as shown in Figure 21. Select Project name as the CodeBuild project node-server created earlier.
Figure 21. Selecting CodeBuild Project
Click on Next step. Next, configure the Deploy stage of the CodePipeline as shown in Figure 22. Select Deployment provider as Amazon ECS.
Figure 22. Selecting Deployment provider as Amazon ECS
The subsequent section displayed is based on the Deployment provider selected as indicated by the Amazon ECS section in Figure 23. In the Amazon ECS section select Cluster name as the cluster in which the ECS service to be deployed to is created; select default.
Figure 23. Selecting ECS Cluster
Select the Service name as the ECS service to deploy to, which is node-server-service, as shown in Figure 24.
Figure 24. Selecting ECS Service
Specify the Image filename as imagedefinitions.json
as shown in Figure 25. If omitted, the default Image filename is imagedefinitions.json
and the file should be available in the Source code GitHub repo. Click on Next step.
Figure 25. Specifying Image filename
Next, select the Service Role name as shown in Figure 26. A new service role is created in IAM automatically the first time a CodePipeline is created. Subsequently, the Service role gets listed to be selected.
Figure 26. Selecting Service Role
Click on Next step. Review the CodePipeline and click on Create pipeline as shown in Figure 27.
Figure 27. Create pipeline
A CodePipeline gets created as shown in Figure 28. After getting created the CodePipeline starts to run automatically as shown by In Progress status for Source stage.
Figure 28. CodePipeline Source stage in Progress
When the Source stage completes, its status becomes Succeeded as shown in Figure 29. And the Build stage starts to run as indicated by the In Progress status.
Figure 29. Source stage completed and Build stage In Progress
The Build stage also gets completed as indicated by the Succeeded status in Figure 30. The Staging stage starts to run.
Figure 30. Build stage succeeded and Staging stage In Progress
We had already configured all the CodePiepline stages; why do we need to modify the settings for Input/Output Artifacts? Because for the example ECS application deployed by CodePipeline, the default Input/Output artifacts settings are not as required to run the CodePipeline. By default, the input artifacts to the Staging stage are the output artifacts from the Build stage. For a CodeBuild that only builds a Docker image and uploads the Docker image to Docker Hub or Amazon ECR, no output artifacts are generated by the CodeBuild stage. The input artifacts to the Staging stage need to be set to the output artifacts from the Source stage. Without modifying the settings for the Input/Output artifacts, the Staging stage would fail. The CodePipeline that was started automatically after being created fails. To modify the Input/Output artifacts click on Edit as shown in Figure 31.
Figure 31. CodePipeline>Edit
After modifying the Input/Output artifacts, click on Save pipeline changes as shown in Figure 32.
Figure 32. Save pipeline changes
To run the CodePipeline after modification, click on Release change as shown in Figure 33.
Figure 33. Release change
In the Release change confirmation dialog, click on Release. The modifications made to the CodePipeline get applied and the CodePipeline starts to run from the start as indicated by the In Progress status for the Source stage in Figure 34. The status for the Build and Staging stages are from the previous run and does not indicate the current status of the stages.
Figure 34. Source stage In Progress
All stages of the CodePipeline get completed with status Succeeded as shown in Figure 35.
Figure 35. All stages of CodePipeline completed successfully
The previous service task gets stopped and a task based on the revised task definition starts to run as indicated by the RUNNING task status in Figure 36.
Figure 36. New task running
To invoke the new task, find its Public IP as before from the Elastic Network interface for the task. Open URL :8080 in a web browser to invoke the task. The Node server application response is shown in Figure 37.
Figure 37. Node Server application response
What is the advantage of running a CodePipeline, if the ECS service response is the same as when invoking the service directly without a CodePipeline? Typically the source code for an ECS deployed in production would need to be updated periodically, which implies that the Docker image would need to be rebuilt. The Docker image deployed in an ECS service task would also need to be updated. The Docker image needs to be updated without any discontinuation of the ECS based service. With a CodePipeline, a new run of the CodePipeline is started automatically every time source code modifications are made. Without any user intervention, the ECS deployment gets updated when source code changes are made.
To demonstrate, make a slight modification to source code in the GitHub repo, such as make the Hello message in server.js
different. Click on Commit changes in the repo. The CodePipeline starts to re-run automatically as shown by the Source stage having completed successfully and the Build stage in progress in Figure 38.
Figure 38. CodePipeline restarted automatically
After the Build stage completes successfully, the Staging starts to run as indicated by the status messages in Figure 39.
Figure 39. Build stage completed and Staging stage started
The Staging also completes successfully. The new task gets started. Using the Public IP of the new task, open the URL :8080 in a web browser. The new task gets invoked and the Node server response gets displayed as shown in Figure 40. The Node server response is the modified response from server.js.
Figure 40. Modified Node Server Response from new task
In this article, we discussed deploying a Docker container application to ECS Fargate. We have demonstrated updating source code for a Docker image and deploying the new Docker image to a running container service without stopping the container service, all using an AWS CodePipeline.
#docker #aws #devops