Urban  Bayer

Urban Bayer

1589903880

Building an IoT CI/CD pipeline (AWS, Ansible, Jenkins, Docker and git)

he rise of the Internet of Things has forced people to re-think how software for embedded systems gets developed. As devices become more connected, traditional project management frameworks are no longer viable. With these small devices with less computational ability and memory a continuous integration and deployment pipeline that allows the team to follow Agile/Scrum methodologies is needed.

#devops #machine-learning #docker #iot #aws

What is GEEK

Buddha Community

Building an IoT CI/CD pipeline (AWS, Ansible, Jenkins, Docker and git)
Matt  Towne

Matt Towne

1589791867

Serverless CI/CD on the AWS Cloud

CI/CD pipelines have long played a major role in speeding up the development and deployment of cloud-native apps. Cloud services like AWS lend themselves to more agile deployment through the services they offer as well as approaches such as Infrastructure as Code. There is no shortage of tools to help you manage your CI/CD pipeline as well.

While the majority of development teams have streamlined their pipelines to take full advantage of cloud-native features, there is still so much that can be done to refine CI/CD even further. The entire pipeline can now be built as code and managed either via Git as a single source of truth or by using visual tools to help guide the process.

The entire process can be fully automated. Even better, it can be made serverless, which allows the CI/CD pipeline to operate with immense efficiency. Git branches can even be utilized as a base for multiple pipelines. Thanks to the three tools from Amazon; AWS CodeCommit, AWS CodeBuild, and AWS CodeDeploy, serverless CI/CD on the AWS cloud is now easy to set up.

#aws #aws codebuild #aws codecommit #aws codedeploy #cd #cd pipeline #ci #ci/cd processes #ci/cd workflow #serverless

Hyman  Simonis

Hyman Simonis

1625911080

Build a complete CI/CD Pipeline with AWS - Jenkins - Bitbucket - Docker - Terraform

In this fourth part, we are going to complete the construction of our AWS infrastructure. What we are left to create are:

  • S3 Buckets for logs and to store the Jenkins User Data (which we are going to implement in the fifth Part)
  • ECR Repository where the Docker images will be uploaded
  • IAM Policies to let the EC2 instance have the right access to other AWS resources.
  • Secrets Manager Secret to store our bitbucket SSH keys (that we are going to create) which will be needed by the Jenkins instance to pull down the repository at the start of the pipeline. In this regard we will create so-called Jenkins credentials which will store the private key and allow jenkins to comunicate with the Bibucket repo.

#ci-cd-pipeline #jenkins #aws #terraform #bitbucket #docker

Hyman  Simonis

Hyman Simonis

1624264070

Build a complete CI/CD Pipeline with AWS - Jenkins - Bitbucket - Docker - Terraform

In this second part we are going to setup Slack o be ready to be implemented in the pipeline that we are going to create in the next parts. We are going to create an Account,Workspace,App (a bot) and mess around a bit to get some confidence with the Slack API.

Account, Workspace and Channel Creation

If you do not have already a Slack account, just navigate to:

https://slack.com/

And we can sign up with a google account. After some possible verifications, we should be redirected to the following page:

Slack home page after logging in

Slack get-started page after signing up

Let’s click the button Create a Workspace , and name it Simple Web App :

naming workspace - screen

Naming the workspace

After clicking Next we will be prompted with a question asking us what we are currently working on. Let’s put there CI-CD Pipeline :

naming channel — screen

Put ‘CI-CD Pipeline’ in the input field, which will be converted in a channel

As we can see after hitting Next , on the left we have:

screen of the new channel just created

And this will be the channel through which we will communicate from the Jenkins server.

In the next Step we can hit Skip this step and confirm.

screen for skipping the next step

Cool, we created a workspace and a new channel. At this point, we need to create an App in order to use the Slack API.

#ci-cd-pipeline #jenkins #aws #terraform #docker #bitbucket

Hyman  Simonis

Hyman Simonis

1623835410

Build a complete CI/CD Pipeline with AWS - Jenkins - Bitbucket - Docker - Terraform

Hello guys and welcome to this tutorial in which I will be guiding you through the creation of a complete CI / CD Pipeline governed by Jenkins with all the infrastructure on AWS.

Let me first lay down a summary of what we are going to build and the different steps we’ll be taking.

GOAL

Suppose we are building / working on a Web App and we would like to implement a Continuous Integration (CI) / Continuous Deploy (CD) Pipeline to be able to develop and release working parts of the software in a more reliable and faster way in the spirit of DevOps foundations. The pipeline needs to run on a push on the remote repository (Bitbucket) and go through various steps (specified in a Jenkinsfile) in which unit / integration / load balancing tests will be run alongside some security / vulnerability scans, while saving artifacts along the way with tests’ results uploaded to an S3 bucket (for future use) and a Slack notification for immediate feedback. Artifacts will be saved in AWS Elastic Container Registry as Docker images and we would like to build all the infrastructure with Terraform, implementing our Infrastructure as Code (IaS), allowing us to programmatically define everything and treat our infrastructure (and the pipeline thanks to the use of a Jenkinsfile and the multibranch pipeline) as normal code.

BONUS

As a bonus we will also set up Jenkins completely from the user data of the AWS EC2 instance (namely from Bash at creation time). It will automatically create a jenkins admin, confirm the jenkins URL, download and install all the necessary plugins, create ssh credentials to let jenkins access Bitbucket and finally set up a multibranch pipeline which will be triggered by a push on the bitbucket repository.

CONSIDERATIONS

The pipeline we are going to create relies on a single branch. In a more mature and bigger project, you’d want to implement a Test, Staging and Production branches, to better manage all these different phases of the software development process. Still, I believe the techniques we are going to employ are the foundations for more advanced ones and with not that many modifications one could easily implement more branches in the project.

#ci-cd-pipeline #aws #jenkins #terraform #bitbucket #docker

Urban  Bayer

Urban Bayer

1589903880

Building an IoT CI/CD pipeline (AWS, Ansible, Jenkins, Docker and git)

he rise of the Internet of Things has forced people to re-think how software for embedded systems gets developed. As devices become more connected, traditional project management frameworks are no longer viable. With these small devices with less computational ability and memory a continuous integration and deployment pipeline that allows the team to follow Agile/Scrum methodologies is needed.

#devops #machine-learning #docker #iot #aws