Rory  West

Rory West

1623282000

Private PyPi Server on AWS with Terraform

Deploy it in less than 5 minutes with Terraform

While working with complex and multi-module Python projects it quickly becomes crucial to share libraries across different components, enable developers to easily install those libraries into their local development environment, and use them in continuous integration tools. A private PyPi repository is a good solution to this problem since it allows installing internal libraries anywhere just by using regular pip install commands while keeping full control over the Python packages.

If your application is running on the cloud, you likely want to deploy your PyPi server within your infrastructure. In this post, I focus on the AWS cloud and show how to deploy a password-protected PyPi server on a small EC2 instance within an existing VPC. As a server, I am going to use a minimal PyPi server implementation that is easy to set up, not demanding in terms of resources, and, most importantly, still actively maintained on GitHub. The cloud infrastructure is built using Terraform, a great tool that has become the de-facto standard for infrastructure-as-a-code (IaaC) provisioning. Thanks to Terraform, running your own PyPi repository on AWS can be done in less than a minute. But now, let’s dive in.

#terraform #python #pypi #aws

What is GEEK

Buddha Community

Private PyPi Server on AWS with Terraform

Private PyPi Server on AWS with Terraform

While working with complex and multi-module Python projects it quickly becomes crucial to share libraries across different components, enable developers to easily install those libraries into their local development environment, and use them in continuous integration tools. A private PyPi repository is a good solution to this problem since it allows installing internal libraries anywhere just by using regular pip install commands while keeping full control over the Python packages.

If your application is running on the cloud, you likely want to deploy your PyPi server within your infrastructure. In this post, I focus on the AWS cloud and show how to deploy a password-protected PyPi server on a small EC2 instance within an existing VPC. As a server, I am going to use a minimal PyPi server implementation that is easy to set up, not demanding in terms of resources, and, most importantly, still actively maintained on GitHub. The cloud infrastructure is built using Terraform, a great tool that has become the de-facto standard for infrastructure-as-a-code (IaaC) provisioning. Thanks to Terraform, running your own PyPi repository on AWS can be done in less than a minute. But now, let’s dive in.

#terraform #python #pypi #aws

Rory  West

Rory West

1623282000

Private PyPi Server on AWS with Terraform

Deploy it in less than 5 minutes with Terraform

While working with complex and multi-module Python projects it quickly becomes crucial to share libraries across different components, enable developers to easily install those libraries into their local development environment, and use them in continuous integration tools. A private PyPi repository is a good solution to this problem since it allows installing internal libraries anywhere just by using regular pip install commands while keeping full control over the Python packages.

If your application is running on the cloud, you likely want to deploy your PyPi server within your infrastructure. In this post, I focus on the AWS cloud and show how to deploy a password-protected PyPi server on a small EC2 instance within an existing VPC. As a server, I am going to use a minimal PyPi server implementation that is easy to set up, not demanding in terms of resources, and, most importantly, still actively maintained on GitHub. The cloud infrastructure is built using Terraform, a great tool that has become the de-facto standard for infrastructure-as-a-code (IaaC) provisioning. Thanks to Terraform, running your own PyPi repository on AWS can be done in less than a minute. But now, let’s dive in.

#terraform #python #pypi #aws

Rory  West

Rory West

1619263860

Why Terraform? How to Getting Started with Terraform Using AWS

Terraform is a tool for building, changing, and versioning infrastructure safely and efficiently. Terraform can manage existing and popular service providers as well as custom in-house solutions.

Traditional Infrastructure vs Modern Infrastructure

Traditional Infrastructure

  • Mutable
  • Operational Complexity
  • No Central Control on Infrastructure

Modern Infrastructure

  • Immutable
  • Less Operational Complexity
  • Faster time to the market
  • single point for state management

#terraform-aws #terraform #aws #aws-ec2

Rory  West

Rory West

1620959460

Complete Guide to Terraform AWS

We’re continuing our series on Terraform AWS with a post that breaks down the basics. The world of Terraform AWS can be described as complex — from AWS storage to AWS best practices, there’s a depth of knowledge necessary to get familiar with Terraform AWS.

Whether you’re an expert at Terraform AWS or just getting started, it’s our goal at InfraCode to provide you with clear and easy-to-understand information at every level. The number of resources out there is abundant but overwhelming. That’s why we create simplified guides that are immediately usable and always understandable.

In this article, we’ll dive into:

  • A Beginner’s Overview to Terraform AWS
  • Managing AWS Storage
  • Terraform AWS Best Practices

#aws-ec2 #aws #terraform #terraform aws

Ruby  Schmitt

Ruby Schmitt

1597945860

Terraform: Iterating through a Map of Lists To Define AWS Roles and Permissions

A few months ago, I was working on a Terraform module to manage all the roles and their permissions in our AWS accounts. This on the surface seems like a straight forward project, but there was a curveball that required some research, trial & error, and finesse to address.

The teams/permissions were not consistent across the AWS accounts. TeamA might have read/write access to s3 in account A, but only have read access to s3 in account B. Team A does not even exist in account C. Multiply this conundrum by 10+ teams across 10+ accounts.

In thinking about how to best tackle this issue, there were a couple bad ways to solve this that immediately come to mind:

  • Brute force — define the permission for every team in every environment.

This approach is horrible. It would have been tedious, hard to maintain, and the amount of repeated code would have been astronomical, but it would have worked.

  • Ask the business to standardize permissions.

This on the surface seems reasonable but it is not. First, your code is dictating business logic/function. Secondly, the principle of least privilege means that you should only allow enough access to perform the required job. Third, there are AWS accounts which certain teams should not have access to (e.g. secops, networking, & IT accounts). Last, the business would never agree to it.


The right approach needed to something that could account for all the variability across the accounts. Additionally, the end result needed to be clean, easy to maintain/update, and easy to use without requiring a deep understanding of how the module worked.

What I envisioned was something that allowed me to define the permissions as part of the config. This design addressed the variability issues across the accounts by allowing me to define the permissions per iteration of the module. Additionally, it was easy to understand and manage (even if you didn’t know what the module was doing).

This looked something like:

module usermap {
  source = "../modules/example-module"

  role_map_aws_policies = {
    TeamA = ["AdministratorAccess"]
    TeamB = ["AmazonS3FullAccess", "AmazonEC2FullAccess"]
    TeamC = ["AdministratorAccess"]
    TeamD = ["ReadOnlyAccess", "AmazonInspectorFullAccess"]
  }
}

#aws #aws-iam #automating-aws-iam #terraform #terraform-modules