Comment Déployer une Application FastAPI sur AWS Elastic Beanstalk

Dans ce didacticiel, nous allons parcourir le processus de déploiement d'une application FastAPI sur AWS Elastic Beanstalk .

Objectifs

À la fin de ce didacticiel, vous serez en mesure de :

1. Expliquer ce qu'est Elastic Beanstalk
2. Initialiser et configurer Elastic Beanstalk
3. Dépanner une application exécutée sur Elastic Beanstalk
4. Intégrer Elastic Beanstalk à RDS
5. Obtenir un certificat SSL via AWS Certificate Manager
6. Utilisez un certificat SSL pour servir votre application sur HTTPS

Qu'est-ce qu'Elastic Beanstalk ?

AWS Elastic Beanstalk (EB) est un service facile à utiliser pour le déploiement et la mise à l'échelle des applications Web. Il connecte plusieurs services AWS, comme les instances de calcul ( EC2 ), les bases de données ( RDS ), les équilibreurs de charge ( Application Load Balancer ) et les systèmes de stockage de fichiers ( S3 ), pour n'en nommer que quelques-uns. EB vous permet de développer et de déployer rapidement votre application Web sans vous soucier de l'infrastructure sous-jacente. Il prend en charge les applications développées en Go, Java, .NET, Node.js, PHP, Python et Ruby. EB prend également en charge Docker si vous avez besoin de configurer votre propre pile logicielle ou de déployer une application développée dans un langage (ou une version) qu'EB ne prend pas actuellement en charge.

Configuration typique d'Elastic Beanstalk :

There's no additional charge for AWS Elastic Beanstalk. You only pay for the resources that your application consumes.

To learn more about Elastic Beanstalk, check out What is AWS Elastic Beanstalk? from the official AWS Elastic Beanstalk documentation.

Elastic Beanstalk Concepts

Before diving into tutorial itself, let's look at a few key concepts related to Elastic Beanstalk:

1. An application is a logical collection of Elastic Beanstalk components, including environments, versions, and environment configurations. An application can have multiple versions.
2. An environment is a collection of AWS resources running an application version.
3. A platform is a combination of an operating system, programming language runtime, web server, application server, and Elastic Beanstalk components.

These terms will be used throughout the tutorial.

Project Setup

We'll be deploying a simple FastAPI application called fastapi-songs in this tutorial.

Check your understanding by deploying your own application as you follow along with the tutorial.

First, grab the code from the repository on GitHub:

$ git clone git@github.com:duplxey/fastapi-songs.git
$ cd fastapi-songs

Create a new virtual environment and activate it:

$ python3 -m venv venv && source venv/bin/activate

Install the requirements and initialize the database:

(venv)$ pip install -r requirements.txt
(venv)$ python init_db.py

Run the server:

(venv)$ uvicorn main:app --reload

Open your favorite web browser and navigate to:

1. http://localhost:8000 - should display "fastapi-songs" text
2. http://localhost:8000/songs - should display a list of songs

Elastic Beanstalk CLI

Be sure to register for an AWS account before continuing. By creating an account you might also be eligible for the AWS Free Tier.

The Elastic Beanstalk command line interface (EB CLI) allows you to perform a variety of operations to deploy and manage your Elastic Beanstalk applications and environments.

There are two ways of installing EB CLI:

1. Via the EB CLI installer
2. With pip (awsebcli)

It's recommended to install the EB CLI globally (outside any specific virtual environment) using the installer (first option) to avoid possible dependency conflicts. Refer to this explanation for more details.

After you've installed the EB CLI, you can check the version by running:

$ eb --version

EB CLI 3.20.3 (Python 3.10.)

If the command doesn't work, you may need to add the EB CLI to $PATH.

A list of EB CLI commands and their descriptions can be found in the EB CLI command reference.

Initialize Elastic Beanstalk

Once we have the EB CLI running we can start interacting with Elastic Beanstalk. Let's initialize a new project along with an EB Environment.

Init

Within the project root ("fastapi-songs"), run:

$ eb init

You'll be prompted with a number of questions.

Default Region

The AWS region of your Elastic Beanstalk environment (and resources). If you're not familiar with the different AWS regions, check out AWS Regions and Availability Zones. Generally, you should pick the region that's closest to your customers. Keep in mind that resource prices vary from region to region.

Application Name

This is the name of your Elastic Beanstalk application. I recommend just pressing enter and going with the default: "fastapi-songs".

Platform and Platform Branch

The EB CLI will detect that you're using a Python environment. After that, it'll give you different Python versions and Amazon Linux versions you can work with. Pick "Python 3.8 running on 64bit Amazon Linux 2".

CodeCommit

CodeCommit is a secure, highly scalable, managed source control service that hosts private Git repositories. We won't be using it since we're already using GitHub for source control. So say "no".

SSH

To connect to the EC2 instances later we need to set up SSH. Say "yes" when prompted.

Keypair

To connect to EC2 instances, we'll need an RSA keypair. Go ahead and generate one, which will be added to your "~/.ssh" folder.

After you answer all the questions, you'll notice a hidden directory inside your project root named ".elasticbeanstalk". The directory should contain a config.yml file, with all the data you've just provided.

.elasticbeanstalk
└── config.yml

The file should contain something similar to:

branch-defaults:
  master:
    environment: null
    group_suffix: null
global:
  application_name: fastapi-songs
  branch: null
  default_ec2_keyname: aws-eb
  default_platform: Python 3.8 running on 64bit Amazon Linux 2
  default_region: us-west-2
  include_git_submodules: true
  instance_profile: null
  platform_name: null
  platform_version: null
  profile: eb-cli
  repository: null
  sc: git
  workspace_type: Application

Create

Next, let's create the Elastic Beanstalk environment and deploy the application:

$ eb create

Again, you'll be prompted with a few questions.

Environment Name

This represents the name of the EB environment. I'd recommend sticking with the default: "fastapi-songs-env".

It's considered good practice to add └-env or └-dev suffix to your environments so you can easily differentiate EB apps from environments.

DNS CNAME Prefix

Your web application will be accessible at %cname%.%region%.elasticbeanstalk.com. Again, use the default.

Load balancer

A load balancer distributes traffic amongst your environment's instances. Select "application".

If you want to learn about the different load balancer types, review Load balancer for your Elastic Beanstalk Environment.

Spot Fleet Requests

Spot Fleet requests allow you to launch instances on-demand based on your criteria. We won't be using them in this tutorial, so say "no".

--

With that, the environment will be spun up:

1. Your code will be zipped up and uploaded to a new S3 Bucket.
2. After that, the various AWS resources will be created, like the load balancer, security and auto-scaling groups, and EC2 instances.

A new application will be deployed as well.

This will take about three minutes so feel free to grab a cup of coffee.

After the deployment is done, the EB CLI will modify .elasticbeanstalk/config.yml.

Your project structure should now look like this:

|-- .elasticbeanstalk
|   └-- config.yml
|-- .gitignore
|-- README.md
|-- database.py
|-- default.db
|-- init_db.py
|-- main.py
|-- models.py
└-- requirements.txt

Status

Once you've deployed your app you can check its status by running:

$ eb status

Environment details for: fastapi-songs-env
  Application name: fastapi-songs
  Region: us-west-2
  Deployed Version: app-82fb-220311_171256090207
  Environment ID: e-nsizyek74z
  Platform: arn:aws:elasticbeanstalk:us-west-2::platform/Python 3.8 running on 64bit Amazon Linux 2/3.3.11
  Tier: WebServer-Standard-1.0
  CNAME: fastapi-songs-env.us-west-2.elasticbeanstalk.com
  Updated: 2022-03-11 23:16:03.822000+00:00
  Status: Launching
  Health: Red

You can see that our environment's current health is Red, which means that something went wrong. Don't worry about this just yet, we'll fix it in the next steps.

You can also see that AWS assigned us a CNAME which is our EB environment's domain name. We can access the web application by opening a browser and navigating to the CNAME.

Open

$ eb open

This command will open your default browser and navigate to the CNAME domain. You'll see 502 Bad Gateway, which we'll fix here shortly.

Console

$ eb console

This command will open the Elastic Beanstalk console in your default browser:

Again, you can see that the health of the environment is "Severe", which we'll fix in the next step.

Configure an Environment

In the previous step, we tried accessing our application and it returned 502 Bad Gateway. There are three reasons behind it:

1. Python needs PYTHONPATH in order to find modules in our application.
2. By default, Elastic Beanstalk attempts to launch the WSGI application from application.py, which doesn't exist.
3. If not specified otherwise, Elastic Beanstalk tries to serve Python applications with Gunicorn. Gunicorn by itself is not compatible with FastAPI since FastAPI uses the newest ASGI standard.

Let's fix these errors.

Create a new folder in the project root called ".ebextensions". Within the newly created folder create a file named 01_fastapi.config:

# .ebextensions/01_fastapi.config

option_settings:
  aws:elasticbeanstalk:application:environment:
    PYTHONPATH: "/var/app/current:$PYTHONPATH"
  aws:elasticbeanstalk:container:python:
    WSGIPath: "main:app"

Notes:

1. We set the PYTHONPATH to the Python path on our EC2 instance (docs).
2. We changed the WSGIPath to our WSGI application (docs).

How do EB .config files work?

1. You can have as many as you want.
2. They are loaded in the following order: 01_x, 02_x, 03_x, etc.
3. You do not have to memorize these settings; you can list all your environmental settings by running eb config.

If you want to learn more about advanced environment customization check out Advanced environment customization with configuration files.

Since FastAPI doesn't support Gunicorn by itself, we'll have to modify our web process command to use a Uvicorn worker class.

Uvicorn is the recommended ASGI web application server for FastAPI.

Create a new file called Procfile inside the project root:

# Procfile

web: gunicorn main:app --workers=4 --worker-class=uvicorn.workers.UvicornWorker

If you're deploying your own application, make sure to add uvicorn as a dependency to requirements.txt.

For more information on Procfiles review Configuring the WSGI server with a Procfile.

Lastly, we have to tell Elastic Beanstalk to initialize the database when a new application version gets deployed. Add the following to the end of .ebextensions/01_fastapi.config:

# .ebextensions/01_fastapi.config

container_commands:
  01_initdb:
    command: "source /var/app/venv/*/bin/activate && python3 init_db.py"
    leader_only: true

The EB environment will now execute the above command every time we deploy a new application version. We used leader_only, so only the first EC2 instance executes them (in case our EB environment runs multiple EC2 instances).

Elastic Beanstalk configs support two different command sections, commands and container_commands. The main difference between them is when they are run in the deployment process:

1. commands run before the application and web server are set up and the application version file is extracted.
2. container_commands run after the application and web server have been set up and the application version archive has been extracted, but before the application version is deployed (before the files are moved from the staging folder to their final location).

At this point your project structure should look like this:

|-- .ebextensions
|   └-- 01_fastapi.config
|-- .elasticbeanstalk
|   └-- config.yml
|-- .gitignore
|-- Procfile
|-- README.md
|-- database.py
|-- default.db
|-- init_db.py
|-- main.py
|-- models.py
`-- requirements.txt

Commit the changes to git and deploy:

$ git add .
$ git commit -m "updates for eb"

$ eb deploy

You'll notice that Elastic Beanstalk won't detect the changes if you don't commit. That's because EB integrates with git and only detects the committed (changed) files.

After the deployment is done, run eb open to see if everything worked. After that, append /songs to the URL to see if the songs still get displayed.

Yay! The first version of our app is now deployed.

Configure RDS

If you're deploying fastapi-songs, you'll notice that it uses a SQLite database by default. While this is perfect for development, you'll typically want to move to a more robust database, like Postgres or MySQL, for production. Let's look at how to swap SQLite for Postgres.

Local Postgres

First, let's get Postgres running locally. You can either download it from PostgreSQL Downloads or spin up a Docker container:

$ docker run --name fastapi-songs-postgres -p 5432:5432 \
    -e POSTGRES_USER=fastapi-songs -e POSTGRES_PASSWORD=complexpassword123 \
    -e POSTGRES_DB=fastapi-songs -d postgres

Check if the container is running:

$ docker ps -f name=fastapi-songs-postgres

CONTAINER ID   IMAGE      COMMAND                  CREATED              STATUS              PORTS                    NAMES
c05621dac852   postgres   "docker-entrypoint.s…"   About a minute ago   Up About a minute   0.0.0.0:5432->5432/tcp   fastapi-songs-postgres

Now, let's try connecting to it with our FastAPI app.

Inside database.py change DATABASE_URL like so:

# database.py

DATABASE_URL = \
    'postgresql://{username}:{password}@{host}:{port}/{database}'.format(
        username='fastapi-songs',
        password='complexpassword123',
        host='localhost',
        port='5432',
        database='fastapi-songs',
    )

After that remove connect_args from create_engine since check_same_thread is only required for SQLite.

# database.py

engine = create_engine(
    DATABASE_URL,
)

Next, install psycopg2-binary, which is required for Postgres:

(venv)$ pip install psycopg2-binary==2.9.3

Add it to requirements.txt:

fastapi==0.75.0
psycopg2-binary==2.9.3
SQLAlchemy==1.4.32
uvicorn[standard]==0.17.6

Remove the existing database, default.db, and then initialize the new database:

(venv)$ python init_db.py

Run the server:

(venv)$ uvicorn main:app --reload

Make sure the songs still get served correctly by checking out http://localhost:8000/songs.

AWS RDS Postgres

To set up Postgres for production, start by running the following command to open the AWS console:

$ eb console

Click "Configuration" on the left side bar, scroll down to "Database", and then click "Edit".

Create a DB with the following settings and click on "Apply":

• Engine: postgres
• Engine version: 12.9 (older Postgres version since db.t2.micro is not available with 13.1+)
• Instance class: db.t2.micro
• Storage: 5 GB (should be more than enough)
• Username: pick a username
• Password: pick a strong password

If you want to stay within the AWS Free Tier make sure you pick db.t2.micro. RDS prices increase exponentially based on the instance class you pick. If you don't want to go with micro make sure to review AWS PostgreSQL pricing.

After the environmental update is done, EB will automatically pass the following DB credentials to our FastAPI app:

RDS_DB_NAME
RDS_USERNAME
RDS_PASSWORD
RDS_HOSTNAME
RDS_PORT

We can now use these variables in database.py to connect to our database. Replace DATABASE_URL with the following:

if 'RDS_DB_NAME' in os.environ:
    DATABASE_URL = \
        'postgresql://{username}:{password}@{host}:{port}/{database}'.format(
            username=os.environ['RDS_USERNAME'],
            password=os.environ['RDS_PASSWORD'],
            host=os.environ['RDS_HOSTNAME'],
            port=os.environ['RDS_PORT'],
            database=os.environ['RDS_DB_NAME'],
        )
else:
    DATABASE_URL = \
        'postgresql://{username}:{password}@{host}:{port}/{database}'.format(
            username='fastapi-songs',
            password='complexpassword123',
            host='localhost',
            port='5432',
            database='fastapi-songs',
        )

Don't forget to import the os package at the top of database.py:

import os

Your final database.py file should look like this.

Commit the changes to git and deploy:

$ git add .
$ git commit -m "updates for eb"

$ eb deploy

Wait for the deployment to finish. Once done, run eb open to open your app in a new browser tab. Make sure everything still works correctly by listing the songs at /songs.

HTTPS with Certificate Manager

This part of the tutorial requires that you have a domain name.

Need a cheap domain to practice with? Several domain registrars have specials on '.xyz' domains. Alternatively, you can create a free domain at Freenom. If you don't own a domain name, but would still like to use HTTPS you can create and sign with an X509 certificate.

To serve your application via HTTPS, we'll need to:

1. Request and validate an SSL/TLS certificate
2. Point your domain name to your EB CNAME
3. Modify the load balancer to serve HTTPS
4. Modify your application settings

Request and Validate an SSL/TLS Certificate

Navigate to the AWS Certificate Manager console. Click "Request a certificate". Set the certificate type to "Public" and click "Next". Enter your fully qualified domain name into the form input, set the "Validation method" to "DNS validation", and click "Request".

You'll then be redirected to a page where you can see all your certificates. The certificate that you just created should have a status of "Pending validation".

For AWS to issue a certificate, you first have to prove that you're the owner of the domain. In the table, click on the certificate to view the "Certificate details". Take note of the "CNAME name" and "CNAME value". To validate the ownership of the domain, you'll need to create a CNAME Record" in your domain's DNS settings. Use the "CNAME name" and "CNAME value" for this. Once done, it will take a few minutes for Amazon to pick up the domain changes and issue the certificate. The status should change from "Pending validation" to "Issued".

Point the Domain Name to the EB CNAME

Next, you need to point your domain (or subdomain) to your EB environment CNAME. Back in your domain's DNS settings, add another CNAME record with the value being your EB CNAME -- e.g., fastapi-songs-dev.us-west-2.elasticbeanstalk.com.

Wait a few minutes for your DNS to refresh before testing things out from the http:// flavor of your domain name in your browser.

Modify the Load Balancer to serve HTTPS

Back in the Elastic Beanstalk console, click "Configuration". Then, within the "Load balancer" category, click "Edit". Click "Add listener" and create a listener with the following details:

1. Port - 443
2. Protocol - HTTPS
3. SSL certificate - select the certificate that you just created

Click "Add". Then, scroll to the bottom of the page and click "Apply". It will take a few minutes for the environment to update.

Modify your Application Settings

Next, we need to make a few changes to our FastAPI application.

We need to redirect all traffic from HTTP to HTTPS. There are multiple ways of doing this, but the easiest way is to set up Apache as a proxy host. We can achieve this programmatically by adding the following to the end of the option_settings in .ebextensions/01_fastapi.config:

# .ebextensions/01_fastapi.config

option_settings:
  # ...
  aws:elasticbeanstalk:environment:proxy:  # new
    ProxyServer: apache                    # new

Your final 01_fastapi.config file should now look like this:

# .ebextensions/01_fastapi.config

option_settings:
  aws:elasticbeanstalk:application:environment:
    PYTHONPATH: "/var/app/current:$PYTHONPATH"
  aws:elasticbeanstalk:container:python:
    WSGIPath: "main:app"
  aws:elasticbeanstalk:environment:proxy:
    ProxyServer: apache
container_commands:
  01_initdb:
    command: "source /var/app/venv/*/bin/activate && python3 init_db.py"
    leader_only: true

Next, create a ".platform" folder in the project root and add the following files and folders:

└-- .platform
    └-- httpd
        └-- conf.d
            └-- ssl_rewrite.conf

ssl_rewrite.conf:

# .platform/httpd/conf.d/ssl_rewrite.conf

RewriteEngine On
<If "-n '%{HTTP:X-Forwarded-Proto}' && %{HTTP:X-Forwarded-Proto} != 'https'">
RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI} [R,L]
</If>

Your project structure should now look like this:

|-- .ebextensions
|   └-- 01_fastapi.config
|-- .elasticbeanstalk
|   └-- config.yml
|-- .gitignore
├── .platform
│   └── httpd
│       └── conf.d
│           └── ssl_rewrite.conf
|-- Procfile
|-- README.md
|-- database.py
|-- default.db
|-- init_db.py
|-- main.py
|-- models.py
└-- requirements.txt

Validez les modifications sur git et déployez :

$ git add .
$ git commit -m "updates for eb"

$ eb deploy

Maintenant, dans votre navigateur, la https://saveur de votre application devrait fonctionner. Essayez d'aller à la http://saveur. Vous devriez être redirigé vers la https://saveur. Assurez-vous également que le certificat est correctement chargé :

Variables d'environnement

En production, il est préférable de stocker la configuration spécifique à l'environnement dans des variables d'environnement . Avec Elastic Beanstalk, vous pouvez définir des variables d'environnement personnalisées de deux manières différentes.

Variables d'environnement via EB CLI

Vous pouvez exécuter une commande pour les définir comme ceci :

$ eb setenv VARIABLE_NAME='variable value'

Vous pouvez définir plusieurs variables d'environnement avec une seule commande en les séparant par des espaces. Il s'agit de l'approche recommandée car elle n'entraîne qu'une seule mise à jour de l'environnement EB.

Vous pouvez ensuite accéder à ces variables dans votre environnement Python via os.environ.

Par exemple:

VARIABLE_NAME = os.environ['VARIABLE_NAME']

Variables d'environnement via la console EB

Accédez à la console Elastic Beanstalk via eb open. Naviguez jusqu'à "Configuration" > "Logiciel" > "Modifier". Ensuite, faites défiler jusqu'aux "Propriétés de l'environnement".

Une fois que vous avez terminé, cliquez sur "Appliquer" et votre environnement sera mis à jour.

Encore une fois, vous pouvez accéder aux variables d'environnement en Python via os.environ.

Débogage d'Elastic Beanstalk

Lorsque vous travaillez avec Elastic Beanstalk, il peut être assez frustrant de comprendre ce qui ne va pas si vous ne savez pas comment accéder aux fichiers journaux. Dans cette section, nous n'examinerons que cela.

Il existe deux façons d'accéder aux journaux :

1. CLI ou console Elastic Beanstalk
2. SSH dans l'instance EC2

Par expérience personnelle, j'ai pu résoudre tous les problèmes avec la première approche.

CLI ou console Elastic Beanstalk

CLI :

$ eb logs

Cette commande récupère les 100 dernières lignes des fichiers suivants :

/var/log/web.stdout.log
/var/log/eb-hooks.log
/var/log/nginx/access.log
/var/log/nginx/error.log
/var/log/eb-engine.log

L'exécution eb logséquivaut à se connecter à la console EB et à naviguer vers "Journaux".

Je recommande de diriger les journaux vers CloudWatch . Exécutez la commande suivante pour l'activer :

$ eb logs --cloudwatch-logs enable

Vous trouverez généralement des erreurs FastAPI dans /var/log/web.stdout.log ou /var/log/eb-engine.log .

Pour en savoir plus sur les journaux Elastic Beanstalk, consultez Affichage des journaux des instances Amazon EC2 .

SSH dans l'instance EC2

Pour vous connecter à une instance EC2 sur laquelle votre application FastAPI est en cours d'exécution, exécutez :

$ eb ssh

Vous serez invité à ajouter l'hôte à vos hôtes connus la première fois. Dis oui. Avec cela, vous aurez désormais un accès complet à votre instance EC2. N'hésitez pas à vérifier certains des fichiers journaux mentionnés dans la section précédente.

Gardez à l'esprit qu'Elastic Beanstalk met automatiquement à l'échelle et déploie de nouvelles instances EC2. Les modifications que vous apportez à cette instance EC2 spécifique ne seront pas répercutées sur les instances EC2 nouvellement lancées. Une fois cette instance EC2 spécifique remplacée, vos modifications seront effacées.

Conclusion

Dans ce didacticiel, nous avons parcouru le processus de déploiement d'une application FastAPI sur AWS Elastic Beanstalk. Vous devriez maintenant avoir une bonne compréhension du fonctionnement d'Elastic Beanstalk. Effectuez une auto-vérification rapide en passant en revue les objectifs au début du didacticiel.

Prochaines étapes:

1. Vous devriez envisager de créer deux environnements EB distincts ( devet production).
2. Consultez le groupe Auto Scaling pour votre environnement Elastic Beanstalk pour savoir comment configurer des déclencheurs pour la mise à l'échelle automatique de votre application.

Pour supprimer toutes les ressources AWS que nous avons créées tout au long du didacticiel, arrêtez d'abord l'environnement Elastic Beanstalk :

$ eb terminate

Vous devrez supprimer manuellement le certificat SSL.

Enfin, vous pouvez trouver la version finale du code dans le référentiel fastapi-elastic-beanstalk sur GitHub.

Source :  https://testdrive.io

#fastapi #aws #elastic 

What is GEEK

Buddha Community

Comment Déployer une Application FastAPI sur AWS Elastic Beanstalk

Dans ce didacticiel, nous allons parcourir le processus de déploiement d'une application FastAPI sur AWS Elastic Beanstalk .

Objectifs

À la fin de ce didacticiel, vous serez en mesure de :

1. Expliquer ce qu'est Elastic Beanstalk
2. Initialiser et configurer Elastic Beanstalk
3. Dépanner une application exécutée sur Elastic Beanstalk
4. Intégrer Elastic Beanstalk à RDS
5. Obtenir un certificat SSL via AWS Certificate Manager
6. Utilisez un certificat SSL pour servir votre application sur HTTPS

Qu'est-ce qu'Elastic Beanstalk ?

AWS Elastic Beanstalk (EB) est un service facile à utiliser pour le déploiement et la mise à l'échelle des applications Web. Il connecte plusieurs services AWS, comme les instances de calcul ( EC2 ), les bases de données ( RDS ), les équilibreurs de charge ( Application Load Balancer ) et les systèmes de stockage de fichiers ( S3 ), pour n'en nommer que quelques-uns. EB vous permet de développer et de déployer rapidement votre application Web sans vous soucier de l'infrastructure sous-jacente. Il prend en charge les applications développées en Go, Java, .NET, Node.js, PHP, Python et Ruby. EB prend également en charge Docker si vous avez besoin de configurer votre propre pile logicielle ou de déployer une application développée dans un langage (ou une version) qu'EB ne prend pas actuellement en charge.

Configuration typique d'Elastic Beanstalk :

There's no additional charge for AWS Elastic Beanstalk. You only pay for the resources that your application consumes.

To learn more about Elastic Beanstalk, check out What is AWS Elastic Beanstalk? from the official AWS Elastic Beanstalk documentation.

Elastic Beanstalk Concepts

Before diving into tutorial itself, let's look at a few key concepts related to Elastic Beanstalk:

1. An application is a logical collection of Elastic Beanstalk components, including environments, versions, and environment configurations. An application can have multiple versions.
2. An environment is a collection of AWS resources running an application version.
3. A platform is a combination of an operating system, programming language runtime, web server, application server, and Elastic Beanstalk components.

These terms will be used throughout the tutorial.

Project Setup

We'll be deploying a simple FastAPI application called fastapi-songs in this tutorial.

Check your understanding by deploying your own application as you follow along with the tutorial.

First, grab the code from the repository on GitHub:

$ git clone git@github.com:duplxey/fastapi-songs.git
$ cd fastapi-songs

Create a new virtual environment and activate it:

$ python3 -m venv venv && source venv/bin/activate

Install the requirements and initialize the database:

(venv)$ pip install -r requirements.txt
(venv)$ python init_db.py

Run the server:

(venv)$ uvicorn main:app --reload

Open your favorite web browser and navigate to:

1. http://localhost:8000 - should display "fastapi-songs" text
2. http://localhost:8000/songs - should display a list of songs

Elastic Beanstalk CLI

Be sure to register for an AWS account before continuing. By creating an account you might also be eligible for the AWS Free Tier.

The Elastic Beanstalk command line interface (EB CLI) allows you to perform a variety of operations to deploy and manage your Elastic Beanstalk applications and environments.

There are two ways of installing EB CLI:

1. Via the EB CLI installer
2. With pip (awsebcli)

It's recommended to install the EB CLI globally (outside any specific virtual environment) using the installer (first option) to avoid possible dependency conflicts. Refer to this explanation for more details.

After you've installed the EB CLI, you can check the version by running:

$ eb --version

EB CLI 3.20.3 (Python 3.10.)

If the command doesn't work, you may need to add the EB CLI to $PATH.

A list of EB CLI commands and their descriptions can be found in the EB CLI command reference.

Initialize Elastic Beanstalk

Once we have the EB CLI running we can start interacting with Elastic Beanstalk. Let's initialize a new project along with an EB Environment.

Init

Within the project root ("fastapi-songs"), run:

$ eb init

You'll be prompted with a number of questions.

Default Region

The AWS region of your Elastic Beanstalk environment (and resources). If you're not familiar with the different AWS regions, check out AWS Regions and Availability Zones. Generally, you should pick the region that's closest to your customers. Keep in mind that resource prices vary from region to region.

Application Name

This is the name of your Elastic Beanstalk application. I recommend just pressing enter and going with the default: "fastapi-songs".

Platform and Platform Branch

The EB CLI will detect that you're using a Python environment. After that, it'll give you different Python versions and Amazon Linux versions you can work with. Pick "Python 3.8 running on 64bit Amazon Linux 2".

CodeCommit

CodeCommit is a secure, highly scalable, managed source control service that hosts private Git repositories. We won't be using it since we're already using GitHub for source control. So say "no".

SSH

To connect to the EC2 instances later we need to set up SSH. Say "yes" when prompted.

Keypair

To connect to EC2 instances, we'll need an RSA keypair. Go ahead and generate one, which will be added to your "~/.ssh" folder.

After you answer all the questions, you'll notice a hidden directory inside your project root named ".elasticbeanstalk". The directory should contain a config.yml file, with all the data you've just provided.

.elasticbeanstalk
└── config.yml

The file should contain something similar to:

branch-defaults:
  master:
    environment: null
    group_suffix: null
global:
  application_name: fastapi-songs
  branch: null
  default_ec2_keyname: aws-eb
  default_platform: Python 3.8 running on 64bit Amazon Linux 2
  default_region: us-west-2
  include_git_submodules: true
  instance_profile: null
  platform_name: null
  platform_version: null
  profile: eb-cli
  repository: null
  sc: git
  workspace_type: Application

Create

Next, let's create the Elastic Beanstalk environment and deploy the application:

$ eb create

Again, you'll be prompted with a few questions.

Environment Name

This represents the name of the EB environment. I'd recommend sticking with the default: "fastapi-songs-env".

It's considered good practice to add └-env or └-dev suffix to your environments so you can easily differentiate EB apps from environments.

DNS CNAME Prefix

Your web application will be accessible at %cname%.%region%.elasticbeanstalk.com. Again, use the default.

Load balancer

A load balancer distributes traffic amongst your environment's instances. Select "application".

If you want to learn about the different load balancer types, review Load balancer for your Elastic Beanstalk Environment.

Spot Fleet Requests

Spot Fleet requests allow you to launch instances on-demand based on your criteria. We won't be using them in this tutorial, so say "no".

--

With that, the environment will be spun up:

1. Your code will be zipped up and uploaded to a new S3 Bucket.
2. After that, the various AWS resources will be created, like the load balancer, security and auto-scaling groups, and EC2 instances.

A new application will be deployed as well.

This will take about three minutes so feel free to grab a cup of coffee.

After the deployment is done, the EB CLI will modify .elasticbeanstalk/config.yml.

Your project structure should now look like this:

|-- .elasticbeanstalk
|   └-- config.yml
|-- .gitignore
|-- README.md
|-- database.py
|-- default.db
|-- init_db.py
|-- main.py
|-- models.py
└-- requirements.txt

Status

Once you've deployed your app you can check its status by running:

$ eb status

Environment details for: fastapi-songs-env
  Application name: fastapi-songs
  Region: us-west-2
  Deployed Version: app-82fb-220311_171256090207
  Environment ID: e-nsizyek74z
  Platform: arn:aws:elasticbeanstalk:us-west-2::platform/Python 3.8 running on 64bit Amazon Linux 2/3.3.11
  Tier: WebServer-Standard-1.0
  CNAME: fastapi-songs-env.us-west-2.elasticbeanstalk.com
  Updated: 2022-03-11 23:16:03.822000+00:00
  Status: Launching
  Health: Red

You can see that our environment's current health is Red, which means that something went wrong. Don't worry about this just yet, we'll fix it in the next steps.

You can also see that AWS assigned us a CNAME which is our EB environment's domain name. We can access the web application by opening a browser and navigating to the CNAME.

Open

$ eb open

This command will open your default browser and navigate to the CNAME domain. You'll see 502 Bad Gateway, which we'll fix here shortly.

Console

$ eb console

This command will open the Elastic Beanstalk console in your default browser:

Again, you can see that the health of the environment is "Severe", which we'll fix in the next step.

Configure an Environment

In the previous step, we tried accessing our application and it returned 502 Bad Gateway. There are three reasons behind it:

1. Python needs PYTHONPATH in order to find modules in our application.
2. By default, Elastic Beanstalk attempts to launch the WSGI application from application.py, which doesn't exist.
3. If not specified otherwise, Elastic Beanstalk tries to serve Python applications with Gunicorn. Gunicorn by itself is not compatible with FastAPI since FastAPI uses the newest ASGI standard.

Let's fix these errors.

Create a new folder in the project root called ".ebextensions". Within the newly created folder create a file named 01_fastapi.config:

# .ebextensions/01_fastapi.config

option_settings:
  aws:elasticbeanstalk:application:environment:
    PYTHONPATH: "/var/app/current:$PYTHONPATH"
  aws:elasticbeanstalk:container:python:
    WSGIPath: "main:app"

Notes:

1. We set the PYTHONPATH to the Python path on our EC2 instance (docs).
2. We changed the WSGIPath to our WSGI application (docs).

How do EB .config files work?

1. You can have as many as you want.
2. They are loaded in the following order: 01_x, 02_x, 03_x, etc.
3. You do not have to memorize these settings; you can list all your environmental settings by running eb config.

If you want to learn more about advanced environment customization check out Advanced environment customization with configuration files.

Since FastAPI doesn't support Gunicorn by itself, we'll have to modify our web process command to use a Uvicorn worker class.

Uvicorn is the recommended ASGI web application server for FastAPI.

Create a new file called Procfile inside the project root:

# Procfile

web: gunicorn main:app --workers=4 --worker-class=uvicorn.workers.UvicornWorker

If you're deploying your own application, make sure to add uvicorn as a dependency to requirements.txt.

For more information on Procfiles review Configuring the WSGI server with a Procfile.

Lastly, we have to tell Elastic Beanstalk to initialize the database when a new application version gets deployed. Add the following to the end of .ebextensions/01_fastapi.config:

# .ebextensions/01_fastapi.config

container_commands:
  01_initdb:
    command: "source /var/app/venv/*/bin/activate && python3 init_db.py"
    leader_only: true

The EB environment will now execute the above command every time we deploy a new application version. We used leader_only, so only the first EC2 instance executes them (in case our EB environment runs multiple EC2 instances).

Elastic Beanstalk configs support two different command sections, commands and container_commands. The main difference between them is when they are run in the deployment process:

1. commands run before the application and web server are set up and the application version file is extracted.
2. container_commands run after the application and web server have been set up and the application version archive has been extracted, but before the application version is deployed (before the files are moved from the staging folder to their final location).

At this point your project structure should look like this:

|-- .ebextensions
|   └-- 01_fastapi.config
|-- .elasticbeanstalk
|   └-- config.yml
|-- .gitignore
|-- Procfile
|-- README.md
|-- database.py
|-- default.db
|-- init_db.py
|-- main.py
|-- models.py
`-- requirements.txt

Commit the changes to git and deploy:

$ git add .
$ git commit -m "updates for eb"

$ eb deploy

You'll notice that Elastic Beanstalk won't detect the changes if you don't commit. That's because EB integrates with git and only detects the committed (changed) files.

After the deployment is done, run eb open to see if everything worked. After that, append /songs to the URL to see if the songs still get displayed.

Yay! The first version of our app is now deployed.

Configure RDS

If you're deploying fastapi-songs, you'll notice that it uses a SQLite database by default. While this is perfect for development, you'll typically want to move to a more robust database, like Postgres or MySQL, for production. Let's look at how to swap SQLite for Postgres.

Local Postgres

First, let's get Postgres running locally. You can either download it from PostgreSQL Downloads or spin up a Docker container:

$ docker run --name fastapi-songs-postgres -p 5432:5432 \
    -e POSTGRES_USER=fastapi-songs -e POSTGRES_PASSWORD=complexpassword123 \
    -e POSTGRES_DB=fastapi-songs -d postgres

Check if the container is running:

$ docker ps -f name=fastapi-songs-postgres

CONTAINER ID   IMAGE      COMMAND                  CREATED              STATUS              PORTS                    NAMES
c05621dac852   postgres   "docker-entrypoint.s…"   About a minute ago   Up About a minute   0.0.0.0:5432->5432/tcp   fastapi-songs-postgres

Now, let's try connecting to it with our FastAPI app.

Inside database.py change DATABASE_URL like so:

# database.py

DATABASE_URL = \
    'postgresql://{username}:{password}@{host}:{port}/{database}'.format(
        username='fastapi-songs',
        password='complexpassword123',
        host='localhost',
        port='5432',
        database='fastapi-songs',
    )

After that remove connect_args from create_engine since check_same_thread is only required for SQLite.

# database.py

engine = create_engine(
    DATABASE_URL,
)

Next, install psycopg2-binary, which is required for Postgres:

(venv)$ pip install psycopg2-binary==2.9.3

Add it to requirements.txt:

fastapi==0.75.0
psycopg2-binary==2.9.3
SQLAlchemy==1.4.32
uvicorn[standard]==0.17.6

Remove the existing database, default.db, and then initialize the new database:

(venv)$ python init_db.py

Run the server:

(venv)$ uvicorn main:app --reload

Make sure the songs still get served correctly by checking out http://localhost:8000/songs.

AWS RDS Postgres

To set up Postgres for production, start by running the following command to open the AWS console:

$ eb console

Click "Configuration" on the left side bar, scroll down to "Database", and then click "Edit".

Create a DB with the following settings and click on "Apply":

• Engine: postgres
• Engine version: 12.9 (older Postgres version since db.t2.micro is not available with 13.1+)
• Instance class: db.t2.micro
• Storage: 5 GB (should be more than enough)
• Username: pick a username
• Password: pick a strong password

If you want to stay within the AWS Free Tier make sure you pick db.t2.micro. RDS prices increase exponentially based on the instance class you pick. If you don't want to go with micro make sure to review AWS PostgreSQL pricing.

After the environmental update is done, EB will automatically pass the following DB credentials to our FastAPI app:

RDS_DB_NAME
RDS_USERNAME
RDS_PASSWORD
RDS_HOSTNAME
RDS_PORT

We can now use these variables in database.py to connect to our database. Replace DATABASE_URL with the following:

if 'RDS_DB_NAME' in os.environ:
    DATABASE_URL = \
        'postgresql://{username}:{password}@{host}:{port}/{database}'.format(
            username=os.environ['RDS_USERNAME'],
            password=os.environ['RDS_PASSWORD'],
            host=os.environ['RDS_HOSTNAME'],
            port=os.environ['RDS_PORT'],
            database=os.environ['RDS_DB_NAME'],
        )
else:
    DATABASE_URL = \
        'postgresql://{username}:{password}@{host}:{port}/{database}'.format(
            username='fastapi-songs',
            password='complexpassword123',
            host='localhost',
            port='5432',
            database='fastapi-songs',
        )

Don't forget to import the os package at the top of database.py:

import os

Your final database.py file should look like this.

Commit the changes to git and deploy:

$ git add .
$ git commit -m "updates for eb"

$ eb deploy

Wait for the deployment to finish. Once done, run eb open to open your app in a new browser tab. Make sure everything still works correctly by listing the songs at /songs.

HTTPS with Certificate Manager

This part of the tutorial requires that you have a domain name.

Need a cheap domain to practice with? Several domain registrars have specials on '.xyz' domains. Alternatively, you can create a free domain at Freenom. If you don't own a domain name, but would still like to use HTTPS you can create and sign with an X509 certificate.

To serve your application via HTTPS, we'll need to:

1. Request and validate an SSL/TLS certificate
2. Point your domain name to your EB CNAME
3. Modify the load balancer to serve HTTPS
4. Modify your application settings

Request and Validate an SSL/TLS Certificate

Navigate to the AWS Certificate Manager console. Click "Request a certificate". Set the certificate type to "Public" and click "Next". Enter your fully qualified domain name into the form input, set the "Validation method" to "DNS validation", and click "Request".

You'll then be redirected to a page where you can see all your certificates. The certificate that you just created should have a status of "Pending validation".

For AWS to issue a certificate, you first have to prove that you're the owner of the domain. In the table, click on the certificate to view the "Certificate details". Take note of the "CNAME name" and "CNAME value". To validate the ownership of the domain, you'll need to create a CNAME Record" in your domain's DNS settings. Use the "CNAME name" and "CNAME value" for this. Once done, it will take a few minutes for Amazon to pick up the domain changes and issue the certificate. The status should change from "Pending validation" to "Issued".

Point the Domain Name to the EB CNAME

Next, you need to point your domain (or subdomain) to your EB environment CNAME. Back in your domain's DNS settings, add another CNAME record with the value being your EB CNAME -- e.g., fastapi-songs-dev.us-west-2.elasticbeanstalk.com.

Wait a few minutes for your DNS to refresh before testing things out from the http:// flavor of your domain name in your browser.

Modify the Load Balancer to serve HTTPS

Back in the Elastic Beanstalk console, click "Configuration". Then, within the "Load balancer" category, click "Edit". Click "Add listener" and create a listener with the following details:

1. Port - 443
2. Protocol - HTTPS
3. SSL certificate - select the certificate that you just created

Click "Add". Then, scroll to the bottom of the page and click "Apply". It will take a few minutes for the environment to update.

Modify your Application Settings

Next, we need to make a few changes to our FastAPI application.

We need to redirect all traffic from HTTP to HTTPS. There are multiple ways of doing this, but the easiest way is to set up Apache as a proxy host. We can achieve this programmatically by adding the following to the end of the option_settings in .ebextensions/01_fastapi.config:

# .ebextensions/01_fastapi.config

option_settings:
  # ...
  aws:elasticbeanstalk:environment:proxy:  # new
    ProxyServer: apache                    # new

Your final 01_fastapi.config file should now look like this:

# .ebextensions/01_fastapi.config

option_settings:
  aws:elasticbeanstalk:application:environment:
    PYTHONPATH: "/var/app/current:$PYTHONPATH"
  aws:elasticbeanstalk:container:python:
    WSGIPath: "main:app"
  aws:elasticbeanstalk:environment:proxy:
    ProxyServer: apache
container_commands:
  01_initdb:
    command: "source /var/app/venv/*/bin/activate && python3 init_db.py"
    leader_only: true

Next, create a ".platform" folder in the project root and add the following files and folders:

└-- .platform
    └-- httpd
        └-- conf.d
            └-- ssl_rewrite.conf

ssl_rewrite.conf:

# .platform/httpd/conf.d/ssl_rewrite.conf

RewriteEngine On
<If "-n '%{HTTP:X-Forwarded-Proto}' && %{HTTP:X-Forwarded-Proto} != 'https'">
RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI} [R,L]
</If>

Your project structure should now look like this:

|-- .ebextensions
|   └-- 01_fastapi.config
|-- .elasticbeanstalk
|   └-- config.yml
|-- .gitignore
├── .platform
│   └── httpd
│       └── conf.d
│           └── ssl_rewrite.conf
|-- Procfile
|-- README.md
|-- database.py
|-- default.db
|-- init_db.py
|-- main.py
|-- models.py
└-- requirements.txt

Validez les modifications sur git et déployez :

$ git add .
$ git commit -m "updates for eb"

$ eb deploy

Maintenant, dans votre navigateur, la https://saveur de votre application devrait fonctionner. Essayez d'aller à la http://saveur. Vous devriez être redirigé vers la https://saveur. Assurez-vous également que le certificat est correctement chargé :

Variables d'environnement

En production, il est préférable de stocker la configuration spécifique à l'environnement dans des variables d'environnement . Avec Elastic Beanstalk, vous pouvez définir des variables d'environnement personnalisées de deux manières différentes.

Variables d'environnement via EB CLI

Vous pouvez exécuter une commande pour les définir comme ceci :

$ eb setenv VARIABLE_NAME='variable value'

Vous pouvez définir plusieurs variables d'environnement avec une seule commande en les séparant par des espaces. Il s'agit de l'approche recommandée car elle n'entraîne qu'une seule mise à jour de l'environnement EB.

Vous pouvez ensuite accéder à ces variables dans votre environnement Python via os.environ.

Par exemple:

VARIABLE_NAME = os.environ['VARIABLE_NAME']

Variables d'environnement via la console EB

Accédez à la console Elastic Beanstalk via eb open. Naviguez jusqu'à "Configuration" > "Logiciel" > "Modifier". Ensuite, faites défiler jusqu'aux "Propriétés de l'environnement".

Une fois que vous avez terminé, cliquez sur "Appliquer" et votre environnement sera mis à jour.

Encore une fois, vous pouvez accéder aux variables d'environnement en Python via os.environ.

Débogage d'Elastic Beanstalk

Lorsque vous travaillez avec Elastic Beanstalk, il peut être assez frustrant de comprendre ce qui ne va pas si vous ne savez pas comment accéder aux fichiers journaux. Dans cette section, nous n'examinerons que cela.

Il existe deux façons d'accéder aux journaux :

1. CLI ou console Elastic Beanstalk
2. SSH dans l'instance EC2

Par expérience personnelle, j'ai pu résoudre tous les problèmes avec la première approche.

CLI ou console Elastic Beanstalk

CLI :

$ eb logs

Cette commande récupère les 100 dernières lignes des fichiers suivants :

/var/log/web.stdout.log
/var/log/eb-hooks.log
/var/log/nginx/access.log
/var/log/nginx/error.log
/var/log/eb-engine.log

L'exécution eb logséquivaut à se connecter à la console EB et à naviguer vers "Journaux".

Je recommande de diriger les journaux vers CloudWatch . Exécutez la commande suivante pour l'activer :

$ eb logs --cloudwatch-logs enable

Vous trouverez généralement des erreurs FastAPI dans /var/log/web.stdout.log ou /var/log/eb-engine.log .

Pour en savoir plus sur les journaux Elastic Beanstalk, consultez Affichage des journaux des instances Amazon EC2 .

SSH dans l'instance EC2

Pour vous connecter à une instance EC2 sur laquelle votre application FastAPI est en cours d'exécution, exécutez :

$ eb ssh

Vous serez invité à ajouter l'hôte à vos hôtes connus la première fois. Dis oui. Avec cela, vous aurez désormais un accès complet à votre instance EC2. N'hésitez pas à vérifier certains des fichiers journaux mentionnés dans la section précédente.

Gardez à l'esprit qu'Elastic Beanstalk met automatiquement à l'échelle et déploie de nouvelles instances EC2. Les modifications que vous apportez à cette instance EC2 spécifique ne seront pas répercutées sur les instances EC2 nouvellement lancées. Une fois cette instance EC2 spécifique remplacée, vos modifications seront effacées.

Conclusion

Dans ce didacticiel, nous avons parcouru le processus de déploiement d'une application FastAPI sur AWS Elastic Beanstalk. Vous devriez maintenant avoir une bonne compréhension du fonctionnement d'Elastic Beanstalk. Effectuez une auto-vérification rapide en passant en revue les objectifs au début du didacticiel.

Prochaines étapes:

1. Vous devriez envisager de créer deux environnements EB distincts ( devet production).
2. Consultez le groupe Auto Scaling pour votre environnement Elastic Beanstalk pour savoir comment configurer des déclencheurs pour la mise à l'échelle automatique de votre application.

Pour supprimer toutes les ressources AWS que nous avons créées tout au long du didacticiel, arrêtez d'abord l'environnement Elastic Beanstalk :

$ eb terminate

Vous devrez supprimer manuellement le certificat SSL.

Enfin, vous pouvez trouver la version finale du code dans le référentiel fastapi-elastic-beanstalk sur GitHub.

Source :  https://testdrive.io

#fastapi #aws #elastic 

How to Use AWS Elastic Beanstalk to Reduce Risk of Deployment Downtime

You can use AWS Elastic Beanstalk to create and deploy an updated or upgrated application version with blue-green deployment using cloned configs.

In this piece, I’ll be demonstrating how AWS Elastic Beanstalk can simplify deployments by doing all the hard work for you – and with no risk of downtime – by employing a Blue/Green deployment strategy.

Using AWS means combining a large number of tools to complete projects. Personally, I choose to streamline this process by using Elastic Beanstalk, as it enables me and the rest of the dev team to control the AWS resources which power the applications we support and gives us full access to the underlying resources at any time.

#cloud #aws #elastic beanstalk #aws tools #aws elastic beanstalk

How to Deploying A FastAPI Application To AWS Elastic Beanstalk

In this tutorial, we'll walk through the process of deploying a FastAPI application to AWS Elastic Beanstalk.

Source: https://testdriven.io

#fastapi #aws #elastic 

How To Unite AWS KMS with Serverless Application Model (SAM)

The Basics

AWS KMS is a Key Management Service that let you create Cryptographic keys that you can use to encrypt and decrypt data and also other keys. You can read more about it here.

Important points about Keys

Please note that the customer master keys(CMK) generated can only be used to encrypt small amount of data like passwords, RSA key. You can use AWS KMS CMKs to generate, encrypt, and decrypt data keys. However, AWS KMS does not store, manage, or track your data keys, or perform cryptographic operations with data keys.

You must use and manage data keys outside of AWS KMS. KMS API uses AWS KMS CMK in the encryption operations and they cannot accept more than 4 KB (4096 bytes) of data. To encrypt application data, use the server-side encryption features of an AWS service, or a client-side encryption library, such as the AWS Encryption SDK or the Amazon S3 encryption client.

Scenario

We want to create signup and login forms for a website.

Passwords should be encrypted and stored in DynamoDB database.

What do we need?

1. KMS key to encrypt and decrypt data
2. DynamoDB table to store password.
3. Lambda functions & APIs to process Login and Sign up forms.
4. Sign up/ Login forms in HTML.

Lets Implement it as Serverless Application Model (SAM)!

Lets first create the Key that we will use to encrypt and decrypt password.

KmsKey:
    Type: AWS::KMS::Key
    Properties: 
      Description: CMK for encrypting and decrypting
      KeyPolicy:
        Version: '2012-10-17'
        Id: key-default-1
        Statement:
        - Sid: Enable IAM User Permissions
          Effect: Allow
          Principal:
            AWS: !Sub arn:aws:iam::${AWS::AccountId}:root
          Action: kms:*
          Resource: '*'
        - Sid: Allow administration of the key
          Effect: Allow
          Principal:
            AWS: !Sub arn:aws:iam::${AWS::AccountId}:user/${KeyAdmin}
          Action:
          - kms:Create*
          - kms:Describe*
          - kms:Enable*
          - kms:List*
          - kms:Put*
          - kms:Update*
          - kms:Revoke*
          - kms:Disable*
          - kms:Get*
          - kms:Delete*
          - kms:ScheduleKeyDeletion
          - kms:CancelKeyDeletion
          Resource: '*'
        - Sid: Allow use of the key
          Effect: Allow
          Principal:
            AWS: !Sub arn:aws:iam::${AWS::AccountId}:user/${KeyUser}
          Action:
          - kms:DescribeKey
          - kms:Encrypt
          - kms:Decrypt
          - kms:ReEncrypt*
          - kms:GenerateDataKey
          - kms:GenerateDataKeyWithoutPlaintext
          Resource: '*'

The important thing in above snippet is the KeyPolicy. KMS requires a Key Administrator and Key User. As a best practice your Key Administrator and Key User should be 2 separate user in your Organisation. We are allowing all permissions to the root users.

So if your key Administrator leaves the organisation, the root user will be able to delete this key. As you can see **KeyAdmin **can manage the key but not use it and KeyUser can only use the key. ${KeyAdmin} and **${KeyUser} **are parameters in the SAM template.

You would be asked to provide values for these parameters during SAM Deploy.

#aws #serverless #aws-sam #aws-key-management-service #aws-certification #aws-api-gateway #tutorial-for-beginners #aws-blogs

Deploying A FastAPI Application to Elastic Beanstalk

In this tutorial, we'll walk through the process of deploying a FastAPI application to AWS Elastic Beanstalk.

Objectives

By the end of this tutorial, you'll be able to:

1. Explain what Elastic Beanstalk is
2. Initialize and configure Elastic Beanstalk
3. Troubleshoot an application running on Elastic Beanstalk
4. Integrate Elastic Beanstalk with RDS
5. Obtain an SSL certificate via AWS Certificate Manager
6. Use an SSL certificate to serve your application on HTTPS

What is Elastic Beanstalk?

AWS Elastic Beanstalk (EB) is an easy-to-use service for deploying and scaling web applications. It connects multiple AWS services, like compute instances (EC2), databases (RDS), load balancers (Application Load Balancer), and file storage systems (S3), to name a few. EB allows you to quickly develop and deploy your web application without thinking about the underlying infrastructure. It supports applications developed in Go, Java, .NET, Node.js, PHP, Python, and Ruby. EB also supports Docker if you need to configure your own software stack or deploy an application developed in a language (or version) that EB doesn't currently support.

Typical Elastic Beanstalk setup:

There's no additional charge for AWS Elastic Beanstalk. You only pay for the resources that your application consumes.

To learn more about Elastic Beanstalk, check out What is AWS Elastic Beanstalk? from the official AWS Elastic Beanstalk documentation.

Elastic Beanstalk Concepts

Before diving into tutorial itself, let's look at a few key concepts related to Elastic Beanstalk:

1. An application is a logical collection of Elastic Beanstalk components, including environments, versions, and environment configurations. An application can have multiple versions.
2. An environment is a collection of AWS resources running an application version.
3. A platform is a combination of an operating system, programming language runtime, web server, application server, and Elastic Beanstalk components.

These terms will be used throughout the tutorial.

Project Setup

We'll be deploying a simple FastAPI application called fastapi-songs in this tutorial.

Check your understanding by deploying your own application as you follow along with the tutorial.

First, grab the code from the repository on GitHub:

$ git clone git@github.com:duplxey/fastapi-songs.git
$ cd fastapi-songs

Create a new virtual environment and activate it:

$ python3 -m venv venv && source venv/bin/activate

Install the requirements and initialize the database:

(venv)$ pip install -r requirements.txt
(venv)$ python init_db.py

Run the server:

(venv)$ uvicorn main:app --reload

Open your favorite web browser and navigate to:

1. http://localhost:8000 - should display "fastapi-songs" text
2. http://localhost:8000/songs - should display a list of songs

Elastic Beanstalk CLI

Be sure to register for an AWS account before continuing. By creating an account you might also be eligible for the AWS Free Tier.

The Elastic Beanstalk command line interface (EB CLI) allows you to perform a variety of operations to deploy and manage your Elastic Beanstalk applications and environments.

There are two ways of installing EB CLI:

1. Via the EB CLI installer
2. With pip (awsebcli)

It's recommended to install the EB CLI globally (outside any specific virtual environment) using the installer (first option) to avoid possible dependency conflicts. Refer to this explanation for more details.

After you've installed the EB CLI, you can check the version by running:

$ eb --version

EB CLI 3.20.3 (Python 3.10.)

If the command doesn't work, you may need to add the EB CLI to $PATH.

A list of EB CLI commands and their descriptions can be found in the EB CLI command reference.

Initialize Elastic Beanstalk

Once we have the EB CLI running we can start interacting with Elastic Beanstalk. Let's initialize a new project along with an EB Environment.

Init

Within the project root ("fastapi-songs"), run:

$ eb init

You'll be prompted with a number of questions.

Default Region

The AWS region of your Elastic Beanstalk environment (and resources). If you're not familiar with the different AWS regions, check out AWS Regions and Availability Zones. Generally, you should pick the region that's closest to your customers. Keep in mind that resource prices vary from region to region.

Application Name

This is the name of your Elastic Beanstalk application. I recommend just pressing enter and going with the default: "fastapi-songs".

Platform and Platform Branch

The EB CLI will detect that you're using a Python environment. After that, it'll give you different Python versions and Amazon Linux versions you can work with. Pick "Python 3.8 running on 64bit Amazon Linux 2".

CodeCommit

CodeCommit is a secure, highly scalable, managed source control service that hosts private Git repositories. We won't be using it since we're already using GitHub for source control. So say "no".

SSH

To connect to the EC2 instances later we need to set up SSH. Say "yes" when prompted.

Keypair

To connect to EC2 instances, we'll need an RSA keypair. Go ahead and generate one, which will be added to your "~/.ssh" folder.

After you answer all the questions, you'll notice a hidden directory inside your project root named ".elasticbeanstalk". The directory should contain a config.yml file, with all the data you've just provided.

.elasticbeanstalk
└── config.yml

The file should contain something similar to:

branch-defaults:
  master:
    environment: null
    group_suffix: null
global:
  application_name: fastapi-songs
  branch: null
  default_ec2_keyname: aws-eb
  default_platform: Python 3.8 running on 64bit Amazon Linux 2
  default_region: us-west-2
  include_git_submodules: true
  instance_profile: null
  platform_name: null
  platform_version: null
  profile: eb-cli
  repository: null
  sc: git
  workspace_type: Application

Create

Next, let's create the Elastic Beanstalk environment and deploy the application:

$ eb create

Again, you'll be prompted with a few questions.

Environment Name

This represents the name of the EB environment. I'd recommend sticking with the default: "fastapi-songs-env".

It's considered good practice to add └-env or └-dev suffix to your environments so you can easily differentiate EB apps from environments.

DNS CNAME Prefix

Your web application will be accessible at %cname%.%region%.elasticbeanstalk.com. Again, use the default.

Load balancer

A load balancer distributes traffic amongst your environment's instances. Select "application".

If you want to learn about the different load balancer types, review Load balancer for your Elastic Beanstalk Environment.

Spot Fleet Requests

Spot Fleet requests allow you to launch instances on-demand based on your criteria. We won't be using them in this tutorial, so say "no".

--

With that, the environment will be spun up:

1. Your code will be zipped up and uploaded to a new S3 Bucket.
2. After that, the various AWS resources will be created, like the load balancer, security and auto-scaling groups, and EC2 instances.

A new application will be deployed as well.

This will take about three minutes so feel free to grab a cup of coffee.

After the deployment is done, the EB CLI will modify .elasticbeanstalk/config.yml.

Your project structure should now look like this:

|-- .elasticbeanstalk
|   └-- config.yml
|-- .gitignore
|-- README.md
|-- database.py
|-- default.db
|-- init_db.py
|-- main.py
|-- models.py
└-- requirements.txt

Status

Once you've deployed your app you can check its status by running:

$ eb status

Environment details for: fastapi-songs-env
  Application name: fastapi-songs
  Region: us-west-2
  Deployed Version: app-82fb-220311_171256090207
  Environment ID: e-nsizyek74z
  Platform: arn:aws:elasticbeanstalk:us-west-2::platform/Python 3.8 running on 64bit Amazon Linux 2/3.3.11
  Tier: WebServer-Standard-1.0
  CNAME: fastapi-songs-env.us-west-2.elasticbeanstalk.com
  Updated: 2022-03-11 23:16:03.822000+00:00
  Status: Launching
  Health: Red

You can see that our environment's current health is Red, which means that something went wrong. Don't worry about this just yet, we'll fix it in the next steps.

You can also see that AWS assigned us a CNAME which is our EB environment's domain name. We can access the web application by opening a browser and navigating to the CNAME.

Open

$ eb open

This command will open your default browser and navigate to the CNAME domain. You'll see 502 Bad Gateway, which we'll fix here shortly.

Console

$ eb console

This command will open the Elastic Beanstalk console in your default browser:

Again, you can see that the health of the environment is "Severe", which we'll fix in the next step.

Configure an Environment

In the previous step, we tried accessing our application and it returned 502 Bad Gateway. There are three reasons behind it:

1. Python needs PYTHONPATH in order to find modules in our application.
2. By default, Elastic Beanstalk attempts to launch the WSGI application from application.py, which doesn't exist.
3. If not specified otherwise, Elastic Beanstalk tries to serve Python applications with Gunicorn. Gunicorn by itself is not compatible with FastAPI since FastAPI uses the newest ASGI standard.

Let's fix these errors.

Create a new folder in the project root called ".ebextensions". Within the newly created folder create a file named 01_fastapi.config:

# .ebextensions/01_fastapi.config

option_settings:
  aws:elasticbeanstalk:application:environment:
    PYTHONPATH: "/var/app/current:$PYTHONPATH"
  aws:elasticbeanstalk:container:python:
    WSGIPath: "main:app"

Notes:

1. We set the PYTHONPATH to the Python path on our EC2 instance (docs).
2. We changed the WSGIPath to our WSGI application (docs).

How do EB .config files work?

1. You can have as many as you want.
2. They are loaded in the following order: 01_x, 02_x, 03_x, etc.
3. You do not have to memorize these settings; you can list all your environmental settings by running eb config.

If you want to learn more about advanced environment customization check out Advanced environment customization with configuration files.

Since FastAPI doesn't support Gunicorn by itself, we'll have to modify our web process command to use a Uvicorn worker class.

Uvicorn is the recommended ASGI web application server for FastAPI.

Create a new file called Procfile inside the project root:

# Procfile

web: gunicorn main:app --workers=4 --worker-class=uvicorn.workers.UvicornWorker

If you're deploying your own application, make sure to add uvicorn as a dependency to requirements.txt.

For more information on Procfiles review Configuring the WSGI server with a Procfile.

Lastly, we have to tell Elastic Beanstalk to initialize the database when a new application version gets deployed. Add the following to the end of .ebextensions/01_fastapi.config:

# .ebextensions/01_fastapi.config

container_commands:
  01_initdb:
    command: "source /var/app/venv/*/bin/activate && python3 init_db.py"
    leader_only: true

The EB environment will now execute the above command every time we deploy a new application version. We used leader_only, so only the first EC2 instance executes them (in case our EB environment runs multiple EC2 instances).

Elastic Beanstalk configs support two different command sections, commands and container_commands. The main difference between them is when they are run in the deployment process:

1. commands run before the application and web server are set up and the application version file is extracted.
2. container_commands run after the application and web server have been set up and the application version archive has been extracted, but before the application version is deployed (before the files are moved from the staging folder to their final location).

At this point your project structure should look like this:

|-- .ebextensions
|   └-- 01_fastapi.config
|-- .elasticbeanstalk
|   └-- config.yml
|-- .gitignore
|-- Procfile
|-- README.md
|-- database.py
|-- default.db
|-- init_db.py
|-- main.py
|-- models.py
`-- requirements.txt

Commit the changes to git and deploy:

$ git add .
$ git commit -m "updates for eb"

$ eb deploy

You'll notice that Elastic Beanstalk won't detect the changes if you don't commit. That's because EB integrates with git and only detects the committed (changed) files.

After the deployment is done, run eb open to see if everything worked. After that, append /songs to the URL to see if the songs still get displayed.

Yay! The first version of our app is now deployed.

Configure RDS

If you're deploying fastapi-songs, you'll notice that it uses a SQLite database by default. While this is perfect for development, you'll typically want to move to a more robust database, like Postgres or MySQL, for production. Let's look at how to swap SQLite for Postgres.

Local Postgres

First, let's get Postgres running locally. You can either download it from PostgreSQL Downloads or spin up a Docker container:

$ docker run --name fastapi-songs-postgres -p 5432:5432 \
    -e POSTGRES_USER=fastapi-songs -e POSTGRES_PASSWORD=complexpassword123 \
    -e POSTGRES_DB=fastapi-songs -d postgres

Check if the container is running:

$ docker ps -f name=fastapi-songs-postgres

CONTAINER ID   IMAGE      COMMAND                  CREATED              STATUS              PORTS                    NAMES
c05621dac852   postgres   "docker-entrypoint.s…"   About a minute ago   Up About a minute   0.0.0.0:5432->5432/tcp   fastapi-songs-postgres

Now, let's try connecting to it with our FastAPI app.

Inside database.py change DATABASE_URL like so:

# database.py

DATABASE_URL = \
    'postgresql://{username}:{password}@{host}:{port}/{database}'.format(
        username='fastapi-songs',
        password='complexpassword123',
        host='localhost',
        port='5432',
        database='fastapi-songs',
    )

After that remove connect_args from create_engine since check_same_thread is only required for SQLite.

# database.py

engine = create_engine(
    DATABASE_URL,
)

Next, install psycopg2-binary, which is required for Postgres:

(venv)$ pip install psycopg2-binary==2.9.3

Add it to requirements.txt:

fastapi==0.75.0
psycopg2-binary==2.9.3
SQLAlchemy==1.4.32
uvicorn[standard]==0.17.6

Remove the existing database, default.db, and then initialize the new database:

(venv)$ python init_db.py

Run the server:

(venv)$ uvicorn main:app --reload

Make sure the songs still get served correctly by checking out http://localhost:8000/songs.

AWS RDS Postgres

To set up Postgres for production, start by running the following command to open the AWS console:

$ eb console

Click "Configuration" on the left side bar, scroll down to "Database", and then click "Edit".

Create a DB with the following settings and click on "Apply":

• Engine: postgres
• Engine version: 12.9 (older Postgres version since db.t2.micro is not available with 13.1+)
• Instance class: db.t2.micro
• Storage: 5 GB (should be more than enough)
• Username: pick a username
• Password: pick a strong password

If you want to stay within the AWS Free Tier make sure you pick db.t2.micro. RDS prices increase exponentially based on the instance class you pick. If you don't want to go with micro make sure to review AWS PostgreSQL pricing.

After the environmental update is done, EB will automatically pass the following DB credentials to our FastAPI app:

RDS_DB_NAME
RDS_USERNAME
RDS_PASSWORD
RDS_HOSTNAME
RDS_PORT

We can now use these variables in database.py to connect to our database. Replace DATABASE_URL with the following:

if 'RDS_DB_NAME' in os.environ:
    DATABASE_URL = \
        'postgresql://{username}:{password}@{host}:{port}/{database}'.format(
            username=os.environ['RDS_USERNAME'],
            password=os.environ['RDS_PASSWORD'],
            host=os.environ['RDS_HOSTNAME'],
            port=os.environ['RDS_PORT'],
            database=os.environ['RDS_DB_NAME'],
        )
else:
    DATABASE_URL = \
        'postgresql://{username}:{password}@{host}:{port}/{database}'.format(
            username='fastapi-songs',
            password='complexpassword123',
            host='localhost',
            port='5432',
            database='fastapi-songs',
        )

Don't forget to import the os package at the top of database.py:

import os

Your final database.py file should look like this.

Commit the changes to git and deploy:

$ git add .
$ git commit -m "updates for eb"

$ eb deploy

Wait for the deployment to finish. Once done, run eb open to open your app in a new browser tab. Make sure everything still works correctly by listing the songs at /songs.

HTTPS with Certificate Manager

This part of the tutorial requires that you have a domain name.

Need a cheap domain to practice with? Several domain registrars have specials on '.xyz' domains. Alternatively, you can create a free domain at Freenom. If you don't own a domain name, but would still like to use HTTPS you can create and sign with an X509 certificate.

To serve your application via HTTPS, we'll need to:

1. Request and validate an SSL/TLS certificate
2. Point your domain name to your EB CNAME
3. Modify the load balancer to serve HTTPS
4. Modify your application settings

Request and Validate an SSL/TLS Certificate

Navigate to the AWS Certificate Manager console. Click "Request a certificate". Set the certificate type to "Public" and click "Next". Enter your fully qualified domain name into the form input, set the "Validation method" to "DNS validation", and click "Request".

You'll then be redirected to a page where you can see all your certificates. The certificate that you just created should have a status of "Pending validation".

For AWS to issue a certificate, you first have to prove that you're the owner of the domain. In the table, click on the certificate to view the "Certificate details". Take note of the "CNAME name" and "CNAME value". To validate the ownership of the domain, you'll need to create a CNAME Record" in your domain's DNS settings. Use the "CNAME name" and "CNAME value" for this. Once done, it will take a few minutes for Amazon to pick up the domain changes and issue the certificate. The status should change from "Pending validation" to "Issued".

Point the Domain Name to the EB CNAME

Next, you need to point your domain (or subdomain) to your EB environment CNAME. Back in your domain's DNS settings, add another CNAME record with the value being your EB CNAME -- e.g., fastapi-songs-dev.us-west-2.elasticbeanstalk.com.

Wait a few minutes for your DNS to refresh before testing things out from the http:// flavor of your domain name in your browser.

Modify the Load Balancer to serve HTTPS

Back in the Elastic Beanstalk console, click "Configuration". Then, within the "Load balancer" category, click "Edit". Click "Add listener" and create a listener with the following details:

1. Port - 443
2. Protocol - HTTPS
3. SSL certificate - select the certificate that you just created

Click "Add". Then, scroll to the bottom of the page and click "Apply". It will take a few minutes for the environment to update.

Modify your Application Settings

Next, we need to make a few changes to our FastAPI application.

We need to redirect all traffic from HTTP to HTTPS. There are multiple ways of doing this, but the easiest way is to set up Apache as a proxy host. We can achieve this programmatically by adding the following to the end of the option_settings in .ebextensions/01_fastapi.config:

# .ebextensions/01_fastapi.config

option_settings:
  # ...
  aws:elasticbeanstalk:environment:proxy:  # new
    ProxyServer: apache                    # new

Your final 01_fastapi.config file should now look like this:

# .ebextensions/01_fastapi.config

option_settings:
  aws:elasticbeanstalk:application:environment:
    PYTHONPATH: "/var/app/current:$PYTHONPATH"
  aws:elasticbeanstalk:container:python:
    WSGIPath: "main:app"
  aws:elasticbeanstalk:environment:proxy:
    ProxyServer: apache
container_commands:
  01_initdb:
    command: "source /var/app/venv/*/bin/activate && python3 init_db.py"
    leader_only: true

Next, create a ".platform" folder in the project root and add the following files and folders:

└-- .platform
    └-- httpd
        └-- conf.d
            └-- ssl_rewrite.conf

ssl_rewrite.conf:

# .platform/httpd/conf.d/ssl_rewrite.conf

RewriteEngine On
<If "-n '%{HTTP:X-Forwarded-Proto}' && %{HTTP:X-Forwarded-Proto} != 'https'">
RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI} [R,L]
</If>

Your project structure should now look like this:

|-- .ebextensions
|   └-- 01_fastapi.config
|-- .elasticbeanstalk
|   └-- config.yml
|-- .gitignore
├── .platform
│   └── httpd
│       └── conf.d
│           └── ssl_rewrite.conf
|-- Procfile
|-- README.md
|-- database.py
|-- default.db
|-- init_db.py
|-- main.py
|-- models.py
└-- requirements.txt

Commit the changes to git and deploy:

$ git add .
$ git commit -m "updates for eb"

$ eb deploy

Now, in your browser, the https:// flavor of your application should work. Try going to the http:// flavor. You should be redirected to the https:// flavor. Ensure the certificate is loaded properly as well:

Environment Variables

In production, it's best to store environment-specific config in environment variables. With Elastic Beanstalk you can set custom environmental variables two different ways.

Environment Variables via EB CLI

You can execute a command to set them like so:

$ eb setenv VARIABLE_NAME='variable value'

You can set multiple environmental variables with one command by separating them with spaces. This is the recommended approach as it results in only a single update to the EB environment.

You can then access these variables in your Python environment via os.environ.

For example:

VARIABLE_NAME = os.environ['VARIABLE_NAME']

Environment Variables via EB Console

Enter the Elastic Beanstalk console via eb open. Navigate to "Configuration" > "Software" > "Edit". Then, scroll down to the "Environment properties".

After you're done, click "Apply" and your environment will update.

Again, you can access the environmental variables in Python via os.environ.

Debugging Elastic Beanstalk

When working with Elastic Beanstalk, it can be pretty frustrating to figure out what went wrong if you don't know how to access the log files. In this section we will look at just that.

There are two ways to access the logs:

1. Elastic Beanstalk CLI or console
2. SSH into EC2 instance

From personal experience, I've been able to solve all issues with the first approach.

Elastic Beanstalk CLI or Console

CLI:

$ eb logs

This command will fetch the last 100 lines from the following files:

/var/log/web.stdout.log
/var/log/eb-hooks.log
/var/log/nginx/access.log
/var/log/nginx/error.log
/var/log/eb-engine.log

Running eb logs is equivalent to logging into the EB console and navigating to "Logs".

I recommend piping the logs to CloudWatch. Run the following command to enable this:

$ eb logs --cloudwatch-logs enable

You'll typically find FastAPI errors in /var/log/web.stdout.log or /var/log/eb-engine.log.

To learn more about Elastic Beanstalk logs check out Viewing logs from Amazon EC2 instances.

SSH into EC2 Instance

To connect to an EC2 instance where your FastAPI application is running, run:

$ eb ssh

You'll be prompted to add the host to your known hosts the first time. Say yes. With that, you'll now have full access to your EC2 instance. Feel free to check some of the log files mentioned in the previous section.

Keep in mind that Elastic Beanstalk automatically scales and deploys new EC2 instances. The changes you make on this specific EC2 instance won't be reflected on newly launched EC2 instances. Once this specific EC2 instance is replaced, your changes will be wiped.

Conclusion

In this tutorial, we walked through the process of deploying a FastAPI application to AWS Elastic Beanstalk. By now you should have a fair understanding of how Elastic Beanstalk works. Perform a quick self-check by reviewing the objectives at the beginning of the tutorial.

Next steps:

1. You should consider creating two separate EB environments (dev and production).
2. Review Auto Scaling group for your Elastic Beanstalk environment to learn about how to configure triggers for auto scaling your application.

To remove all the AWS resources we created throughout the tutorial, first terminate the Elastic Beanstalk environment:

$ eb terminate

You'll need to manually remove the SSL certificate.

Finally, you can find the final version of the code in the fastapi-elastic-beanstalk repo on GitHub.

Original article source at: https://testdriven.io/blog/

#fastapi #elastic #aws