What are we going to do?

With this article I want to share my experience with AWS Lambda in combination with Serverless Framworkand AWS S3. At the end we have a Lambda function reading csv data located in a S3 bucket, convert them into JSON format and provide it via a public REST endpoint. It does not raise the claim to be the best solution rather than show you what my experience was and how I implemented it on my first try.

Step by step we are going to do the following:

1. Create S3 Bucket and upload csv file
2. Install Serverless Framework locally on your machine
3. Create a Serverless project (Node.js/Express.js)
4. Prepare and configure roles, permissions and policies in AWS
5. Deploy Lambda to AWS

I assume you already have some experience with AWS and you know the basics like creating a role. If not please have a look at the official AWS documentation.

Architecture

First let’s have a look at the overall architecture of what we are going to build.

Architecture

As you can see the architecture is actually really simple. The client sends a request to an endpoint provided by the AWS API gateway which forwards it to our Lambda function. Let’s start!

---

1. Create S3 Bucket

In this step we are going to create a S3 bucket which contains the .csv file from where we want to read the data. Because I assume you are a little bit familiar with AWS I think you know how to create a S3 bucket. You can do it either with the AWS console or via cli.

Name the bucket like you want (or can because it has to be unique as you know). I named my bucket ‘MediumBucketLambda’.

Ok now upload a .csv file to this bucket. I just downloaded a sample file here and renamed it to ‘salesrecords.csv’.

Don’t use the biggest one (100 records is enough). The bucket/file doesn’t have to be public available.

**Note: **If you want to you use your own .csv file and you just copy and paste the code later here ensure you have a .csv file separating the data with a comma not a semicolon.

2. Install Serverless Framework

Because we don’ t want to set up and configure the Lambda and API gateway manually and especially not separately we use the Serverless Framework which is doing all the work for us. Serverless Frameworkis an open source framework (Serverless Framework Pro costs a monthly fee). What it does it creates a CloudFormation Stack and deploys everything we need to make it run. We just need to specify a single .yaml file.

First of all we need to install the latest version via npm:

sudo npm i serverless -g

And now run:

serverless version 

**Note: **Instead of the command ‘severless’ you can also use the shorter command ‘sls’

You should see something like this:

Ok! Now you are ready to use the serverless cli!

Serverless needs to know to what AWS account you want to deploy your Lambda. We don’t use AWS cli profiles instead we set the IAM user of the AWS account where we want to deploy our Lambda to the serverless config credentials:

sls config credentials --provider aws --key KEY --secret SECRET

As you probably know you get the key and secret from the AWS IAM console.

Now every time you deploy something with Serverless this account and IAM user is going to be used.

Note: Please make sure your IAM user has the permissions to deploy the Lambda!

Of course if you would work with different AWS accounts and different users and roles where you need to deploy your Lambdas you would managed everything with AWS CLI profiles/credentials (/.aws/credentials and /.aws/config) rather than setting one default account.

#aws #lambda #serverless #s3-bucket #nodejs