How do I Upload File To AWS S3 Using AWS CLI?

AWS Certified Solution Architect Associate

AWS Certified Solution Architect Associate

This course will help you in the final preparation steps for your AWS Certified Solution Architect Associate - Certification Exam.

In this course , we will go through the different concepts that get asked in the exam and map them to the different domain objectives for the exam.

This is good revision guide before you attempt the certification exam

Thanks for reading

If you liked this post, share it with all of your programming buddies!

Follow us on Facebook | Twitter

Further reading about AWS

A Complete Guide on Deploying a Node app to AWS with Docker

AWS Certified Solutions Architect - Associate 2019

AWS Lambda vs. Azure Functions vs. Google Functions

AWS Certified Developer - Associate 2019

Create and Deploy AWS and AWS Lambda using Serverless Framework

Introduction To AWS Lambda

Why Azure is Better Than AWS

Top 5 Common Node.js mistakes in Lambda

Top 5 Common Node.js mistakes in Lambda

Top 5 common mistakes to avoid when working with Node.js in Lambda

I have helped quite a few clients with their Node.js serverless projects. In doing so I have seen some recurring mistakes around async/await.

1. Still using callbacks

Many people are still using the callbacks in their async handler functions:

module.exports.handler = async (event, context, cb) => {
  const response = {
    statusCode: 200,
    body: JSON.stringify({ message: 'hello world' })
  }

  cb(null, response)
}

instead of the simpler alternative:

module.exports.handler = async (event, context) => {
  const response = {
    statusCode: 200,
    body: JSON.stringify({ message: 'hello world' })
  }

  return response
}

2. Not using promisify

Before Node8, bluebird filled a massive gap. It provided the utility to convert callback-based functions to promise-based. But Node8's built-in util module has filled that gap with the promisify function.

For example, we can now transform the readFile function from the fs module like this:

const fs = require('fs')
const { promisify } = require('util')
const readFile = promisify(fs.readFile)

No need to use bluebird anymore. That's one less dependency, which helps reduce the cold start time for our functions.

3. Too sequential

async/await lets you write asynchronous code as if they're synchronous, which is awesome. No more dealing with callback hell!

On the flip side, we can also miss a trick and not perform tasks concurrently where appropriate.

Take the following code as an example:

async function getFixturesAndTeam(teamId) {
  const fixtures = await fixtureModel.fetchAll()
  const team = await teamModel.fetch(teamId)
  return {
    team,
    fixtures: fixtures.filter(x => x.teamId === teamId)
  }
}

This function is easy to follow, but it's hardly optimal. teamModel.fetch doesn't depend on the result of fixtureModel.fetchAll, so they should run concurrently.

Here is how you can improve it:

async function getFixturesAndTeam(teamId) {
  const fixturesPromise = fixtureModel.fetchAll()
  const teamPromise = teamModel.fetch(teamId)

  const fixtures = await fixturesPromise
  const team = await teamPromise

  return {
    team,
    fixtures: fixtures.filter(x => x.teamId === teamId)
  }
}

In this version, both fixtureModel.fetchAll and teamModel.fetch are started concurrently.

You also need to watch out when using map with async/await. The following will call teamModel.fetch one after another:

async function getTeams(teamIds) {
  const teams = _.map(teamIds, id => await teamModel.fetch(id))
  return teams
}

Instead, you should write it as the following:

async function getTeams(teamIds) {
  const promises = _.map(teamIds, id => teamModel.fetch(id))
  const teams = await Promise.all(promises)
  return teams
}

In this version, we map teamIds to an array of Promise. We can then use Promise.all to turn this array into a single Promise that returns an array of teams.

In this case, teamModel.fetch is called concurrently and can significantly improve execution time.

4. async/await inside forEach()

This is a tricky one, and can sometimes catch out even experienced Node.js developers.

The problem is that code like this doesn't behave the way you'd expect it to:

[ 1, 2, 3 ].forEach(async (x) => {
  await sleep(x)
  console.log(x)
})

console.log('all done.')

When you run this you'll get the following output:

all done.

See this post for a longer explanation about why this doesn't work. For now, just remember to avoid using async/await inside a forEach!

5. Not using AWSSDK’s .promise()

Did you know that the AWS SDK clients support both callbacks and promises? To use async/await with the AWS SDK, add .promise() to client methods like this:

const AWS = require('aws-sdk')
const Lambda = new AWS.Lambda()

async function invokeLambda(functionName) {
  const req = {
    FunctionName: functionName,
    Payload: JSON.stringify({ message: 'hello world' })
  }
  await Lambda.invoke(req).promise()
}

No more callback functions, yay!

Building Bliss- Serverless Fullstack React with Prisma 2 and GraphQL

Building Bliss- Serverless Fullstack React with Prisma 2 and GraphQL

In this post, we will show how you can deploy a totally serverless stack using Prisma 2 and Next.js.

This type of solution has only been recently available and while it is still in beta, it really represents a full stack developer's paradise because you can develop an app, deploy it, forget about worrying about any of the DevOps particulars and be confident that it will work regardless of load.

Benefits:
  • One command to deploy the entire stack (Now)
  • Infinitely scalable, pay for what you use (lambda functions)
  • No servers to maintain (lambda functions)
  • All the advantages of React (composability, reusability and strong community support)
  • Server-side rendering for SEO (Next.js)
  • Correctly rendered social media link shares in Facebook and Twitter (Next.js)
  • Easy to evolve api (GraphQL)
  • One Schema to maintain for the entire stack (Prisma 2)
  • Secure secret management (Now)
  • Easy to set up development environment with hot code reloading (Docker)
  • Strongly typed (GraphQL and Typescript) that is autogenerated when possible (graphql-gen)

Before you start, you should go ahead and set up an RDS instance and configured like our previous blog post.

Videos:

I. Install Dependencies

II. Add Environmental Parameters

III. Configure the Backend

IV. Configure the Now Service

V. Set up Now Secrets and Deploy!

We will pick up from the example from our multi-part blog series [1][2][3]. If you aren't interested in following along from the start, you can start by checking out the repo from the now-serverless-start tag:

git clone https://github.com/CaptainChemist/blog-prisma2
git fetch && git fetch --tags
git checkout now-serverless-start
I. Install and clean up dependencies

Upgrade to Next v9

In the frontend/package.json make sure that next has a version of "9.02" or greater. Previously we were using a canary version of 8.1.1 for typescript support, but since the post version 9 of next was released so we want to make sure we can take advantage of all the latest goodies.

Install webpack to the frontend

As a precaution, you should install webpack to the frontend folder. I've seen inconsistent behavior with now where if webpack is not installed, sometimes the deploy will fail saying that it needs webpack. When I read online it sounds like it shouldn't be required so this is likely a bug, but it can't hurt to add it:

npm install --save-dev webpack

Remove the main block from package.json and frontend/package.json

When we generated our package.json files, it auto-populated the main field. Since we are not using this feature and don't even have an index.js file in either folder, we should go ahead and remove them. In frontend/package.json go ahead and remove line 5. We didn't use it previously and it has the potential to confuse the now service.

"main": "index.js",

Also, do the same in the package.json in the root folder.

Install Prisma2 to the backend

Although we globally install prisma2 in our docker containers, we need to now add it to our backend package.json file so that when we use the now service it will be available during the build step up in AWS. Navigate to the backend folder and install prisma2:

npm install --save-dev prisma2

Install Zeit Now

We should install now globally so that we will be able to run it from the command line:

npm install -g now
II. Add Environmental Variables

Add a .env file to the root of your project. Add the following variables which we will use across our docker environment.

MYSQL_URL=mysql://root:[email protected]:3306/prisma
BACKEND_URL=http://backend:4000/graphql
FRONTEND_URL=http://localhost:3000

Modify the docker-compose.yml file to inject these new variables into our docker containers. This is what the updated file looks like:

docker-compose.yml

version: '3.7'
services:
  mysql:
    container_name: mysql
    ports:
      - '3306:3306'
    image: mysql:5.7
    restart: always
    environment:
      MYSQL_DATABASE: prisma
      MYSQL_ROOT_PASSWORD: prisma
    volumes:
      - mysql:/var/lib/mysql
  prisma:
    links:
      - mysql
    depends_on:
      - mysql
    container_name: prisma
    ports:
      - '5555:5555'
    build:
      context: backend/prisma
      dockerfile: Dockerfile
    environment:
      MYSQL_URL: ${MYSQL_URL}
    volumes:
      - /app/prisma
  backend:
    links:
      - mysql
    depends_on:
      - mysql
      - prisma
    container_name: backend
    ports:
      - '4000:4000'
    build:
      context: backend
      dockerfile: Dockerfile
      args:
        - MYSQL_URL=${MYSQL_URL}
    environment:
      MYSQL_URL: ${MYSQL_URL}
      FRONTEND_URL: ${FRONTEND_URL}
    volumes:
      - ./backend:/app
      - /app/node_modules
      - /app/prisma
  frontend:
    container_name: frontend
    ports:
      - '3000:3000'
    build:
      context: frontend
      dockerfile: Dockerfile
    environment:
      BACKEND_URL: ${BACKEND_URL}
    volumes:
      - ./frontend:/app
      - /app/node_modules
      - /app/.next

volumes: #define our mysql volume used above
mysql:

Let's take a look at the parts that were changed, below are the parts snipped out that we added to the above file:

prisma:
environment:
MYSQL_URL: ${MYSQL_URL}

..more lines

backend:
build:
context: backend
dockerfile: Dockerfile
args:
- MYSQL_URL=${MYSQL_URL}
environment:
MYSQL_URL: ${MYSQL_URL}
FRONTEND_URL: ${FRONTEND_URL}

..more lines

frontend:
environment:
BACKEND_URL: ${BACKEND_URL}

We added environment blocks to the prisma studio, backend, and frontend containers. Since we have the .env file, any variables that we define in the .env file, such as VAR1=my-variable, we can call it in the yml as ${VAR1} and that will be like we used the my-variable string directly in that spot of the yml file.

Dynamically set Backend url on the frontend

We need to set the uri that the frontend connects to dynamically instead of hardcoding it. In the frontend/utils/init-apollo.js we previously had this line which would connect to localhost if the request came from a user or from the backend if it came from the next.js server:

uri: isBrowser ? 'http://localhost:4000' : 'http://backend:4000', // Server URL (must be absolute)

We need to still keep track of whether we are in the browser or server in the docker environment. In addition, though, we need to check whether we are in a docker environment or whether we are deployed via now into a lambda function.

We can access environment variables by using the process.env.ENVIRONMENTAL_VARIABLE. We check if the url matches our local environment url and if so, we know that we are in a docker environment. Now our logic is that if we are in a docker environment and the browser is making the request, we return the localhost, otherwise we pass the BACKEND_URL as the uri.

frontend/utils/init-apollo.js

function create(initialState) {
// Check out https://github.com/zeit/next.js/pull/4611 if you want to use the AWSAppSyncClient
const isBrowser = typeof window !== 'undefined'
const isDocker = process.env.BACKEND_URL === 'http://backend:4000/graphql'
return new ApolloClient({
connectToDevTools: isBrowser,
ssrMode: !isBrowser, // Disables forceFetch on the server (so queries are only run once)
link: new HttpLink({
uri:
isDocker && isBrowser
? 'http://localhost:4000/graphql'
: process.env.BACKEND_URL,
credentials: 'same-origin', // Additional fetch() options like credentials or headers
// Use fetch() polyfill on the server
fetch: !isBrowser && fetch,
}),
cache: new InMemoryCache().restore(initialState || {}),
})
}

Now that should really be all that we need to do, but since Next.js is both rendered on the server and in the client, we won't have access to server environmental variables unless we take one more step. We need to expose the variable in our frontend/next.config.js file:

frontend/next.config.js

const withCSS = require('@zeit/next-css')

module.exports = withCSS({
target: 'serverless',
env: {
BACKEND_URL: process.env.BACKEND_URL,
},
})

Note that due to how exactly Next.js handles process.env, you cannot destructure variables off of it. So the line below will not work, we need to use the entire process.env.BACKEND_URL variable.

const { BACKEND_URL } = process.env // NO!
III. Configure our backend server

Update the backend server to the /graphql backend and configure CORS

We updated the url above to the /graphql endpoint for the backend server. We are doing this because in now we will deploy our backend graphql server to ourdomain.com/graphql. We need to make this change in our backend/src/index.ts so that the server will be running at the /graphqlendpoint instead of /.

In addition, while we are here, we will disable subscriptions and enable CORS. CORS stands for cross origin resource sharing and it tells the backend server which frontend servers it should accept requests from. This ensures that if someone else stood up a frontend next server that pointed to our backend server that all requests would fail. We need this because you could imagine how damaging this could potentially be if someone bought a domain crazyamazondeals.com (I'm just making this up) and pointed their frontend server to the real backend server of amazon's shopping portal. This would allow a fake amazon frontend to gather all sorts of customer information while still sending real requests to amazon's actual backend server. Yikes!

In order to enable CORS we will pass in our frontend url. We will also enable credentials for future authentication-related purposes.

backend/src/index.ts

server.start(
{
endpoint: '/graphql',
playground: '/graphql',
subscriptions: false,
cors: {
credentials: true,
origin: process.env.FRONTEND_URL,
},
},
() => console.log(🚀 Server ready)
)

Update the backend/prisma/project.prisma file to use environmental variables and set our platform.

We can use the env("MYSQL_URL") which will take our MYSQL_URLenvironmental variable. Starting with prisma preview-3+ we need to specify which platforms that we plan to use with prisma2. We can use "native" for our docker work, but we need to use "linux-glibc-libssl1.0.2" for Zeit Now.

backend/prisma/project.prisma

datasource db {
provider = "mysql"
url = env("MYSQL_URL")
}

generator photon {
provider = "photonjs"
platforms = ["native", "linux-glibc-libssl1.0.2"]
}
// Rest of file

Update the backend/Dockerfile to pass the environmental variable into the prisma2 generate. We first have to define a docker argument using ARG named MYSQL_URL. Then, we take the MYSQL_URLenvironmental variable and assign it to this newly created ARG.

We need the MYSQL_URL environment variable so that our url from the prisma file gets evaluated properly.

backend/Dockerfile

FROM node:10.16.0
RUN npm install -g --unsafe-perm prisma2

RUN mkdir /app
WORKDIR /app

COPY package*.json ./
COPY prisma ./prisma/

ARG MYSQL_URL
ENV MYSQL_URL "$MYSQL_URL"

RUN npm install
RUN prisma2 generate

CMD ["npm", "start" ]

Note that the only reason we have access to the $MYSQL_URL variable in this Dockerfile is due to an args block that we previously added to the docker-compose.yml file. Adding variables to the environment block of docker-compose is only accessible during the runtime of the containers, not the building step which is where we are at when the Dockerfile is being executed.

backend:
build:
context: backend
dockerfile: Dockerfile
args:
- MYSQL_URL=${MYSQL_URL}
IV. Add our Now Configuration

Create now secrets

Locally, we have been using the .env file to store our secrets. Although we commit that file to our repo, the only reason why we can do that is because there are no sensitive environmental variables there. Ensure that if you ever add real secrets to that file, such as a stripe key, you need to never commit that to github or else you risk them being compromised!

For production, we need a more secure way to store secrets. Now provides a nice way to do this:

now secret add my_secret my_value

Now will encrypt and store these secrets on their servers and when we upload our app we can use them but we won't be able to read them out even if we try to be sneaky and read it out using console.logs. We need to create variables for the following variables that were in our .env file:

MYSQL_URL=mysql://user:[email protected]:3306/prisma
BACKEND_URL=https://your-now-url.sh/graphql
FRONTEND_URL=https://your-now-url

Note that by default your-now-url will be yourProjecFoldername.yourNowUsername.now.sh but you can always skip this step for now, get to Step V of this tutorial, deploy your site and then look at where it deploys to because it will be the last line of the console output. Then you come back to this step and add the now secrets and redeploy the site.

Add a now.json file to the root directory

We need to create a now.json file which will dictate details about how we should deploy our site. The first part of it has environmental variables for both the build and the runtime. We will be using secrets that we created in the previous step by using the @our-secret-name. If you forget what names you used, you can always type now secrets ls and you will get the names of the secrets (but critically not the secrets themselves).

Next we have to define our build steps. In our case we have to build both our nextjs application and our graphql-yoga server. The nextjs is built using a specially designed @now/next builder and we can just point it to our next.config.js file which is in our frontend folder. Our other build will use the index.ts file in our backend/src directory and the builder is smart enough to compile the code down into javascript and deploy it to a lambda function.

Finally, we have to define our routes. The backend server will end up at the /graphql endpoint while the frontend directory will use everything else. This ensures that any page we go to under ourdomain.com will be forwarded onto the nextjs server except the /graphql endpoint.

now.json

{
"version": 2,
"build": {
"env": {
"MYSQL_URL": "@mysql_url",
"BACKEND_URL": "@backend_url",
"FRONTEND_URL": "@frontend_url"
}
},
"env": {
"MYSQL_URL": "@mysql_url",
"BACKEND_URL": "@backend_url",
"FRONTEND_URL": "@frontend_url"
},
"builds": [
{
"src": "frontend/next.config.js",
"use": "@now/next"
},
{
"src": "backend/src/index.ts",
"use": "@now/node",
"config": { "maxLambdaSize": "20mb" }
}
],
"routes": [
{ "src": "/graphql", "dest": "/backend/src/index.ts" },
{
"src": "/(.*)",
"dest": "/frontend/$1",
"headers": {
"x-request-path": "$1"
}
}
]
}

Add a .nowignore file to the root directory

Finally, we can add our ignore file which will tell now which things it shouldn't bother to upload.

.nowignore

**/node_modules
.next
Dockerfile
README.MD
V. Deploy our now full stack site

This part is easy. Simply type now from the root folder and let it fly!

Thanks for reading. If you liked this post, share it with all of your programming buddies!

Further reading

☞ The Complete JavaScript Course 2019: Build Real Projects!

☞  Show Suggestions on Typing using Javascript

☞ JavaScript Bootcamp - Build Real World Applications

☞ The Web Developer Bootcamp

☞ JavaScript Programming Tutorial - Full JavaScript Course for Beginners

☞ Best JavaScript Frameworks, Libraries and Tools to Use in 2019

☞ What JavaScript Framework You Should Learn to Get a Job in 2019?

☞ Best JavaScript Frameworks, Libraries and Tools to Use in 2019

☞ Microfrontends — Connecting JavaScript frameworks together (React, Angular, Vue etc)

☞ Do we still need JavaScript frameworks?


Originally published on https://www.codemochi.com

Running TensorFlow on AWS Lambda using Serverless

Running TensorFlow on AWS Lambda using Serverless

For smaller workloads, serverless platforms such as AWS Lambda can be a fast and low-cost option for deploying machine learning models. As the application grows, pieces can then be moved to dedicated servers, or PaaS options such as AWS Sagemaker, if necessary.

For smaller workloads, serverless platforms such as AWS Lambda can be a fast and low-cost option for deploying machine learning models. As the application grows, pieces can then be moved to dedicated servers, or PaaS options such as AWS Sagemaker, if necessary.

Although it is instructive to first use Lambda by uploading code directly, it is best to select a framework so that you can leverage additional integrations, such as API Gateway with AWS. For this example we will use TensorFlow as the machine learning library, and so we will look for frameworks that can deploy Python applications.

Zappa is well known for being able to easily deploy existing Flask or Django apps, however since we are creating this with serverless in mind from the start we will select the ubiqutous and powerful Serverless framework.

When treating infrastructure configuration as a first-class citizen it is advisable to first create a shell of the application and deploy it, and thenwrite the actual code. This allows for rapid iterations that are close to the end-state, and avoids costly surprises down the road.

Structuring the project

For machine learning most of the work can be categorized into three critical steps:

  • Retrieving, cleaning, and uploading the input data
  • Training the model and saving the results
  • Inferring (i.e. predicting) a new result based on a new set of data

At its core, designing for serverless platforms means thinking of how to segment your code by individual deployable functions. In light of the categories above, we will structure our project like so:

├── TfLambdaDemo
│   ├── upload.py
│   ├── train.py
│   └── infer.py

Be sure to also create a new virtualenv:

$ pyenv virtualenv 3.6.5 tflambdademo
$ pyenv activate tflambdademo

Adding Lambda handlers

A “handler” is the term used for the function that will actually be invoked by Lambda, and is always called with two parameters, event and context. From the docs:

<em>event</em>* – AWS Lambda uses this parameter to pass in event data to the handler. This parameter is usually of the Python <em>dict</em> type. It can also be <em>list</em>, <em>str</em>, <em>int</em>, <em>float</em>, or <em>NoneType</em> type.> <em>context</em> – AWS Lambda uses this parameter to provide runtime information to your handler. This parameter is of the *<em>LambdaContext</em>type.
If you were invoking the functions directly, event would be of a type made in that call. However, we will plan to invoke using an HTTP POST request through API Gateway, which means the data will be contained in event['body'] and we will need to return a compatible response.

To get started, add boilerplate functions into each of the .py files mentioned above:

import json
def uploadHandler(event, context):
    body = json.loads(event.get('body'))
    response = {
        "statusCode": 200,
        "body": json.dumps(body)
    }
    return response

Installing and configuring Serverless

If you are not familiar with Serverless, the first thing to note is that it is actually based on Node.js. This may seem odd since we are building a Python app, but it makes sense once you realize that this framework is really a developer CLI tool and not something that ships with your product.

On a Mac you can install via Homebrew:

$ brew update
$ brew install node

Verify that you have node installed, as well as the package manager npm:

$ node --version
$ npm --version

Install Serverless as a global package (-g) and verify that the serverlesscommand is now available on your CLI:

$ npm install -g serverless
$ serverless

Create a new Serverless project in the TfLambdaDemo directory:

$ serverless create --template aws-python

Notice the new file serverless.yml and how your .gitignore file was auto-populated. Serverless also created handler.py template file, but you can delete this.

When you open serverless.yml you will see a lot of boilerplate information, which is good to familiarize yourself with. First update the service name totflambdademo, and then update theprovider section to AWS running Python 3.6 in the region of your choice. Defining the stage is useful when managing production deployments, but for now we will leave it as dev.

In the functions section list out upload, train, and infer with a handler format of <filename>.<function>. The events section contains the information for how the function will be called. Since we plan to use API Gateway, we will add the http trigger, and set the timeouts to 30 seconds to match.

In AWS Lambda the allocated memory can be configured, and then CPU is scaled accordingly. Since the machine learning training operation will be computationally intensive change from the default of 1024 MB to the maximum of 3008 MB (we can optimize later).

Your serverless.yml file should now look like:

service: tflambdademo
provider:
  name: aws
  region: us-east-1
  runtime: python3.6
  stage: dev
functions:
  upload:
    handler: upload.uploadHandler
    timeout: 30
    events:
      - http:
          path: upload
          method: post
  train:
    handler: train.trainHandler
    timeout: 30
    memory: 3008
    events:
      - http:
          path: train
          method: post
          async: true
  infer:
    handler: infer.inferHandler
    timeout: 30
    events:
      - http:
          path: infer
          method: post

Since we already added boilerplate functionality into the handlers, we can deploy with the following command:

Note: To deploy you will need an AWS account and your credentials properly configured. For details see the docs.

$ serverless deploy -v

At the end you should see the following information, where <id> will be custom to your deployment:

Service Information
service: tflambdademo
stage: dev
region: us-east-1
stack: tflambdademo-dev
resources: 22
api keys:
  None
endpoints:
  POST - https://<id>.execute-api.us-east-1.amazonaws.com/dev/upload
  POST - https://<id>.execute-api.us-east-1.amazonaws.com/dev/train
  POST - https://<id>.execute-api.us-east-1.amazonaws.com/dev/infer
functions:
  upload: tflambdademo-dev-upload
  train: tflambdademo-dev-train
  infer: tflambdademo-dev-infer
layers:
  None

What just happened? Serverless took care of all the heavy lifting by first creating a new S3 bucket and uploading your code, and then creating a CloudFormation template that executed the following:

  • Create the Lambda functions
  • Create API gateway with the defined endpoints configured to integrated with the handlers
  • Create a new IAM role and the proper permissions
  • Create a new log group viewable in CloudWatch

Test out the new endpoint by verifying that a request body is sent back (remember to replace <id>):

$ curl -X POST https://<id>.execute-api.us-east-1.amazonaws.com/dev/infer -d '{"foo": "bar"}'
{"foo": "bar"}

Magic!

Adding in TensorFlow

Before doing anything else, let’s see if we can successfully add TensorFlow to our project. To each of the .py files add import TensorFlow as tf. Then install via pip and re-deploy.

$ pip install tensorflow
$ pip freeze > requirements.txt
$ serverless deploy -v

Everything looks fine, but when we try to test the endpoint we get an error:

$ curl -X POST https://<id>.execute-api.us-east-1.amazonaws.com/dev/infer -d '{"foo": "bar"}'
{"message": "Internal server error"}

If we go to CloudWatch we can see the following error:

Unable to import module 'infer': No module named 'tensorflow'

This seems surprising since invoking the function locally is successful:

$ serverless invoke local --function infer
{
    "statusCode": 200,
    "body": "null"
}

The reason is that even though we installed TensorFlow to our virtual environment, there was no mechanism to add these libraries to our Lambda package. The only content in that package was our raw code, which up until now depended only on the pre-bundled library json.

Adding the serverless-python-requirements plugin

One great thing about Serverless is the extensibility via plugins. In order to bundle dependencies from our requirements.txt file we will use serverless-python-requirements.

Installing this plugin will add a package.json file, thenode_modules directory (be sure to add this to your gitignore!), and a plugins section to your serverless.yml file.

$ serverless plugin install -n serverless-python-requirements

To give us the most flexibility we will use the Dockerize option. This avoids some complexity with using the binaries from our virtual environment, and also allows for compiling non-pure-Python libraries. To select this option, odd the following section to your serverless.yml file:

Note: You will need Docker installed on your machine to continue.

custom:
  pythonRequirements:
    dockerizePip: true

If we now run serverless deploy -v we can see additional upfront actions to create the Docker image. However, it still fails!

An error occurred: UploadLambdaFunction - Unzipped size must be smaller than 262144000 bytes (Service: AWSLambdaInternal; Status Code: 400; Error Code: InvalidParameterValueException; Request ID: c3b94dc7-6a06-11e9-8823-bb373647997a).

Our zipped payload for Lambda balloons from 6.5KB to 126.9MB, but more importantly the unzipped size is 509MB which is not even close to the 262MB limit! If we download the zip file from S3 we can see that 399MBs are coming from the tensorflow folder.

How do I fit all this stuff in that box?

To get everything down to size we will employ three techniques available in the serverless-python-requirements plugin:

  • zip — Compresses the libraries in an additional .requirements.zip file and addsunzip_requirements.py in the final bundle.
  • slim — Removes unneeded files and directories such as *.so, *.pyc, dist-info, etc.
  • noDeploy — Omits certain packages from deployment. We will use the standard list that includes those already built into Lambda, as well as Tensorboard.

The custom section in your serverless.yml file should now look like:

custom:
  pythonRequirements:
    dockerizePip: true
    zip: true
    slim: true
    noDeploy:
      - boto3
      - botocore
      - docutils
      - jmespath
      - pip
      - python-dateutil
      - s3transfer
      - setuptools
      - six
      - tensorboard

You will also need to add the following as the first four lines in the .py files. This step will unzip the requirements file on Lambda, but will be skipped when running locally since unzip_requirements.py only exists in the final bundle.

try:
  import unzip_requirements
except ImportError:
  pass

Running deploy will now succeed, and we can again test our endpoint to verify the function works.

$ serverless deploy -v
...
$ curl -X POST https://<id>.execute-api.us-east-1.amazonaws.com/dev/infer -d '{"foo": "bar"}'
{"foo": "bar"}

Inspecting the file on S3, we can see that our semi-unzipped packaged is now 103MB (under the 262MB limit), and the fully unzipped package with all of the libraries is 473MB (narrowly under the 500MB total local storage limit). Success!

It’s important to recognize that we haven’t actually written a real line of code related to machine learning yet. If the infrastructure configuration is critical, the above is a validation of why it is important to start with a deployable shell first. It will help inform what restrictions you may have as you start to build out the application (e.g. you will not be able to use another large library in combination with Tensorflow), or whether it is even possible.

Creating the Machine Learning functions

For this demonstration we will leverage a Linear Classifier example from TensorFlow, which uses the higher-level Estimator API:

Using census data which contains data a person’s age, education, marital status, and occupation (the features), we will try to predict whether or not the person earns more than 50,000 dollars a year (the target label). We will train a logistic regression model that, given an individual’s information, outputs a number between 0 and 1 — this can be interpreted as the probability that the individual has an annual income of over 50,000 dollars.
Specifically we will clone census_data.py from that project, which provides the functions for downloading and cleaning the data, as well as the input function.

upload.py

Since we will be using S3 to store our data, we need to add this resource into the serverless.yml file. First add an environment variable to define the bucket name and a new IAM role. Note that we can now refer to BUCKETinside this file as well as our application.

provider:
  name: aws
  region: us-east-1
  runtime: python3.6
  stage: dev
iamRoleStatements:
  - Effect: Allow
    Action:
      - s3:*
    Resource:
     Fn::Join:
       - ""
       - - "arn:aws:s3:::"
         - ${self:provider.environment.BUCKET}
         - "/*"
environment:
  BUCKET: tflambdademo

Next add a new resource section which will actually create the S3 bucket:

resources:
  Resources:
    SageBucket:
      Type: AWS::S3::Bucket
      Properties:
        BucketName: ${self:provider.environment.BUCKET}

Update the upload handler to download the new data to the local Lambda storage location /tmp/, and then upload to S3. We will use an epoch prefix to separate each data-model pair.

Since no data needs to be read in from the request body we will delete that line, and also modify the response to return the epoch.

try:
  import unzip_requirements
except ImportError:
  pass
import os
import json
import time
import boto3
import tensorflow as tf
import census_data
FILE_DIR = '/tmp/'
BUCKET = os.environ['BUCKET']
def uploadHandler(event, context):
    # Download data to local tmp directory
    census_data.download(FILE_DIR)
    # Upload files to S3
    epoch_now = str(int(time.time()))
    boto3.Session(
        ).resource('s3'
        ).Bucket(BUCKET
        ).Object(os.path.join(epoch_now,census_data.TRAINING_FILE)
        ).upload_file(FILE_DIR+census_data.TRAINING_FILE)
    boto3.Session(
        ).resource('s3'
        ).Bucket(BUCKET
        ).Object(os.path.join(epoch_now,census_data.EVAL_FILE)
        ).upload_file(FILE_DIR+census_data.EVAL_FILE)
    response = {
        "statusCode": 200,
        "body": json.dumps({'epoch': epoch_now})
    }
    return response

At this point we can re-deploy the functions and trigger the upload function:

$ serverless deploy -v
...
$ curl -X POST https://<id>.execute-api.us-east-1.amazonaws.com/dev/upload
{"epoch": "1556995767"}

If you navigate to the new S3 bucket you should see the CSV files adult.dataand adult.test under a folder prefix defined by the epoch in the response.

train.py

The train function will download the data from S3 based on an epoch passed in the POST body.

def trainHandler(event, context):
    time_start = time.time()
    body = json.loads(event.get('body'))
    # Read in epoch
    epoch_files = body['epoch']
    # Download files from S3
    boto3.Session(
        ).resource('s3'
        ).Bucket(BUCKET
        ).download_file(
            os.path.join(epoch_files,census_data.TRAINING_FILE),
            FILE_DIR+census_data.TRAINING_FILE)
    boto3.Session(
        ).resource('s3'
        ).Bucket(BUCKET
        ).download_file(
            os.path.join(epoch_files,census_data.EVAL_FILE),
            FILE_DIR+census_data.EVAL_FILE)

In order to setup the estimator a set of feature columns must be provided. These columns can be thought of as placeholders to tell the model how to handle raw inputs. The census_data.py module provides a function to create two sets of columns for a wide and deep model. For this simple example we will only use the wide columns.

To actually execute the training we must provide an input function to the newly configured estimator. The input function reads data (in our case from a CSV file), and converts it into a TensorFlow tensor. However, the estimator expects an input function that has no arguments, and therefore we will use a partial function to create a new callable.

def trainHandler(event, context):
    
    ...
    # Create feature columns
    wide_cols, deep_cols = census_data.build_model_columns()
    # Setup estimator
    classifier = tf.estimator.LinearClassifier(
                        feature_columns=wide_cols,
                        model_dir=FILE_DIR+'model_'+epoch_files+'/')
    # Create callable input function and execute train
    train_inpf = functools.partial(
                    census_data.input_fn,
                    FILE_DIR+census_data.TRAINING_FILE,
                    num_epochs=2, shuffle=True,
                    batch_size=64)
    classifier.train(train_inpf)

We will then repeat this with the test data we held back in order to evaluate the model, and print the results to the logs.

def trainHandler(event, context):
    ...
    # Create callable input function and execute evaluation
    test_inpf = functools.partial(
                    census_data.input_fn,
                    FILE_DIR+census_data.EVAL_FILE,
                    num_epochs=1, shuffle=False,
                    batch_size=64)
    result = classifier.evaluate(test_inpf)
    print('Evaluation result: %s' % result)

In order to save the model to re-use for creating predictions, we will zip the files up and save to the same S3 folder.

def trainHandler(event, context):
    ...
    # Zip up model files and store in s3
    with tarfile.open(FILE_DIR+'model.tar.gz', mode='w:gz') as arch:
        arch.add(FILE_DIR+'model_'+epoch_files+'/', recursive=True)
    boto3.Session(
        ).resource('s3'
        ).Bucket(BUCKET
        ).Object(os.path.join(epoch_files,'model.tar.gz')
        ).upload_file(FILE_DIR+'model.tar.gz')

Finally, we will prepare the result data for JSON serialization in the response, and also add in a runtime calculation.

def trainHandler(event, context):
    ...
    # Convert result from float32 for json serialization
    for key in result:
        result[key] = result[key].item()
    runtime = round(time.time()-time_start, 1)
    response = {
        "statusCode": 200,
        "body": json.dumps({'epoch': epoch_files,
                            'runtime': runtime,
                            'result': result})
    }
    return response

Assuming that the test run after updating upload.py was successful, you can now deploy and test the function with that epoch key.

$ serverless deploy -v
...
$ curl -X POST https://<id>.execute-api.us-east-1.amazonaws.com/dev/train -d '{"epoch": "1556995767"}'
{"epoch": "1556995767", "runtime": 11.6, "result": {"accuracy": 0.8363736867904663, "accuracy_baseline": 0.7637737393379211, "auc": 0.8843450546264648, "auc_precision_recall": 0.697192907333374, "average_loss": 0.35046106576919556, "label/mean": 0.23622627556324005, "loss": 22.37590789794922, "precision": 0.698722243309021, "prediction/mean": 0.23248426616191864, "recall": 0.5403016209602356, "global_step": 1018}}

83.6% accuracy…not too bad, and in line with the results from the TensorFlow official example! If you visit the S3 bucket you should also see a saved file model.tar.gz in the epoch folder.

infer.py

The first step to building out the inference function is to think about how the data is coming in. The raw data should look just like the CSV input file we used above, except now it will come in the POST body.

The input function in census_data.py was built to stream CSV data from disk, which is scaleable for larger volumes. In our application we would only expect to make a small number of predictions at once, which would have no problem fitting into a small memory footprint, so we can use an easy input methodology.

To infer.py add a new function that will take in a dictionary where the keys represent the same columns that were in the CSV file and map to lists of values. Each “column” will then be converted to a numpy array with its datatype specified according to census_data.py.

Being able to use numpy makes it easy to convert to tensors, and there is no cost to our package size since it is a dependency for TensorFlow already.

def _easy_input_function(data_dict, batch_size=64):
    """
    data_dict = {
        '<csv_col_1>': ['<first_pred_value>', '<second_pred_value>']
        '<csv_col_2>': ['<first_pred_value>', '<second_pred_value>']
        ...
    }
    """
    # Convert input data to numpy arrays
    for col in data_dict:
        col_ind = census_data._CSV_COLUMNS.index(col)
        dtype = type(census_data._CSV_COLUMN_DEFAULTS[col_ind][0])
        data_dict[col] = np.array(data_dict[col],
                                        dtype=dtype)
    labels = data_dict.pop('income_bracket')
    ds = tf.data.Dataset.from_tensor_slices((data_dict, labels))
    ds = ds.batch(batch_size)
    return ds

Back to the main handler function we will read in the prediction data and epoch identifier, and then download and extract the model file.

def inferHandler(event, context):
    body = json.loads(event.get('body'))
    # Read in prediction data as dictionary
    # Keys should match _CSV_COLUMNS, values should be lists
    predict_input = body['input']
    # Read in epoch
    epoch_files = body['epoch']
    # Download model from S3 and extract
    boto3.Session(
        ).resource('s3'
        ).Bucket(BUCKET
        ).download_file(
            os.path.join(epoch_files,'model.tar.gz'),
            FILE_DIR+'model.tar.gz')
    tarfile.open(FILE_DIR+'model.tar.gz', 'r').extractall(FILE_DIR)

As in train.py we need to setup the estimator, but now warm_start_from is specified which tells TensorFlow to load the previously run model. To setup the prediction we will use thepredict() method, and pass in the previously created input function with a lambda to make it callable.

The output of this method is an iterable, which we will convert to lists and store in a new result variable. Each item in the list will represent the result corresponding to the index of items in the lists from the input data.

def inferHandler(event, context):
    ...
    # Create feature columns
    wide_cols, deep_cols = census_data.build_model_columns()
    # Load model
    classifier = tf.estimator.LinearClassifier(
            feature_columns=wide_cols,
            model_dir=FILE_DIR+'tmp/model_'+epoch_files+'/',
            warm_start_from=FILE_DIR+'tmp/model_'+epoch_files+'/')
    # Setup prediction
    predict_iter = classifier.predict(
                        lambda:_easy_input_function(predict_input))
    # Iterate over prediction and convert to lists
    predictions = []
    for prediction in predict_iter:
        for key in prediction:
            prediction[key] = prediction[key].tolist()
        predictions.append(prediction)
    response = {
        "statusCode": 200,
        "body": json.dumps(predictions,
                            default=lambda x: x.decode('utf-8'))
    }
    return response

Building on invoking upload and train in the previous steps, we can pass in a row from the CSV file to test the function after re-deploying.

$ serverless deploy -v
...
$ curl -X POST https://<id>.execute-api.us-east-1.amazonaws.com/dev/infer -d '{"epoch": "1556995767", "input": {"age": ["34"], "workclass": ["Private"], "fnlwgt": ["357145"], "education": ["Bachelors"], "education_num": ["13"], "marital_status": ["Married-civ-spouse"], "occupation": ["Prof-specialty"], "relationship": ["Wife"], "race": ["White"], "gender": ["Female"], "capital_gain": ["0"], "capital_loss": ["0"], "hours_per_week": ["50"], "native_country": ["United-States"], "income_bracket": [">50K"]}}'
[{"logits": [1.088104009628296], "logistic": [0.7480245232582092], "probabilities": [0.25197547674179077, 0.7480245232582092], "class_ids": [1], "classes": ["1"]}]

Since this was taken from the original dataset we can see that the correct label was >50K which the model successfully predicts at 74.8% versus 25.2% for <=50K.

Feel free to now declare yourself a psychic reader!

Final thoughts

There are distinct boundaries to this type of deployment for TensorFlow, specifically around duration, and it is always good to check whether a serverless infrastructure is actually cost effective.

If serverless is the right choice, there are a few steps that you can take to help expand the duration boundaries.

Call the functions asynchronously…

Serverless allows you to add async: true into the function configuration which would give you access to the full 900 second limit on Lambda, rather than the 30 second limit through API Gateway. In this case the request will only invoke the function, and not actually wait for a response. The downside is that you will need to setup other mechanisms to determine which epoch key you should use to train or invoke the model.

…or don’t use HTTP to trigger the training function at all

For many use cases, you may really only need to provide API Gateway integration with the invoke function. One pattern that could be used for the train function is to configure Serverless to trigger Lambda in response to S3 events. For example, when a new epoch partition is created with CSV files, train.py is automatically invoked to update the model.

Warm your functions

When invoking the train function you may have noticed that the request length was much longer than the actual runtime. This is because when you invoke the function for the first time Lambda must load your code and setup the environment (i.e. cold start). There is a great Serverless plugin available that allows you to configure automatic warming in the serverless.yml file.

To view the final code, visit https://github.com/mikepm35/TfLambdaDemo