Running TensorFlow on AWS Lambda using Serverless

For smaller workloads, serverless platforms such as AWS Lambda can be a fast and low-cost option for deploying machine learning models. As the application grows, pieces can then be moved to dedicated servers, or PaaS options such as AWS Sagemaker, if necessary.

Although it is instructive to first use Lambda by uploading code directly, it is best to select a framework so that you can leverage additional integrations, such as API Gateway with AWS. For this example we will use TensorFlow as the machine learning library, and so we will look for frameworks that can deploy Python applications.

Zappa is well known for being able to easily deploy existing Flask or Django apps, however since we are creating this with serverless in mind from the start we will select the ubiqutous and powerful Serverless framework.

When treating infrastructure configuration as a first-class citizen it is advisable to first create a shell of the application and deploy it, and then write the actual code. This allows for rapid iterations that are close to the end-state, and avoids costly surprises down the road.

Structuring the project

For machine learning most of the work can be categorized into three critical steps:

  • Retrieving, cleaning, and uploading the input data
  • Training the model and saving the results
  • Inferring (i.e. predicting) a new result based on a new set of data

At its core, designing for serverless platforms means thinking of how to segment your code by individual deployable functions. In light of the categories above, we will structure our project like so:

├── TfLambdaDemo
│   ├── upload.py
│   ├── train.py
│   └── infer.py

Be sure to also create a new virtualenv:

$ pyenv virtualenv 3.6.5 tflambdademo
$ pyenv activate tflambdademo

Adding Lambda handlers

A “handler” is the term used for the function that will actually be invoked by Lambda, and is always called with two parameters, event and context. From the docs:

event – AWS Lambda uses this parameter to pass in event data to the handler. This parameter is usually of the Python dict type. It can also be liststr, intfloat, or NoneType type.

context – AWS Lambda uses this parameter to provide runtime information to your handler. This parameter is of the LambdaContexttype.

If you were invoking the functions directly, event would be of a type made in that call. However, we will plan to invoke using an HTTP POST request through API Gateway, which means the data will be contained in event['body'] and we will need to return a compatible response.

To get started, add boilerplate functions into each of the .py files mentioned above:

import json

def uploadHandler(event, context):
    body = json.loads(event.get('body'))
    response = {
        "statusCode": 200,
        "body": json.dumps(body)
    }
    return response

Installing and configuring Serverless

If you are not familiar with Serverless, the first thing to note is that it is actually based on Node.js. This may seem odd since we are building a Python app, but it makes sense once you realize that this framework is really a developer CLI tool and not something that ships with your product.

On a Mac you can install via Homebrew:

$ brew update
$ brew install node

Verify that you have node installed, as well as the package manager npm:

$ node --version
$ npm --version

Install Serverless as a global package (-g) and verify that the serverless command is now available on your CLI:

$ npm install -g serverless
$ serverless

Create a new Serverless project in the TfLambdaDemo directory:

$ serverless create --template aws-python

Notice the new file serverless.yml and how your .gitignore file was auto-populated. Serverless also created handler.py template file, but you can delete this.

When you open serverless.yml you will see a lot of boilerplate information, which is good to familiarize yourself with. First update the service name totflambdademo, and then update theprovider section to AWS running Python 3.6 in the region of your choice. Defining the stage is useful when managing production deployments, but for now we will leave it as dev.

In the functions section list out uploadtrain, and infer with a handler format of <filename>.<function>. The events section contains the information for how the function will be called. Since we plan to use API Gateway, we will add the http trigger, and set the timeouts to 30 seconds to match.

In AWS Lambda the allocated memory can be configured, and then CPU is scaled accordingly. Since the machine learning training operation will be computationally intensive change from the default of 1024 MB to the maximum of 3008 MB (we can optimize later).

Your serverless.yml file should now look like:

service: tflambdademo

provider:
  name: aws
  region: us-east-1
  runtime: python3.6
  stage: dev
functions:
  upload:
    handler: upload.uploadHandler
    timeout: 30
    events:
      - http:
          path: upload
          method: post
  train:
    handler: train.trainHandler
    timeout: 30
    memory: 3008
    events:
      - http:
          path: train
          method: post
          async: true
  infer:
    handler: infer.inferHandler
    timeout: 30
    events:
      - http:
          path: infer
          method: post

Since we already added boilerplate functionality into the handlers, we can deploy with the following command:

Note: To deploy you will need an AWS account and your credentials properly configured. For details see the docs.

$ serverless deploy -v

At the end you should see the following information, where <id> will be custom to your deployment:

Service Information
service: tflambdademo
stage: dev
region: us-east-1
stack: tflambdademo-dev
resources: 22
api keys:
  None
endpoints:
  POST - https://<id>.execute-api.us-east-1.amazonaws.com/dev/upload
  POST - https://<id>.execute-api.us-east-1.amazonaws.com/dev/train
  POST - https://<id>.execute-api.us-east-1.amazonaws.com/dev/infer
functions:
  upload: tflambdademo-dev-upload
  train: tflambdademo-dev-train
  infer: tflambdademo-dev-infer
layers:
  None

What just happened? Serverless took care of all the heavy lifting by first creating a new S3 bucket and uploading your code, and then creating a CloudFormation template that executed the following:

  • Create the Lambda functions
  • Create API gateway with the defined endpoints configured to integrated with the handlers
  • Create a new IAM role and the proper permissions
  • Create a new log group viewable in CloudWatch

Test out the new endpoint by verifying that a request body is sent back (remember to replace <id>):

$ curl -X POST https://<id>.execute-api.us-east-1.amazonaws.com/dev/infer -d '{"foo": "bar"}'

{"foo": "bar"}

Magic!

Adding in TensorFlow

Before doing anything else, let’s see if we can successfully add TensorFlow to our project. To each of the .py files add import TensorFlow as tf. Then install via pip and re-deploy.

$ pip install tensorflow
$ pip freeze > requirements.txt
$ serverless deploy -v

Everything looks fine, but when we try to test the endpoint we get an error:

$ curl -X POST https://<id>.execute-api.us-east-1.amazonaws.com/dev/infer -d '{"foo": "bar"}'

{"message": "Internal server error"}

If we go to CloudWatch we can see the following error:

Unable to import module 'infer': No module named 'tensorflow'

This seems surprising since invoking the function locally is successful:

$ serverless invoke local --function infer
{
    "statusCode": 200,
    "body": "null"
}

The reason is that even though we installed TensorFlow to our virtual environment, there was no mechanism to add these libraries to our Lambda package. The only content in that package was our raw code, which up until now depended only on the pre-bundled library json.

Adding the serverless-python-requirements plugin

One great thing about Serverless is the extensibility via plugins. In order to bundle dependencies from our requirements.txt file we will use serverless-python-requirements.

Installing this plugin will add a package.json file, thenode_modules directory (be sure to add this to your gitignore!), and a plugins section to your serverless.yml file.

$ serverless plugin install -n serverless-python-requirements

To give us the most flexibility we will use the Dockerize option. This avoids some complexity with using the binaries from our virtual environment, and also allows for compiling non-pure-Python libraries. To select this option, odd the following section to your serverless.yml file:

Note: You will need Docker installed on your machine to continue.

custom:
  pythonRequirements:
    dockerizePip: true

If we now run serverless deploy -v we can see additional upfront actions to create the Docker image. However, it still fails!

An error occurred: UploadLambdaFunction - Unzipped size must be smaller than 262144000 bytes (Service: AWSLambdaInternal; Status Code: 400; Error Code: InvalidParameterValueException; Request ID: c3b94dc7-6a06-11e9-8823-bb373647997a).

Our zipped payload for Lambda balloons from 6.5KB to 126.9MB, but more importantly the unzipped size is 509MB which is not even close to the 262MB limit! If we download the zip file from S3 we can see that 399MBs are coming from the tensorflow folder.

How do I fit all this stuff in that box?

To get everything down to size we will employ three techniques available in the serverless-python-requirements plugin:

  • zip — Compresses the libraries in an additional .requirements.zip file and addsunzip_requirements.py in the final bundle.
  • slim — Removes unneeded files and directories such as *.so*.pycdist-info, etc.
  • noDeploy — Omits certain packages from deployment. We will use the standard list that includes those already built into Lambda, as well as Tensorboard.

The custom section in your serverless.yml file should now look like:

custom:
  pythonRequirements:
    dockerizePip: true
    zip: true
    slim: true
    noDeploy:
      - boto3
      - botocore
      - docutils
      - jmespath
      - pip
      - python-dateutil
      - s3transfer
      - setuptools
      - six
      - tensorboard

You will also need to add the following as the first four lines in the .py files. This step will unzip the requirements file on Lambda, but will be skipped when running locally since unzip_requirements.py only exists in the final bundle.

try:
  import unzip_requirements
except ImportError:
  pass

Running deploy will now succeed, and we can again test our endpoint to verify the function works.

$ serverless deploy -v
...

$ curl -X POST https://<id>.execute-api.us-east-1.amazonaws.com/dev/infer -d '{"foo": "bar"}'
{"foo": "bar"}

Inspecting the file on S3, we can see that our semi-unzipped packaged is now 103MB (under the 262MB limit), and the fully unzipped package with all of the libraries is 473MB (narrowly under the 500MB total local storage limit). Success!

It’s important to recognize that we haven’t actually written a real line of code related to machine learning yet. If the infrastructure configuration is critical, the above is a validation of why it is important to start with a deployable shell first. It will help inform what restrictions you may have as you start to build out the application (e.g. you will not be able to use another large library in combination with Tensorflow), or whether it is even possible.

Creating the Machine Learning functions

For this demonstration we will leverage a Linear Classifier example from TensorFlow, which uses the higher-level Estimator API:

Using census data which contains data a person’s age, education, marital status, and occupation (the features), we will try to predict whether or not the person earns more than 50,000 dollars a year (the target label). We will train a logistic regression model that, given an individual’s information, outputs a number between 0 and 1 — this can be interpreted as the probability that the individual has an annual income of over 50,000 dollars.

Specifically we will clone census_data.py from that project, which provides the functions for downloading and cleaning the data, as well as the input function.

upload.py

Since we will be using S3 to store our data, we need to add this resource into the serverless.yml file. First add an environment variable to define the bucket name and a new IAM role. Note that we can now refer to BUCKET inside this file as well as our application.

provider:
  name: aws
  region: us-east-1
  runtime: python3.6
  stage: dev
iamRoleStatements:
  - Effect: Allow
    Action:
      - s3:*
    Resource:
     Fn::Join:
       - ""
       - - "arn:aws:s3:::"
         - ${self:provider.environment.BUCKET}
         - "/*"
environment:
  BUCKET: tflambdademo

Next add a new resource section which will actually create the S3 bucket:

resources:
  Resources:
    SageBucket:
      Type: AWS::S3::Bucket
      Properties:
        BucketName: ${self:provider.environment.BUCKET}

Update the upload handler to download the new data to the local Lambda storage location /tmp/, and then upload to S3. We will use an epoch prefix to separate each data-model pair.

Since no data needs to be read in from the request body we will delete that line, and also modify the response to return the epoch.

try:
  import unzip_requirements
except ImportError:
  pass
import os
import json
import time
import boto3
import tensorflow as tf
import census_data
FILE_DIR = '/tmp/'
BUCKET = os.environ['BUCKET']
def uploadHandler(event, context):
    ## Download data to local tmp directory
    census_data.download(FILE_DIR)
    ## Upload files to S3
    epoch_now = str(int(time.time()))
    boto3.Session(
        ).resource('s3'
        ).Bucket(BUCKET
        ).Object(os.path.join(epoch_now,census_data.TRAINING_FILE)
        ).upload_file(FILE_DIR+census_data.TRAINING_FILE)
    boto3.Session(
        ).resource('s3'
        ).Bucket(BUCKET
        ).Object(os.path.join(epoch_now,census_data.EVAL_FILE)
        ).upload_file(FILE_DIR+census_data.EVAL_FILE)
    response = {
        "statusCode": 200,
        "body": json.dumps({'epoch': epoch_now})
    }
    return response

At this point we can re-deploy the functions and trigger the upload function:

$ serverless deploy -v
...

$ curl -X POST https://<id>.execute-api.us-east-1.amazonaws.com/dev/upload
{"epoch": "1556995767"}

If you navigate to the new S3 bucket you should see the CSV files adult.data and adult.test under a folder prefix defined by the epoch in the response.

train.py

The train function will download the data from S3 based on an epoch passed in the POST body.

def trainHandler(event, context):
    time_start = time.time()

    body = json.loads(event.get('body'))
    ## Read in epoch
    epoch_files = body['epoch']
    ## Download files from S3
    boto3.Session(
        ).resource('s3'
        ).Bucket(BUCKET
        ).download_file(
            os.path.join(epoch_files,census_data.TRAINING_FILE),
            FILE_DIR+census_data.TRAINING_FILE)
    boto3.Session(
        ).resource('s3'
        ).Bucket(BUCKET
        ).download_file(
            os.path.join(epoch_files,census_data.EVAL_FILE),
            FILE_DIR+census_data.EVAL_FILE)

In order to setup the estimator a set of feature columns must be provided. These columns can be thought of as placeholders to tell the model how to handle raw inputs. The census_data.py module provides a function to create two sets of columns for a wide and deep model. For this simple example we will only use the wide columns.

To actually execute the training we must provide an input function to the newly configured estimator. The input function reads data (in our case from a CSV file), and converts it into a TensorFlow tensor. However, the estimator expects an input function that has no arguments, and therefore we will use a partial function to create a new callable.

def trainHandler(event, context):

    ...

    ## Create feature columns
    wide_cols, deep_cols = census_data.build_model_columns()
    ## Setup estimator
    classifier = tf.estimator.LinearClassifier(
                        feature_columns=wide_cols,
                        model_dir=FILE_DIR+'model_'+epoch_files+'/')
    ## Create callable input function and execute train
    train_inpf = functools.partial(
                    census_data.input_fn,
                    FILE_DIR+census_data.TRAINING_FILE,
                    num_epochs=2, shuffle=True,
                    batch_size=64)
    classifier.train(train_inpf)

We will then repeat this with the test data we held back in order to evaluate the model, and print the results to the logs.

def trainHandler(event, context):

    ...
    ## Create callable input function and execute evaluation
    test_inpf = functools.partial(
                    census_data.input_fn,
                    FILE_DIR+census_data.EVAL_FILE,
                    num_epochs=1, shuffle=False,
                    batch_size=64)
    result = classifier.evaluate(test_inpf)
    print('Evaluation result: %s' % result)

In order to save the model to re-use for creating predictions, we will zip the files up and save to the same S3 folder.

def trainHandler(event, context):

    ...
    ## Zip up model files and store in s3
    with tarfile.open(FILE_DIR+'model.tar.gz', mode='w:gz') as arch:
        arch.add(FILE_DIR+'model_'+epoch_files+'/', recursive=True)
    boto3.Session(
        ).resource('s3'
        ).Bucket(BUCKET
        ).Object(os.path.join(epoch_files,'model.tar.gz')
        ).upload_file(FILE_DIR+'model.tar.gz')

Finally, we will prepare the result data for JSON serialization in the response, and also add in a runtime calculation.

def trainHandler(event, context):

    ...
    ## Convert result from float32 for json serialization
    for key in result:
        result[key] = result[key].item()
    runtime = round(time.time()-time_start, 1)
    response = {
        "statusCode": 200,
        "body": json.dumps({'epoch': epoch_files,
                            'runtime': runtime,
                            'result': result})
    }
    return response

Assuming that the test run after updating upload.py was successful, you can now deploy and test the function with that epoch key.

$ serverless deploy -v
...

$ curl -X POST https://<id>.execute-api.us-east-1.amazonaws.com/dev/train -d '{"epoch": "1556995767"}'
{"epoch": "1556995767", "runtime": 11.6, "result": {"accuracy": 0.8363736867904663, "accuracy_baseline": 0.7637737393379211, "auc": 0.8843450546264648, "auc_precision_recall": 0.697192907333374, "average_loss": 0.35046106576919556, "label/mean": 0.23622627556324005, "loss": 22.37590789794922, "precision": 0.698722243309021, "prediction/mean": 0.23248426616191864, "recall": 0.5403016209602356, "global_step": 1018}}

83.6% accuracy…not too bad, and in line with the results from the TensorFlow official example! If you visit the S3 bucket you should also see a saved file model.tar.gz in the epoch folder.

infer.py

The first step to building out the inference function is to think about how the data is coming in. The raw data should look just like the CSV input file we used above, except now it will come in the POST body.

The input function in census_data.py was built to stream CSV data from disk, which is scaleable for larger volumes. In our application we would only expect to make a small number of predictions at once, which would have no problem fitting into a small memory footprint, so we can use an easy input methodology.

To infer.py add a new function that will take in a dictionary where the keys represent the same columns that were in the CSV file and map to lists of values. Each “column” will then be converted to a numpy array with its datatype specified according to census_data.py.

Being able to use numpy makes it easy to convert to tensors, and there is no cost to our package size since it is a dependency for TensorFlow already.

def _easy_input_function(data_dict, batch_size=64):
    """
    data_dict = {
        '<csv_col_1>': ['<first_pred_value>', '<second_pred_value>']
        '<csv_col_2>': ['<first_pred_value>', '<second_pred_value>']
        ...
    }
    """
    ## Convert input data to numpy arrays
    for col in data_dict:
        col_ind = census_data._CSV_COLUMNS.index(col)
        dtype = type(census_data._CSV_COLUMN_DEFAULTS[col_ind][0])
        data_dict[col] = np.array(data_dict[col],
                                        dtype=dtype)
    labels = data_dict.pop('income_bracket')
    ds = tf.data.Dataset.from_tensor_slices((data_dict, labels))
    ds = ds.batch(batch_size)
    return ds

Back to the main handler function we will read in the prediction data and epoch identifier, and then download and extract the model file.

def inferHandler(event, context):
    body = json.loads(event.get('body'))

    ## Read in prediction data as dictionary
    ## Keys should match _CSV_COLUMNS, values should be lists
    predict_input = body['input']
    ## Read in epoch
    epoch_files = body['epoch']
    ## Download model from S3 and extract
    boto3.Session(
        ).resource('s3'
        ).Bucket(BUCKET
        ).download_file(
            os.path.join(epoch_files,'model.tar.gz'),
            FILE_DIR+'model.tar.gz')
    tarfile.open(FILE_DIR+'model.tar.gz', 'r').extractall(FILE_DIR)

As in train.py we need to setup the estimator, but now warm_start_from is specified which tells TensorFlow to load the previously run model. To setup the prediction we will use thepredict() method, and pass in the previously created input function with a lambda to make it callable.

The output of this method is an iterable, which we will convert to lists and store in a new result variable. Each item in the list will represent the result corresponding to the index of items in the lists from the input data.

def inferHandler(event, context):
    ...

    ## Create feature columns
    wide_cols, deep_cols = census_data.build_model_columns()
    ## Load model
    classifier = tf.estimator.LinearClassifier(
            feature_columns=wide_cols,
            model_dir=FILE_DIR+'tmp/model_'+epoch_files+'/',
            warm_start_from=FILE_DIR+'tmp/model_'+epoch_files+'/')
    ## Setup prediction
    predict_iter = classifier.predict(
                        lambda:_easy_input_function(predict_input))
    ## Iterate over prediction and convert to lists
    predictions = []
    for prediction in predict_iter:
        for key in prediction:
            prediction[key] = prediction[key].tolist()
        predictions.append(prediction)
    response = {
        "statusCode": 200,
        "body": json.dumps(predictions,
                            default=lambda x: x.decode('utf-8'))
    }
    return response

Building on invoking upload and train in the previous steps, we can pass in a row from the CSV file to test the function after re-deploying.

$ serverless deploy -v
...

$ curl -X POST https://<id>.execute-api.us-east-1.amazonaws.com/dev/infer -d '{"epoch": "1556995767", "input": {"age": ["34"], "workclass": ["Private"], "fnlwgt": ["357145"], "education": ["Bachelors"], "education_num": ["13"], "marital_status": ["Married-civ-spouse"], "occupation": ["Prof-specialty"], "relationship": ["Wife"], "race": ["White"], "gender": ["Female"], "capital_gain": ["0"], "capital_loss": ["0"], "hours_per_week": ["50"], "native_country": ["United-States"], "income_bracket": [">50K"]}}'
[{"logits": [1.088104009628296], "logistic": [0.7480245232582092], "probabilities": [0.25197547674179077, 0.7480245232582092], "class_ids": [1], "classes": ["1"]}]

Since this was taken from the original dataset we can see that the correct label was >50K which the model successfully predicts at 74.8% versus 25.2% for <=50K.

Feel free to now declare yourself a psychic reader!

Final thoughts

There are distinct boundaries to this type of deployment for TensorFlow, specifically around duration, and it is always good to check whether a serverless infrastructure is actually cost effective.

If serverless is the right choice, there are a few steps that you can take to help expand the duration boundaries.

Call the functions asynchronously…

Serverless allows you to add async: true into the function configuration which would give you access to the full 900 second limit on Lambda, rather than the 30 second limit through API Gateway. In this case the request will only invoke the function, and not actually wait for a response. The downside is that you will need to setup other mechanisms to determine which epoch key you should use to train or invoke the model.

…or don’t use HTTP to trigger the training function at all

For many use cases, you may really only need to provide API Gateway integration with the invoke function. One pattern that could be used for the train function is to configure Serverless to trigger Lambda in response to S3 events. For example, when a new epoch partition is created with CSV files, train.py is automatically invoked to update the model.

Warm your functions

When invoking the train function you may have noticed that the request length was much longer than the actual runtime. This is because when you invoke the function for the first time Lambda must load your code and setup the environment (i.e. cold start). There is a great Serverless plugin available that allows you to configure automatic warming in the serverless.yml file.

To view the final code, visit https://github.com/mikepm35/TfLambdaDemo

#tensorflow #aws #serverless #python #machine-learning

What is GEEK

Buddha Community

Running TensorFlow on AWS Lambda using Serverless
Chloe  Butler

Chloe Butler

1667425440

Pdf2gerb: Perl Script Converts PDF Files to Gerber format

pdf2gerb

Perl script converts PDF files to Gerber format

Pdf2Gerb generates Gerber 274X photoplotting and Excellon drill files from PDFs of a PCB. Up to three PDFs are used: the top copper layer, the bottom copper layer (for 2-sided PCBs), and an optional silk screen layer. The PDFs can be created directly from any PDF drawing software, or a PDF print driver can be used to capture the Print output if the drawing software does not directly support output to PDF.

The general workflow is as follows:

  1. Design the PCB using your favorite CAD or drawing software.
  2. Print the top and bottom copper and top silk screen layers to a PDF file.
  3. Run Pdf2Gerb on the PDFs to create Gerber and Excellon files.
  4. Use a Gerber viewer to double-check the output against the original PCB design.
  5. Make adjustments as needed.
  6. Submit the files to a PCB manufacturer.

Please note that Pdf2Gerb does NOT perform DRC (Design Rule Checks), as these will vary according to individual PCB manufacturer conventions and capabilities. Also note that Pdf2Gerb is not perfect, so the output files must always be checked before submitting them. As of version 1.6, Pdf2Gerb supports most PCB elements, such as round and square pads, round holes, traces, SMD pads, ground planes, no-fill areas, and panelization. However, because it interprets the graphical output of a Print function, there are limitations in what it can recognize (or there may be bugs).

See docs/Pdf2Gerb.pdf for install/setup, config, usage, and other info.


pdf2gerb_cfg.pm

#Pdf2Gerb config settings:
#Put this file in same folder/directory as pdf2gerb.pl itself (global settings),
#or copy to another folder/directory with PDFs if you want PCB-specific settings.
#There is only one user of this file, so we don't need a custom package or namespace.
#NOTE: all constants defined in here will be added to main namespace.
#package pdf2gerb_cfg;

use strict; #trap undef vars (easier debug)
use warnings; #other useful info (easier debug)


##############################################################################################
#configurable settings:
#change values here instead of in main pfg2gerb.pl file

use constant WANT_COLORS => ($^O !~ m/Win/); #ANSI colors no worky on Windows? this must be set < first DebugPrint() call

#just a little warning; set realistic expectations:
#DebugPrint("${\(CYAN)}Pdf2Gerb.pl ${\(VERSION)}, $^O O/S\n${\(YELLOW)}${\(BOLD)}${\(ITALIC)}This is EXPERIMENTAL software.  \nGerber files MAY CONTAIN ERRORS.  Please CHECK them before fabrication!${\(RESET)}", 0); #if WANT_DEBUG

use constant METRIC => FALSE; #set to TRUE for metric units (only affect final numbers in output files, not internal arithmetic)
use constant APERTURE_LIMIT => 0; #34; #max #apertures to use; generate warnings if too many apertures are used (0 to not check)
use constant DRILL_FMT => '2.4'; #'2.3'; #'2.4' is the default for PCB fab; change to '2.3' for CNC

use constant WANT_DEBUG => 0; #10; #level of debug wanted; higher == more, lower == less, 0 == none
use constant GERBER_DEBUG => 0; #level of debug to include in Gerber file; DON'T USE FOR FABRICATION
use constant WANT_STREAMS => FALSE; #TRUE; #save decompressed streams to files (for debug)
use constant WANT_ALLINPUT => FALSE; #TRUE; #save entire input stream (for debug ONLY)

#DebugPrint(sprintf("${\(CYAN)}DEBUG: stdout %d, gerber %d, want streams? %d, all input? %d, O/S: $^O, Perl: $]${\(RESET)}\n", WANT_DEBUG, GERBER_DEBUG, WANT_STREAMS, WANT_ALLINPUT), 1);
#DebugPrint(sprintf("max int = %d, min int = %d\n", MAXINT, MININT), 1); 

#define standard trace and pad sizes to reduce scaling or PDF rendering errors:
#This avoids weird aperture settings and replaces them with more standardized values.
#(I'm not sure how photoplotters handle strange sizes).
#Fewer choices here gives more accurate mapping in the final Gerber files.
#units are in inches
use constant TOOL_SIZES => #add more as desired
(
#round or square pads (> 0) and drills (< 0):
    .010, -.001,  #tiny pads for SMD; dummy drill size (too small for practical use, but needed so StandardTool will use this entry)
    .031, -.014,  #used for vias
    .041, -.020,  #smallest non-filled plated hole
    .051, -.025,
    .056, -.029,  #useful for IC pins
    .070, -.033,
    .075, -.040,  #heavier leads
#    .090, -.043,  #NOTE: 600 dpi is not high enough resolution to reliably distinguish between .043" and .046", so choose 1 of the 2 here
    .100, -.046,
    .115, -.052,
    .130, -.061,
    .140, -.067,
    .150, -.079,
    .175, -.088,
    .190, -.093,
    .200, -.100,
    .220, -.110,
    .160, -.125,  #useful for mounting holes
#some additional pad sizes without holes (repeat a previous hole size if you just want the pad size):
    .090, -.040,  #want a .090 pad option, but use dummy hole size
    .065, -.040, #.065 x .065 rect pad
    .035, -.040, #.035 x .065 rect pad
#traces:
    .001,  #too thin for real traces; use only for board outlines
    .006,  #minimum real trace width; mainly used for text
    .008,  #mainly used for mid-sized text, not traces
    .010,  #minimum recommended trace width for low-current signals
    .012,
    .015,  #moderate low-voltage current
    .020,  #heavier trace for power, ground (even if a lighter one is adequate)
    .025,
    .030,  #heavy-current traces; be careful with these ones!
    .040,
    .050,
    .060,
    .080,
    .100,
    .120,
);
#Areas larger than the values below will be filled with parallel lines:
#This cuts down on the number of aperture sizes used.
#Set to 0 to always use an aperture or drill, regardless of size.
use constant { MAX_APERTURE => max((TOOL_SIZES)) + .004, MAX_DRILL => -min((TOOL_SIZES)) + .004 }; #max aperture and drill sizes (plus a little tolerance)
#DebugPrint(sprintf("using %d standard tool sizes: %s, max aper %.3f, max drill %.3f\n", scalar((TOOL_SIZES)), join(", ", (TOOL_SIZES)), MAX_APERTURE, MAX_DRILL), 1);

#NOTE: Compare the PDF to the original CAD file to check the accuracy of the PDF rendering and parsing!
#for example, the CAD software I used generated the following circles for holes:
#CAD hole size:   parsed PDF diameter:      error:
#  .014                .016                +.002
#  .020                .02267              +.00267
#  .025                .026                +.001
#  .029                .03167              +.00267
#  .033                .036                +.003
#  .040                .04267              +.00267
#This was usually ~ .002" - .003" too big compared to the hole as displayed in the CAD software.
#To compensate for PDF rendering errors (either during CAD Print function or PDF parsing logic), adjust the values below as needed.
#units are pixels; for example, a value of 2.4 at 600 dpi = .0004 inch, 2 at 600 dpi = .0033"
use constant
{
    HOLE_ADJUST => -0.004 * 600, #-2.6, #holes seemed to be slightly oversized (by .002" - .004"), so shrink them a little
    RNDPAD_ADJUST => -0.003 * 600, #-2, #-2.4, #round pads seemed to be slightly oversized, so shrink them a little
    SQRPAD_ADJUST => +0.001 * 600, #+.5, #square pads are sometimes too small by .00067, so bump them up a little
    RECTPAD_ADJUST => 0, #(pixels) rectangular pads seem to be okay? (not tested much)
    TRACE_ADJUST => 0, #(pixels) traces seemed to be okay?
    REDUCE_TOLERANCE => .001, #(inches) allow this much variation when reducing circles and rects
};

#Also, my CAD's Print function or the PDF print driver I used was a little off for circles, so define some additional adjustment values here:
#Values are added to X/Y coordinates; units are pixels; for example, a value of 1 at 600 dpi would be ~= .002 inch
use constant
{
    CIRCLE_ADJUST_MINX => 0,
    CIRCLE_ADJUST_MINY => -0.001 * 600, #-1, #circles were a little too high, so nudge them a little lower
    CIRCLE_ADJUST_MAXX => +0.001 * 600, #+1, #circles were a little too far to the left, so nudge them a little to the right
    CIRCLE_ADJUST_MAXY => 0,
    SUBST_CIRCLE_CLIPRECT => FALSE, #generate circle and substitute for clip rects (to compensate for the way some CAD software draws circles)
    WANT_CLIPRECT => TRUE, #FALSE, #AI doesn't need clip rect at all? should be on normally?
    RECT_COMPLETION => FALSE, #TRUE, #fill in 4th side of rect when 3 sides found
};

#allow .012 clearance around pads for solder mask:
#This value effectively adjusts pad sizes in the TOOL_SIZES list above (only for solder mask layers).
use constant SOLDER_MARGIN => +.012; #units are inches

#line join/cap styles:
use constant
{
    CAP_NONE => 0, #butt (none); line is exact length
    CAP_ROUND => 1, #round cap/join; line overhangs by a semi-circle at either end
    CAP_SQUARE => 2, #square cap/join; line overhangs by a half square on either end
    CAP_OVERRIDE => FALSE, #cap style overrides drawing logic
};
    
#number of elements in each shape type:
use constant
{
    RECT_SHAPELEN => 6, #x0, y0, x1, y1, count, "rect" (start, end corners)
    LINE_SHAPELEN => 6, #x0, y0, x1, y1, count, "line" (line seg)
    CURVE_SHAPELEN => 10, #xstart, ystart, x0, y0, x1, y1, xend, yend, count, "curve" (bezier 2 points)
    CIRCLE_SHAPELEN => 5, #x, y, 5, count, "circle" (center + radius)
};
#const my %SHAPELEN =
#Readonly my %SHAPELEN =>
our %SHAPELEN =
(
    rect => RECT_SHAPELEN,
    line => LINE_SHAPELEN,
    curve => CURVE_SHAPELEN,
    circle => CIRCLE_SHAPELEN,
);

#panelization:
#This will repeat the entire body the number of times indicated along the X or Y axes (files grow accordingly).
#Display elements that overhang PCB boundary can be squashed or left as-is (typically text or other silk screen markings).
#Set "overhangs" TRUE to allow overhangs, FALSE to truncate them.
#xpad and ypad allow margins to be added around outer edge of panelized PCB.
use constant PANELIZE => {'x' => 1, 'y' => 1, 'xpad' => 0, 'ypad' => 0, 'overhangs' => TRUE}; #number of times to repeat in X and Y directions

# Set this to 1 if you need TurboCAD support.
#$turboCAD = FALSE; #is this still needed as an option?

#CIRCAD pad generation uses an appropriate aperture, then moves it (stroke) "a little" - we use this to find pads and distinguish them from PCB holes. 
use constant PAD_STROKE => 0.3; #0.0005 * 600; #units are pixels
#convert very short traces to pads or holes:
use constant TRACE_MINLEN => .001; #units are inches
#use constant ALWAYS_XY => TRUE; #FALSE; #force XY even if X or Y doesn't change; NOTE: needs to be TRUE for all pads to show in FlatCAM and ViewPlot
use constant REMOVE_POLARITY => FALSE; #TRUE; #set to remove subtractive (negative) polarity; NOTE: must be FALSE for ground planes

#PDF uses "points", each point = 1/72 inch
#combined with a PDF scale factor of .12, this gives 600 dpi resolution (1/72 * .12 = 600 dpi)
use constant INCHES_PER_POINT => 1/72; #0.0138888889; #multiply point-size by this to get inches

# The precision used when computing a bezier curve. Higher numbers are more precise but slower (and generate larger files).
#$bezierPrecision = 100;
use constant BEZIER_PRECISION => 36; #100; #use const; reduced for faster rendering (mainly used for silk screen and thermal pads)

# Ground planes and silk screen or larger copper rectangles or circles are filled line-by-line using this resolution.
use constant FILL_WIDTH => .01; #fill at most 0.01 inch at a time

# The max number of characters to read into memory
use constant MAX_BYTES => 10 * M; #bumped up to 10 MB, use const

use constant DUP_DRILL1 => TRUE; #FALSE; #kludge: ViewPlot doesn't load drill files that are too small so duplicate first tool

my $runtime = time(); #Time::HiRes::gettimeofday(); #measure my execution time

print STDERR "Loaded config settings from '${\(__FILE__)}'.\n";
1; #last value must be truthful to indicate successful load


#############################################################################################
#junk/experiment:

#use Package::Constants;
#use Exporter qw(import); #https://perldoc.perl.org/Exporter.html

#my $caller = "pdf2gerb::";

#sub cfg
#{
#    my $proto = shift;
#    my $class = ref($proto) || $proto;
#    my $settings =
#    {
#        $WANT_DEBUG => 990, #10; #level of debug wanted; higher == more, lower == less, 0 == none
#    };
#    bless($settings, $class);
#    return $settings;
#}

#use constant HELLO => "hi there2"; #"main::HELLO" => "hi there";
#use constant GOODBYE => 14; #"main::GOODBYE" => 12;

#print STDERR "read cfg file\n";

#our @EXPORT_OK = Package::Constants->list(__PACKAGE__); #https://www.perlmonks.org/?node_id=1072691; NOTE: "_OK" skips short/common names

#print STDERR scalar(@EXPORT_OK) . " consts exported:\n";
#foreach(@EXPORT_OK) { print STDERR "$_\n"; }
#my $val = main::thing("xyz");
#print STDERR "caller gave me $val\n";
#foreach my $arg (@ARGV) { print STDERR "arg $arg\n"; }

Download Details:

Author: swannman
Source Code: https://github.com/swannman/pdf2gerb

License: GPL-3.0 license

#perl 

Hermann  Frami

Hermann Frami

1655426640

Serverless Plugin for Microservice Code Management and Deployment

Serverless M

Serverless M (or Serverless Modular) is a plugin for the serverless framework. This plugins helps you in managing multiple serverless projects with a single serverless.yml file. This plugin gives you a super charged CLI options that you can use to create new features, build them in a single file and deploy them all in parallel

splash.gif

Currently this plugin is tested for the below stack only

  • AWS
  • NodeJS λ
  • Rest API (You can use other events as well)

Prerequisites

Make sure you have the serverless CLI installed

# Install serverless globally
$ npm install serverless -g

Getting Started

To start the serverless modular project locally you can either start with es5 or es6 templates or add it as a plugin

ES6 Template install

# Step 1. Download the template
$ sls create --template-url https://github.com/aa2kb/serverless-modular/tree/master/template/modular-es6 --path myModularService

# Step 2. Change directory
$ cd myModularService

# Step 3. Create a package.json file
$ npm init

# Step 3. Install dependencies
$ npm i serverless-modular serverless-webpack webpack --save-dev

ES5 Template install

# Step 1. Download the template
$ sls create --template-url https://github.com/aa2kb/serverless-modular/tree/master/template/modular-es5 --path myModularService

# Step 2. Change directory
$ cd myModularService

# Step 3. Create a package.json file
$ npm init

# Step 3. Install dependencies
$ npm i serverless-modular --save-dev

If you dont want to use the templates above you can just add in your existing project

Adding it as plugin

plugins:
  - serverless-modular

Now you are all done to start building your serverless modular functions

API Reference

The serverless CLI can be accessed by

# Serverless Modular CLI
$ serverless modular

# shorthand
$ sls m

Serverless Modular CLI is based on 4 main commands

  • sls m init
  • sls m feature
  • sls m function
  • sls m build
  • sls m deploy

init command

sls m init

The serverless init command helps in creating a basic .gitignore that is useful for serverless modular.

The basic .gitignore for serverless modular looks like this

#node_modules
node_modules

#sm main functions
sm.functions.yml

#serverless file generated by build
src/**/serverless.yml

#main serverless directories generated for sls deploy
.serverless

#feature serverless directories generated sls deploy
src/**/.serverless

#serverless logs file generated for main sls deploy
.sm.log

#serverless logs file generated for feature sls deploy
src/**/.sm.log

#Webpack config copied in each feature
src/**/webpack.config.js

feature command

The feature command helps in building new features for your project

options (feature Command)

This command comes with three options

--name: Specify the name you want for your feature

--remove: set value to true if you want to remove the feature

--basePath: Specify the basepath you want for your feature, this base path should be unique for all features. helps in running offline with offline plugin and for API Gateway

optionsshortcutrequiredvaluesdefault value
--name-nstringN/A
--remove-rtrue, falsefalse
--basePath-pstringsame as name

Examples (feature Command)

Creating a basic feature

# Creating a jedi feature
$ sls m feature -n jedi

Creating a feature with different base path

# A feature with different base path
$ sls m feature -n jedi -p tatooine

Deleting a feature

# Anakin is going to delete the jedi feature
$ sls m feature -n jedi -r true

function command

The function command helps in adding new function to a feature

options (function Command)

This command comes with four options

--name: Specify the name you want for your function

--feature: Specify the name of the existing feature

--path: Specify the path for HTTP endpoint helps in running offline with offline plugin and for API Gateway

--method: Specify the path for HTTP method helps in running offline with offline plugin and for API Gateway

optionsshortcutrequiredvaluesdefault value
--name-nstringN/A
--feature-fstringN/A
--path-pstringsame as name
--method-mstring'GET'

Examples (function Command)

Creating a basic function

# Creating a cloak function for jedi feature
$ sls m function -n cloak -f jedi

Creating a basic function with different path and method

# Creating a cloak function for jedi feature with custom path and HTTP method
$ sls m function -n cloak -f jedi -p powers -m POST

build command

The build command helps in building the project for local or global scope

options (build Command)

This command comes with four options

--scope: Specify the scope of the build, use this with "--feature" tag

--feature: Specify the name of the existing feature you want to build

optionsshortcutrequiredvaluesdefault value
--scope-sstringlocal
--feature-fstringN/A

Saving build Config in serverless.yml

You can also save config in serverless.yml file

custom:
  smConfig:
    build:
      scope: local

Examples (build Command)

all feature build (local scope)

# Building all local features
$ sls m build

Single feature build (local scope)

# Building a single feature
$ sls m build -f jedi -s local

All features build global scope

# Building all features with global scope
$ sls m build -s global

deploy command

The deploy command helps in deploying serverless projects to AWS (it uses sls deploy command)

options (deploy Command)

This command comes with four options

--sm-parallel: Specify if you want to deploy parallel (will only run in parallel when doing multiple deployments)

--sm-scope: Specify if you want to deploy local features or global

--sm-features: Specify the local features you want to deploy (comma separated if multiple)

optionsshortcutrequiredvaluesdefault value
--sm-paralleltrue, falsetrue
--sm-scopelocal, globallocal
--sm-featuresstringN/A
--sm-ignore-buildstringfalse

Saving deploy Config in serverless.yml

You can also save config in serverless.yml file

custom:
  smConfig:
    deploy:
      scope: local
      parallel: true
      ignoreBuild: true

Examples (deploy Command)

Deploy all features locally

# deploy all local features
$ sls m deploy

Deploy all features globally

# deploy all global features
$ sls m deploy --sm-scope global

Deploy single feature

# deploy all global features
$ sls m deploy --sm-features jedi

Deploy Multiple features

# deploy all global features
$ sls m deploy --sm-features jedi,sith,dark_side

Deploy Multiple features in sequence

# deploy all global features
$ sls m deploy  --sm-features jedi,sith,dark_side --sm-parallel false

Author: aa2kb
Source Code: https://github.com/aa2kb/serverless-modular 
License: MIT license

#serverless #aws #node #lambda 

Gordon  Matlala

Gordon Matlala

1617875400

Adding Code to AWS Lambda, Lambda Layers, and Lambda Extensions Using Docker

2020 was a difficult year for all of us, and it was no different for engineering teams. Many software releases were postponed, and the industry slowed its development speed quite a bit.

But at least at AWS, some teams released updates out of the door at the end of the year. AWS Lambda received two significant improvements:

  • AWS Lambda Extensions; and
  • Support of Docker images for your functions.

With these two new features and Lambda Layers, we now have three ways to add code to Lambda that isn’t directly part of our Lambda function.

The question is now: when should we use what?

In this article, I try to shine some light on the Lambda Layers, Lambda Extensions, and Docker image for Lambda.

First things first. All these Lambda features can be used together. So if you think about where to put your code, at least your decisions aren’t mutually exclusive. You can upload a Docker image and attach a regular Lambda Layer and a Lambda Extension. The same is possible if your Lambda function is based on a ZIP archive.

What does this all mean? Keep reading and find out.

#aws #aws-lambda #serverless #devops #docker #lambda

Christa  Stehr

Christa Stehr

1598408880

How To Unite AWS KMS with Serverless Application Model (SAM)

The Basics

AWS KMS is a Key Management Service that let you create Cryptographic keys that you can use to encrypt and decrypt data and also other keys. You can read more about it here.

Important points about Keys

Please note that the customer master keys(CMK) generated can only be used to encrypt small amount of data like passwords, RSA key. You can use AWS KMS CMKs to generate, encrypt, and decrypt data keys. However, AWS KMS does not store, manage, or track your data keys, or perform cryptographic operations with data keys.

You must use and manage data keys outside of AWS KMS. KMS API uses AWS KMS CMK in the encryption operations and they cannot accept more than 4 KB (4096 bytes) of data. To encrypt application data, use the server-side encryption features of an AWS service, or a client-side encryption library, such as the AWS Encryption SDK or the Amazon S3 encryption client.

Scenario

We want to create signup and login forms for a website.

Passwords should be encrypted and stored in DynamoDB database.

What do we need?

  1. KMS key to encrypt and decrypt data
  2. DynamoDB table to store password.
  3. Lambda functions & APIs to process Login and Sign up forms.
  4. Sign up/ Login forms in HTML.

Lets Implement it as Serverless Application Model (SAM)!

Lets first create the Key that we will use to encrypt and decrypt password.

KmsKey:
    Type: AWS::KMS::Key
    Properties: 
      Description: CMK for encrypting and decrypting
      KeyPolicy:
        Version: '2012-10-17'
        Id: key-default-1
        Statement:
        - Sid: Enable IAM User Permissions
          Effect: Allow
          Principal:
            AWS: !Sub arn:aws:iam::${AWS::AccountId}:root
          Action: kms:*
          Resource: '*'
        - Sid: Allow administration of the key
          Effect: Allow
          Principal:
            AWS: !Sub arn:aws:iam::${AWS::AccountId}:user/${KeyAdmin}
          Action:
          - kms:Create*
          - kms:Describe*
          - kms:Enable*
          - kms:List*
          - kms:Put*
          - kms:Update*
          - kms:Revoke*
          - kms:Disable*
          - kms:Get*
          - kms:Delete*
          - kms:ScheduleKeyDeletion
          - kms:CancelKeyDeletion
          Resource: '*'
        - Sid: Allow use of the key
          Effect: Allow
          Principal:
            AWS: !Sub arn:aws:iam::${AWS::AccountId}:user/${KeyUser}
          Action:
          - kms:DescribeKey
          - kms:Encrypt
          - kms:Decrypt
          - kms:ReEncrypt*
          - kms:GenerateDataKey
          - kms:GenerateDataKeyWithoutPlaintext
          Resource: '*'

The important thing in above snippet is the KeyPolicy. KMS requires a Key Administrator and Key User. As a best practice your Key Administrator and Key User should be 2 separate user in your Organisation. We are allowing all permissions to the root users.

So if your key Administrator leaves the organisation, the root user will be able to delete this key. As you can see **KeyAdmin **can manage the key but not use it and KeyUser can only use the key. ${KeyAdmin} and **${KeyUser} **are parameters in the SAM template.

You would be asked to provide values for these parameters during SAM Deploy.

#aws #serverless #aws-sam #aws-key-management-service #aws-certification #aws-api-gateway #tutorial-for-beginners #aws-blogs

Running TensorFlow on AWS Lambda using Serverless

For smaller workloads, serverless platforms such as AWS Lambda can be a fast and low-cost option for deploying machine learning models. As the application grows, pieces can then be moved to dedicated servers, or PaaS options such as AWS Sagemaker, if necessary.

Although it is instructive to first use Lambda by uploading code directly, it is best to select a framework so that you can leverage additional integrations, such as API Gateway with AWS. For this example we will use TensorFlow as the machine learning library, and so we will look for frameworks that can deploy Python applications.

Zappa is well known for being able to easily deploy existing Flask or Django apps, however since we are creating this with serverless in mind from the start we will select the ubiqutous and powerful Serverless framework.

When treating infrastructure configuration as a first-class citizen it is advisable to first create a shell of the application and deploy it, and then write the actual code. This allows for rapid iterations that are close to the end-state, and avoids costly surprises down the road.

Structuring the project

For machine learning most of the work can be categorized into three critical steps:

  • Retrieving, cleaning, and uploading the input data
  • Training the model and saving the results
  • Inferring (i.e. predicting) a new result based on a new set of data

At its core, designing for serverless platforms means thinking of how to segment your code by individual deployable functions. In light of the categories above, we will structure our project like so:

├── TfLambdaDemo
│   ├── upload.py
│   ├── train.py
│   └── infer.py

Be sure to also create a new virtualenv:

$ pyenv virtualenv 3.6.5 tflambdademo
$ pyenv activate tflambdademo

Adding Lambda handlers

A “handler” is the term used for the function that will actually be invoked by Lambda, and is always called with two parameters, event and context. From the docs:

event – AWS Lambda uses this parameter to pass in event data to the handler. This parameter is usually of the Python dict type. It can also be liststr, intfloat, or NoneType type.

context – AWS Lambda uses this parameter to provide runtime information to your handler. This parameter is of the LambdaContexttype.

If you were invoking the functions directly, event would be of a type made in that call. However, we will plan to invoke using an HTTP POST request through API Gateway, which means the data will be contained in event['body'] and we will need to return a compatible response.

To get started, add boilerplate functions into each of the .py files mentioned above:

import json

def uploadHandler(event, context):
    body = json.loads(event.get('body'))
    response = {
        "statusCode": 200,
        "body": json.dumps(body)
    }
    return response

Installing and configuring Serverless

If you are not familiar with Serverless, the first thing to note is that it is actually based on Node.js. This may seem odd since we are building a Python app, but it makes sense once you realize that this framework is really a developer CLI tool and not something that ships with your product.

On a Mac you can install via Homebrew:

$ brew update
$ brew install node

Verify that you have node installed, as well as the package manager npm:

$ node --version
$ npm --version

Install Serverless as a global package (-g) and verify that the serverless command is now available on your CLI:

$ npm install -g serverless
$ serverless

Create a new Serverless project in the TfLambdaDemo directory:

$ serverless create --template aws-python

Notice the new file serverless.yml and how your .gitignore file was auto-populated. Serverless also created handler.py template file, but you can delete this.

When you open serverless.yml you will see a lot of boilerplate information, which is good to familiarize yourself with. First update the service name totflambdademo, and then update theprovider section to AWS running Python 3.6 in the region of your choice. Defining the stage is useful when managing production deployments, but for now we will leave it as dev.

In the functions section list out uploadtrain, and infer with a handler format of <filename>.<function>. The events section contains the information for how the function will be called. Since we plan to use API Gateway, we will add the http trigger, and set the timeouts to 30 seconds to match.

In AWS Lambda the allocated memory can be configured, and then CPU is scaled accordingly. Since the machine learning training operation will be computationally intensive change from the default of 1024 MB to the maximum of 3008 MB (we can optimize later).

Your serverless.yml file should now look like:

service: tflambdademo

provider:
  name: aws
  region: us-east-1
  runtime: python3.6
  stage: dev
functions:
  upload:
    handler: upload.uploadHandler
    timeout: 30
    events:
      - http:
          path: upload
          method: post
  train:
    handler: train.trainHandler
    timeout: 30
    memory: 3008
    events:
      - http:
          path: train
          method: post
          async: true
  infer:
    handler: infer.inferHandler
    timeout: 30
    events:
      - http:
          path: infer
          method: post

Since we already added boilerplate functionality into the handlers, we can deploy with the following command:

Note: To deploy you will need an AWS account and your credentials properly configured. For details see the docs.

$ serverless deploy -v

At the end you should see the following information, where <id> will be custom to your deployment:

Service Information
service: tflambdademo
stage: dev
region: us-east-1
stack: tflambdademo-dev
resources: 22
api keys:
  None
endpoints:
  POST - https://<id>.execute-api.us-east-1.amazonaws.com/dev/upload
  POST - https://<id>.execute-api.us-east-1.amazonaws.com/dev/train
  POST - https://<id>.execute-api.us-east-1.amazonaws.com/dev/infer
functions:
  upload: tflambdademo-dev-upload
  train: tflambdademo-dev-train
  infer: tflambdademo-dev-infer
layers:
  None

What just happened? Serverless took care of all the heavy lifting by first creating a new S3 bucket and uploading your code, and then creating a CloudFormation template that executed the following:

  • Create the Lambda functions
  • Create API gateway with the defined endpoints configured to integrated with the handlers
  • Create a new IAM role and the proper permissions
  • Create a new log group viewable in CloudWatch

Test out the new endpoint by verifying that a request body is sent back (remember to replace <id>):

$ curl -X POST https://<id>.execute-api.us-east-1.amazonaws.com/dev/infer -d '{"foo": "bar"}'

{"foo": "bar"}

Magic!

Adding in TensorFlow

Before doing anything else, let’s see if we can successfully add TensorFlow to our project. To each of the .py files add import TensorFlow as tf. Then install via pip and re-deploy.

$ pip install tensorflow
$ pip freeze > requirements.txt
$ serverless deploy -v

Everything looks fine, but when we try to test the endpoint we get an error:

$ curl -X POST https://<id>.execute-api.us-east-1.amazonaws.com/dev/infer -d '{"foo": "bar"}'

{"message": "Internal server error"}

If we go to CloudWatch we can see the following error:

Unable to import module 'infer': No module named 'tensorflow'

This seems surprising since invoking the function locally is successful:

$ serverless invoke local --function infer
{
    "statusCode": 200,
    "body": "null"
}

The reason is that even though we installed TensorFlow to our virtual environment, there was no mechanism to add these libraries to our Lambda package. The only content in that package was our raw code, which up until now depended only on the pre-bundled library json.

Adding the serverless-python-requirements plugin

One great thing about Serverless is the extensibility via plugins. In order to bundle dependencies from our requirements.txt file we will use serverless-python-requirements.

Installing this plugin will add a package.json file, thenode_modules directory (be sure to add this to your gitignore!), and a plugins section to your serverless.yml file.

$ serverless plugin install -n serverless-python-requirements

To give us the most flexibility we will use the Dockerize option. This avoids some complexity with using the binaries from our virtual environment, and also allows for compiling non-pure-Python libraries. To select this option, odd the following section to your serverless.yml file:

Note: You will need Docker installed on your machine to continue.

custom:
  pythonRequirements:
    dockerizePip: true

If we now run serverless deploy -v we can see additional upfront actions to create the Docker image. However, it still fails!

An error occurred: UploadLambdaFunction - Unzipped size must be smaller than 262144000 bytes (Service: AWSLambdaInternal; Status Code: 400; Error Code: InvalidParameterValueException; Request ID: c3b94dc7-6a06-11e9-8823-bb373647997a).

Our zipped payload for Lambda balloons from 6.5KB to 126.9MB, but more importantly the unzipped size is 509MB which is not even close to the 262MB limit! If we download the zip file from S3 we can see that 399MBs are coming from the tensorflow folder.

How do I fit all this stuff in that box?

To get everything down to size we will employ three techniques available in the serverless-python-requirements plugin:

  • zip — Compresses the libraries in an additional .requirements.zip file and addsunzip_requirements.py in the final bundle.
  • slim — Removes unneeded files and directories such as *.so*.pycdist-info, etc.
  • noDeploy — Omits certain packages from deployment. We will use the standard list that includes those already built into Lambda, as well as Tensorboard.

The custom section in your serverless.yml file should now look like:

custom:
  pythonRequirements:
    dockerizePip: true
    zip: true
    slim: true
    noDeploy:
      - boto3
      - botocore
      - docutils
      - jmespath
      - pip
      - python-dateutil
      - s3transfer
      - setuptools
      - six
      - tensorboard

You will also need to add the following as the first four lines in the .py files. This step will unzip the requirements file on Lambda, but will be skipped when running locally since unzip_requirements.py only exists in the final bundle.

try:
  import unzip_requirements
except ImportError:
  pass

Running deploy will now succeed, and we can again test our endpoint to verify the function works.

$ serverless deploy -v
...

$ curl -X POST https://<id>.execute-api.us-east-1.amazonaws.com/dev/infer -d '{"foo": "bar"}'
{"foo": "bar"}

Inspecting the file on S3, we can see that our semi-unzipped packaged is now 103MB (under the 262MB limit), and the fully unzipped package with all of the libraries is 473MB (narrowly under the 500MB total local storage limit). Success!

It’s important to recognize that we haven’t actually written a real line of code related to machine learning yet. If the infrastructure configuration is critical, the above is a validation of why it is important to start with a deployable shell first. It will help inform what restrictions you may have as you start to build out the application (e.g. you will not be able to use another large library in combination with Tensorflow), or whether it is even possible.

Creating the Machine Learning functions

For this demonstration we will leverage a Linear Classifier example from TensorFlow, which uses the higher-level Estimator API:

Using census data which contains data a person’s age, education, marital status, and occupation (the features), we will try to predict whether or not the person earns more than 50,000 dollars a year (the target label). We will train a logistic regression model that, given an individual’s information, outputs a number between 0 and 1 — this can be interpreted as the probability that the individual has an annual income of over 50,000 dollars.

Specifically we will clone census_data.py from that project, which provides the functions for downloading and cleaning the data, as well as the input function.

upload.py

Since we will be using S3 to store our data, we need to add this resource into the serverless.yml file. First add an environment variable to define the bucket name and a new IAM role. Note that we can now refer to BUCKET inside this file as well as our application.

provider:
  name: aws
  region: us-east-1
  runtime: python3.6
  stage: dev
iamRoleStatements:
  - Effect: Allow
    Action:
      - s3:*
    Resource:
     Fn::Join:
       - ""
       - - "arn:aws:s3:::"
         - ${self:provider.environment.BUCKET}
         - "/*"
environment:
  BUCKET: tflambdademo

Next add a new resource section which will actually create the S3 bucket:

resources:
  Resources:
    SageBucket:
      Type: AWS::S3::Bucket
      Properties:
        BucketName: ${self:provider.environment.BUCKET}

Update the upload handler to download the new data to the local Lambda storage location /tmp/, and then upload to S3. We will use an epoch prefix to separate each data-model pair.

Since no data needs to be read in from the request body we will delete that line, and also modify the response to return the epoch.

try:
  import unzip_requirements
except ImportError:
  pass
import os
import json
import time
import boto3
import tensorflow as tf
import census_data
FILE_DIR = '/tmp/'
BUCKET = os.environ['BUCKET']
def uploadHandler(event, context):
    ## Download data to local tmp directory
    census_data.download(FILE_DIR)
    ## Upload files to S3
    epoch_now = str(int(time.time()))
    boto3.Session(
        ).resource('s3'
        ).Bucket(BUCKET
        ).Object(os.path.join(epoch_now,census_data.TRAINING_FILE)
        ).upload_file(FILE_DIR+census_data.TRAINING_FILE)
    boto3.Session(
        ).resource('s3'
        ).Bucket(BUCKET
        ).Object(os.path.join(epoch_now,census_data.EVAL_FILE)
        ).upload_file(FILE_DIR+census_data.EVAL_FILE)
    response = {
        "statusCode": 200,
        "body": json.dumps({'epoch': epoch_now})
    }
    return response

At this point we can re-deploy the functions and trigger the upload function:

$ serverless deploy -v
...

$ curl -X POST https://<id>.execute-api.us-east-1.amazonaws.com/dev/upload
{"epoch": "1556995767"}

If you navigate to the new S3 bucket you should see the CSV files adult.data and adult.test under a folder prefix defined by the epoch in the response.

train.py

The train function will download the data from S3 based on an epoch passed in the POST body.

def trainHandler(event, context):
    time_start = time.time()

    body = json.loads(event.get('body'))
    ## Read in epoch
    epoch_files = body['epoch']
    ## Download files from S3
    boto3.Session(
        ).resource('s3'
        ).Bucket(BUCKET
        ).download_file(
            os.path.join(epoch_files,census_data.TRAINING_FILE),
            FILE_DIR+census_data.TRAINING_FILE)
    boto3.Session(
        ).resource('s3'
        ).Bucket(BUCKET
        ).download_file(
            os.path.join(epoch_files,census_data.EVAL_FILE),
            FILE_DIR+census_data.EVAL_FILE)

In order to setup the estimator a set of feature columns must be provided. These columns can be thought of as placeholders to tell the model how to handle raw inputs. The census_data.py module provides a function to create two sets of columns for a wide and deep model. For this simple example we will only use the wide columns.

To actually execute the training we must provide an input function to the newly configured estimator. The input function reads data (in our case from a CSV file), and converts it into a TensorFlow tensor. However, the estimator expects an input function that has no arguments, and therefore we will use a partial function to create a new callable.

def trainHandler(event, context):

    ...

    ## Create feature columns
    wide_cols, deep_cols = census_data.build_model_columns()
    ## Setup estimator
    classifier = tf.estimator.LinearClassifier(
                        feature_columns=wide_cols,
                        model_dir=FILE_DIR+'model_'+epoch_files+'/')
    ## Create callable input function and execute train
    train_inpf = functools.partial(
                    census_data.input_fn,
                    FILE_DIR+census_data.TRAINING_FILE,
                    num_epochs=2, shuffle=True,
                    batch_size=64)
    classifier.train(train_inpf)

We will then repeat this with the test data we held back in order to evaluate the model, and print the results to the logs.

def trainHandler(event, context):

    ...
    ## Create callable input function and execute evaluation
    test_inpf = functools.partial(
                    census_data.input_fn,
                    FILE_DIR+census_data.EVAL_FILE,
                    num_epochs=1, shuffle=False,
                    batch_size=64)
    result = classifier.evaluate(test_inpf)
    print('Evaluation result: %s' % result)

In order to save the model to re-use for creating predictions, we will zip the files up and save to the same S3 folder.

def trainHandler(event, context):

    ...
    ## Zip up model files and store in s3
    with tarfile.open(FILE_DIR+'model.tar.gz', mode='w:gz') as arch:
        arch.add(FILE_DIR+'model_'+epoch_files+'/', recursive=True)
    boto3.Session(
        ).resource('s3'
        ).Bucket(BUCKET
        ).Object(os.path.join(epoch_files,'model.tar.gz')
        ).upload_file(FILE_DIR+'model.tar.gz')

Finally, we will prepare the result data for JSON serialization in the response, and also add in a runtime calculation.

def trainHandler(event, context):

    ...
    ## Convert result from float32 for json serialization
    for key in result:
        result[key] = result[key].item()
    runtime = round(time.time()-time_start, 1)
    response = {
        "statusCode": 200,
        "body": json.dumps({'epoch': epoch_files,
                            'runtime': runtime,
                            'result': result})
    }
    return response

Assuming that the test run after updating upload.py was successful, you can now deploy and test the function with that epoch key.

$ serverless deploy -v
...

$ curl -X POST https://<id>.execute-api.us-east-1.amazonaws.com/dev/train -d '{"epoch": "1556995767"}'
{"epoch": "1556995767", "runtime": 11.6, "result": {"accuracy": 0.8363736867904663, "accuracy_baseline": 0.7637737393379211, "auc": 0.8843450546264648, "auc_precision_recall": 0.697192907333374, "average_loss": 0.35046106576919556, "label/mean": 0.23622627556324005, "loss": 22.37590789794922, "precision": 0.698722243309021, "prediction/mean": 0.23248426616191864, "recall": 0.5403016209602356, "global_step": 1018}}

83.6% accuracy…not too bad, and in line with the results from the TensorFlow official example! If you visit the S3 bucket you should also see a saved file model.tar.gz in the epoch folder.

infer.py

The first step to building out the inference function is to think about how the data is coming in. The raw data should look just like the CSV input file we used above, except now it will come in the POST body.

The input function in census_data.py was built to stream CSV data from disk, which is scaleable for larger volumes. In our application we would only expect to make a small number of predictions at once, which would have no problem fitting into a small memory footprint, so we can use an easy input methodology.

To infer.py add a new function that will take in a dictionary where the keys represent the same columns that were in the CSV file and map to lists of values. Each “column” will then be converted to a numpy array with its datatype specified according to census_data.py.

Being able to use numpy makes it easy to convert to tensors, and there is no cost to our package size since it is a dependency for TensorFlow already.

def _easy_input_function(data_dict, batch_size=64):
    """
    data_dict = {
        '<csv_col_1>': ['<first_pred_value>', '<second_pred_value>']
        '<csv_col_2>': ['<first_pred_value>', '<second_pred_value>']
        ...
    }
    """
    ## Convert input data to numpy arrays
    for col in data_dict:
        col_ind = census_data._CSV_COLUMNS.index(col)
        dtype = type(census_data._CSV_COLUMN_DEFAULTS[col_ind][0])
        data_dict[col] = np.array(data_dict[col],
                                        dtype=dtype)
    labels = data_dict.pop('income_bracket')
    ds = tf.data.Dataset.from_tensor_slices((data_dict, labels))
    ds = ds.batch(batch_size)
    return ds

Back to the main handler function we will read in the prediction data and epoch identifier, and then download and extract the model file.

def inferHandler(event, context):
    body = json.loads(event.get('body'))

    ## Read in prediction data as dictionary
    ## Keys should match _CSV_COLUMNS, values should be lists
    predict_input = body['input']
    ## Read in epoch
    epoch_files = body['epoch']
    ## Download model from S3 and extract
    boto3.Session(
        ).resource('s3'
        ).Bucket(BUCKET
        ).download_file(
            os.path.join(epoch_files,'model.tar.gz'),
            FILE_DIR+'model.tar.gz')
    tarfile.open(FILE_DIR+'model.tar.gz', 'r').extractall(FILE_DIR)

As in train.py we need to setup the estimator, but now warm_start_from is specified which tells TensorFlow to load the previously run model. To setup the prediction we will use thepredict() method, and pass in the previously created input function with a lambda to make it callable.

The output of this method is an iterable, which we will convert to lists and store in a new result variable. Each item in the list will represent the result corresponding to the index of items in the lists from the input data.

def inferHandler(event, context):
    ...

    ## Create feature columns
    wide_cols, deep_cols = census_data.build_model_columns()
    ## Load model
    classifier = tf.estimator.LinearClassifier(
            feature_columns=wide_cols,
            model_dir=FILE_DIR+'tmp/model_'+epoch_files+'/',
            warm_start_from=FILE_DIR+'tmp/model_'+epoch_files+'/')
    ## Setup prediction
    predict_iter = classifier.predict(
                        lambda:_easy_input_function(predict_input))
    ## Iterate over prediction and convert to lists
    predictions = []
    for prediction in predict_iter:
        for key in prediction:
            prediction[key] = prediction[key].tolist()
        predictions.append(prediction)
    response = {
        "statusCode": 200,
        "body": json.dumps(predictions,
                            default=lambda x: x.decode('utf-8'))
    }
    return response

Building on invoking upload and train in the previous steps, we can pass in a row from the CSV file to test the function after re-deploying.

$ serverless deploy -v
...

$ curl -X POST https://<id>.execute-api.us-east-1.amazonaws.com/dev/infer -d '{"epoch": "1556995767", "input": {"age": ["34"], "workclass": ["Private"], "fnlwgt": ["357145"], "education": ["Bachelors"], "education_num": ["13"], "marital_status": ["Married-civ-spouse"], "occupation": ["Prof-specialty"], "relationship": ["Wife"], "race": ["White"], "gender": ["Female"], "capital_gain": ["0"], "capital_loss": ["0"], "hours_per_week": ["50"], "native_country": ["United-States"], "income_bracket": [">50K"]}}'
[{"logits": [1.088104009628296], "logistic": [0.7480245232582092], "probabilities": [0.25197547674179077, 0.7480245232582092], "class_ids": [1], "classes": ["1"]}]

Since this was taken from the original dataset we can see that the correct label was >50K which the model successfully predicts at 74.8% versus 25.2% for <=50K.

Feel free to now declare yourself a psychic reader!

Final thoughts

There are distinct boundaries to this type of deployment for TensorFlow, specifically around duration, and it is always good to check whether a serverless infrastructure is actually cost effective.

If serverless is the right choice, there are a few steps that you can take to help expand the duration boundaries.

Call the functions asynchronously…

Serverless allows you to add async: true into the function configuration which would give you access to the full 900 second limit on Lambda, rather than the 30 second limit through API Gateway. In this case the request will only invoke the function, and not actually wait for a response. The downside is that you will need to setup other mechanisms to determine which epoch key you should use to train or invoke the model.

…or don’t use HTTP to trigger the training function at all

For many use cases, you may really only need to provide API Gateway integration with the invoke function. One pattern that could be used for the train function is to configure Serverless to trigger Lambda in response to S3 events. For example, when a new epoch partition is created with CSV files, train.py is automatically invoked to update the model.

Warm your functions

When invoking the train function you may have noticed that the request length was much longer than the actual runtime. This is because when you invoke the function for the first time Lambda must load your code and setup the environment (i.e. cold start). There is a great Serverless plugin available that allows you to configure automatic warming in the serverless.yml file.

To view the final code, visit https://github.com/mikepm35/TfLambdaDemo

#tensorflow #aws #serverless #python #machine-learning