Serverless ML. Is this the cheapest way to host Machine Learning models on AWS? I’m hosting my Resnet-152 (152 layer CNN) model on AWS for pretty much free right now. Does it scale? What’s the trade-offs? This is where “serverless” comes in.
I recently made a video on using vast.ai for inexpensively building Machine Learning models. This is useful for someone like me, who wants to build a model without a budget. Using vast.ai (I’m not affiliated with them), I’m able to build reasonably sized models, such as resnet152 (152 layer CNN) based models for image categorization, for just a few dollars. My workflow is to jump on a vast.ai GPU, such as RTX 3090, paying 80 cents per hour, do some training, do analysis, and then jump back to my laptop for further developing my plan of attack. A few iterations of this and I’m good to go.
But go where?
Ideally you want to host your model on a GPU for wicked fast inference. But for many models normal fast inference can be achieved on a reasonable CPU. Even so, a model can take up a few Gigabytes of RAM and you’ll need that sitting somewhere in the cloud. You can get low cost cloud hosting of a few cents per hour. But those hours add up and if you want to host multiple models, your model inference costs can start to significantly outpace your model building costs if your workflow looks like mine.
I have no obvious initial revenue stream coming from my models to pay for their hosting, but I want to build some models and host them nonetheless. If my friendly neighbour, Fred, is too busy to take a look, then I don’t want to pay for hosting costs of models sitting idle. This is where “serverless” comes in.
Serverless is good spotty traffic patterns. “Serverless” is not serverless. You’re still using servers. You’re just sharing them with others in a blackbox context-switching way. The opaque relationship between you and the server means that it’s hard to optimize the underlying server for your specific use-case. The context-switching between the different users of that server means that, unbeknownst to you, resources are being loaded in and out continuously. These inefficiencies come at a cost, but the per-usage premium you pay for far less usage means everyone wins.
These economics obviously get less compelling to the consumer as utilization increases, but your cloud provider will attempt to close the gap with caching and same server re-use as much as possible.
Adding Code to AWS Lambda, Lambda Layers, and Lambda Extensions Using Docker. With Docker, we have three ways to add code to Lambda that isn’t directly part of our Lambda function. Try to AWS Lambda, Lambda Layers, and Lambda Extensions Using Docker.
Serverless Express enables you to easily host Express.js APIs on AWS Lambda and AWS HTTP API. Here is how to get started and deliver a Serverless Express.js based API with a custom domain, free SSL certificate and much more!
When to use Docker on AWS Lambda, Lambda Layers, and Lambda Extensions. AWS Lambda Extensions; and; Support of Docker images for your functions. With these two new features and Lambda Layers. AWS Lambda Extensions; and Support of Docker images for your functions.
Serverless Proxy with AWS API Gateway and AWS Lambda. We can communicate between Public and Private instance via a Serverless Proxy thanks to AWS Api Gateway and AWS Lambda. Github Webhook calls a Public API Gateway, API Gateway triggers a Lambda attached to VPC.
Serverless Framework: Use AWS S3 Object Lambda to resize images on the fly. I will show you how to use AWS S3 Object Lambda to resize images on the fly. The Serverless Framework will be used to define the Infrastructure as Code and to simplify the deployment.