1565412983
Originally published by PHIL NASH at twilio.com
When AWS launched Lambda in 2014 there was no love for Ruby. Platforms like Python, Node.js, and Java started the serverless revolution for hosting and running functions in the cloud. At the end of 2018, support for Ruby was finally launched.
You can build with Ruby on Lambda using raw functions and Serverless Application Model (SAM) templates as described in the getting started guide for Ruby on Lambda, but Ruby is all about developer happiness and when the config file is longer than your program the process could be described as painful. Enter the Jets framework a framework that "leverages the power of Ruby to make serverless joyful for everyone."
Jets combines the experience of building a Rails application with the ability to deploy to AWS Lambda and related services, including API Gateway, S3, and DynamoDB. In this post we're going to see how to get started with Jets and deploy a Lambda powered Twilio application written in Ruby.
To keep things simple we're going to build an SMS based app and to make it a bit more fun, we're going to include a little humour. When you text into the application it will respond with a bad joke. Sorry, misspelling there, I meant a dad joke, courtesy of the icanhazdadjoke API.
This application will show us how to get started with Jets, create controllers, actions and routes. We'll build a single endpoint that responds to HTTP requests from Twilio when a number receives an incoming SMS message. The endpoint will return TwiML with a random dad joke each time and hilarity will ensue.
To follow along with this project you'll need:
Got all that? Then let's get started.
We'll start by installing the jets
gem globally. On the command line type:
gem install jets
The Jets executable can then be used like Rails as a generator to initiate a new project and then run commands within the project. We'll create a new project now, but given our scope in this post we are going to limit it a little. First up, we're going to create the project in API mode as there's no need for HTML views in this application. That also avoids asset compilation with webpacker, which will save in our build time. We also don't need a database. Run the following command to create a new project:
jets new dad-jokes-sms --mode api --no-database
Once the generator has finished running, change into the dad-jokes-sms
directory and run your application locally with the following command:
jets serve
Open up http://localhost:8888 and you will see a page that look like this:
If you see this page then your new Jets project is running successfully.
Now we have a Jets application we can use the gem to generate parts of our application, from models and controllers to full scaffolds. For our application we're going to need a controller with one action. Generate it with the following:
jets generate controller Messages create
The generator will create and edit a number of files for us. We need to check the new routes so open config/routes.rb
.
Jets generated a GET
route, but Twilio webhooks make POST
requests by default and I prefer to keep it that way. Set the application up to receive POST
request webhooks on the /messages
endpoint instead like so:
Jets.application.routes.draw do post 'messages', to: 'messages#create' root "jets/public#show"# The jets/public#show controller can serve static utf8 content out of the public folder.
# Note, as part of the deploy process Jets uploads files in the public folder to s3
# and serves them out of s3 directly. S3 is well suited to serve static assets.
# More info here: http://rubyonjets.com/docs/assets-serving/
any “*catchall”, to: “jets/public#show”
end
Now, let’s go write our controller action. Open app/controllers/messages_controller.rb
and you will see one method for the create
action. This is the action which will receive our Twilio webhook and respond with TwiML to send back a dad joke.
To send back a dad joke we need to make a call to the icanhazdadjoke API. Let’s write a quick private method we can use to achieve this.
We’ll use open-uri
as it’s useful for making simple web requests (including downloading files and images). The API will respond with plain text if we ask it to, which saves us doing any parsing. Add the following to the MessagesController
:
require ‘open-uri’class MessagesController < ApplicationController
def create
endprivate
def random_joke
open(‘https://icanhazdadjoke.com/’, { ‘Accept’ => ‘text/plain’ }).read
end
end
Now we’re ready to return our joke to Twilio as TwiML.
We’ll build up a response using the helpers from the twilio-ruby helper library. Open the Gemfile
and add twilio-ruby
:
source “https://rubygems.org”gem “jets”
gem “twilio-ruby”
On the command line, run bundle install
to install the gem. Now in the create
action instantiate a new TwiML response object, reply to the incoming message using the <Message>
TwiML element and render the XML response, like so:
require ‘open-uri’class MessagesController < ApplicationController
def create
twiml = Twilio::TwiML::MessagingResponse.new
twiml.message body: random_joke
render xml: twiml.to_xml
endprivate
def random_joke
open(‘https://icanhazdadjoke.com/’, { ‘Accept’ => ‘text/plain’ }).read
end
end
You can read more about how to use the twilio-ruby
helper library for generating TwiML in the documentation.
We can run this locally to test we are getting the expected response. If you stopped the application, start it again with jets serve
. Make a POST
request to localhost:8888/messages using curl
and you’ll see your joke, provided by icanhazdadjoke, in the TwiML response:
curl --data “” http://localhost:8888/messages
<?xml version=“1.0” encoding=“UTF-8”?>
<Response>
<Message>What’s the advantage of living in Switzerland? Well, the flag is a big plus.</Message>
</Response>
Great, our Jets application is working! Now to deploy it to AWS Lambda.
To deploy our Jets application to AWS we first need to set up our project with credentials to allow it to access AWS services. A good practice here is to create a user that has the minimum number of permissions required to do everything it needs to. The Jets documentation describes the minimum permissions that our user will need. Within our AWS account we’re going to create a policy that contains these permissions and a new user that will be assigned the policy. We can then use that user’s credentials to deploy our application.
In your AWS console find the IAM service (or head straight to the IAM section).
Go to the Policies section and create a new policy.
Choose the JSON tab and enter the following JSON from the Jets documentation:
{
“Version”: “2012-10-17”,
“Statement”: [
{
“Effect”: “Allow”,
“Action”: [
“apigateway:",
"cloudformation:”,
“dynamodb:",
"events:”,
“iam:",
"lambda:”,
“logs:",
"route53:”,
“s3:"
],
“Resource”: [
"”
]
}
]
}
Click through to review the policy and give it a name.
Save the policy. Now we need to create a new user and attach the policy to it, giving it the permissions to create the resources Jets needs to deploy. Open the Users section in the IAM console and create a new user.
Give the user a name and select Programmatic Access for the Access Type.
Click Next to choose the permissions for your new user. Choose Attach existing policies directly and filter for the name of the policy you just created. Select that policy and click Next.
Click Next until you reach the success page.
Save the Access key ID and Secret access key from the last screen. We’ll need them to deploy with. Now we’re ready to deploy.
On the command line enter:
AWS_ACCESS_KEY_ID=YOUR_USER_KEY AWS_SECRET_ACCESS_KEY=YOUR_USER_SECRET_KEY jets deploy
Jets will use the credentials as environment variables to set up all the resources in your AWS account to run the application. It takes a little while, but when it is complete you will have a URL where your application is running.
You’ll notice this deployed to a “dev” environment. You can read more about how Jets handles environments in the documentation.
We can now test this URL using curl
. Remember we use the route /messages
so add that to the end of your API Gateway endpoint and make a POST
request.
$ curl --data “” https://YOUR_API_GATEWAY_ENDPOINT/messages
<?xml version=“1.0” encoding=“UTF-8”?>
<Response>
<Message>I knew a guy who collected candy canes, they were all in mint condition</Message>
</Response>
Now to have dad jokes on hand at all times let’s hook this up to a Twilio number.
Head into your Twilio console to your active phone numbers. If you already have a number you want to work with, edit it, otherwise buy a new number that can receive incoming SMS messages. In the field for A message comes in enter your application URL.
Save your number and send it a message. You should get a dad joke in response. Now, in celebration, go and tell someone nearby that joke and let me know whether they laugh or groan.
In this post we’ve seen how to get started with Jets to write Ruby applications that you can deploy to AWS Lambda. You can see the full project on GitHub.
There’s a lot more to what Jets can help you accomplish, including responding to events, storing data in databases and even run your existing Rails application. Check out these articles from the Jets documentation for more on what you can do with Jets.
Originally published by PHIL NASH at twilio.com
==================================================
Thanks for reading :heart: If you liked this post, share it with all of your programming buddies! Follow me on Facebook | Twitter
☞ Ruby on Rails REST API: The Complete Guide
☞ How To Install Ruby On Rails On Windows
☞ The Complete Ruby on Rails Developer Course
☞ Learn Ruby on Rails from Scratch
☞ Testing Ruby with RSpec: The Complete Guide
☞ Build a Crypto Currency Portfolio App With Ruby on Rails
☞ Ruby On Rails Web Development: To-Do List App
☞ Advanced Ruby Programming: 10 Steps to Mastery
#ruby-on-rails
1655426640
Serverless M (or Serverless Modular) is a plugin for the serverless framework. This plugins helps you in managing multiple serverless projects with a single serverless.yml file. This plugin gives you a super charged CLI options that you can use to create new features, build them in a single file and deploy them all in parallel
Currently this plugin is tested for the below stack only
Make sure you have the serverless CLI installed
# Install serverless globally
$ npm install serverless -g
To start the serverless modular project locally you can either start with es5 or es6 templates or add it as a plugin
# Step 1. Download the template
$ sls create --template-url https://github.com/aa2kb/serverless-modular/tree/master/template/modular-es6 --path myModularService
# Step 2. Change directory
$ cd myModularService
# Step 3. Create a package.json file
$ npm init
# Step 3. Install dependencies
$ npm i serverless-modular serverless-webpack webpack --save-dev
# Step 1. Download the template
$ sls create --template-url https://github.com/aa2kb/serverless-modular/tree/master/template/modular-es5 --path myModularService
# Step 2. Change directory
$ cd myModularService
# Step 3. Create a package.json file
$ npm init
# Step 3. Install dependencies
$ npm i serverless-modular --save-dev
If you dont want to use the templates above you can just add in your existing project
plugins:
- serverless-modular
Now you are all done to start building your serverless modular functions
The serverless CLI can be accessed by
# Serverless Modular CLI
$ serverless modular
# shorthand
$ sls m
Serverless Modular CLI is based on 4 main commands
sls m init
sls m feature
sls m function
sls m build
sls m deploy
sls m init
The serverless init command helps in creating a basic .gitignore
that is useful for serverless modular.
The basic .gitignore
for serverless modular looks like this
#node_modules
node_modules
#sm main functions
sm.functions.yml
#serverless file generated by build
src/**/serverless.yml
#main serverless directories generated for sls deploy
.serverless
#feature serverless directories generated sls deploy
src/**/.serverless
#serverless logs file generated for main sls deploy
.sm.log
#serverless logs file generated for feature sls deploy
src/**/.sm.log
#Webpack config copied in each feature
src/**/webpack.config.js
The feature command helps in building new features for your project
This command comes with three options
--name: Specify the name you want for your feature
--remove: set value to true if you want to remove the feature
--basePath: Specify the basepath you want for your feature, this base path should be unique for all features. helps in running offline with offline plugin and for API Gateway
options | shortcut | required | values | default value |
---|---|---|---|---|
--name | -n | ✅ | string | N/A |
--remove | -r | ❎ | true, false | false |
--basePath | -p | ❎ | string | same as name |
Creating a basic feature
# Creating a jedi feature
$ sls m feature -n jedi
Creating a feature with different base path
# A feature with different base path
$ sls m feature -n jedi -p tatooine
Deleting a feature
# Anakin is going to delete the jedi feature
$ sls m feature -n jedi -r true
The function command helps in adding new function to a feature
This command comes with four options
--name: Specify the name you want for your function
--feature: Specify the name of the existing feature
--path: Specify the path for HTTP endpoint helps in running offline with offline plugin and for API Gateway
--method: Specify the path for HTTP method helps in running offline with offline plugin and for API Gateway
options | shortcut | required | values | default value |
---|---|---|---|---|
--name | -n | ✅ | string | N/A |
--feature | -f | ✅ | string | N/A |
--path | -p | ❎ | string | same as name |
--method | -m | ❎ | string | 'GET' |
Creating a basic function
# Creating a cloak function for jedi feature
$ sls m function -n cloak -f jedi
Creating a basic function with different path and method
# Creating a cloak function for jedi feature with custom path and HTTP method
$ sls m function -n cloak -f jedi -p powers -m POST
The build command helps in building the project for local or global scope
This command comes with four options
--scope: Specify the scope of the build, use this with "--feature" tag
--feature: Specify the name of the existing feature you want to build
options | shortcut | required | values | default value |
---|---|---|---|---|
--scope | -s | ❎ | string | local |
--feature | -f | ❎ | string | N/A |
Saving build Config in serverless.yml
You can also save config in serverless.yml file
custom:
smConfig:
build:
scope: local
all feature build (local scope)
# Building all local features
$ sls m build
Single feature build (local scope)
# Building a single feature
$ sls m build -f jedi -s local
All features build global scope
# Building all features with global scope
$ sls m build -s global
The deploy command helps in deploying serverless projects to AWS (it uses sls deploy
command)
This command comes with four options
--sm-parallel: Specify if you want to deploy parallel (will only run in parallel when doing multiple deployments)
--sm-scope: Specify if you want to deploy local features or global
--sm-features: Specify the local features you want to deploy (comma separated if multiple)
options | shortcut | required | values | default value |
---|---|---|---|---|
--sm-parallel | ❎ | ❎ | true, false | true |
--sm-scope | ❎ | ❎ | local, global | local |
--sm-features | ❎ | ❎ | string | N/A |
--sm-ignore-build | ❎ | ❎ | string | false |
Saving deploy Config in serverless.yml
You can also save config in serverless.yml file
custom:
smConfig:
deploy:
scope: local
parallel: true
ignoreBuild: true
Deploy all features locally
# deploy all local features
$ sls m deploy
Deploy all features globally
# deploy all global features
$ sls m deploy --sm-scope global
Deploy single feature
# deploy all global features
$ sls m deploy --sm-features jedi
Deploy Multiple features
# deploy all global features
$ sls m deploy --sm-features jedi,sith,dark_side
Deploy Multiple features in sequence
# deploy all global features
$ sls m deploy --sm-features jedi,sith,dark_side --sm-parallel false
Author: aa2kb
Source Code: https://github.com/aa2kb/serverless-modular
License: MIT license
1598408880
The Basics
AWS KMS is a Key Management Service that let you create Cryptographic keys that you can use to encrypt and decrypt data and also other keys. You can read more about it here.
Important points about Keys
Please note that the customer master keys(CMK) generated can only be used to encrypt small amount of data like passwords, RSA key. You can use AWS KMS CMKs to generate, encrypt, and decrypt data keys. However, AWS KMS does not store, manage, or track your data keys, or perform cryptographic operations with data keys.
You must use and manage data keys outside of AWS KMS. KMS API uses AWS KMS CMK in the encryption operations and they cannot accept more than 4 KB (4096 bytes) of data. To encrypt application data, use the server-side encryption features of an AWS service, or a client-side encryption library, such as the AWS Encryption SDK or the Amazon S3 encryption client.
Scenario
We want to create signup and login forms for a website.
Passwords should be encrypted and stored in DynamoDB database.
What do we need?
Lets Implement it as Serverless Application Model (SAM)!
Lets first create the Key that we will use to encrypt and decrypt password.
KmsKey:
Type: AWS::KMS::Key
Properties:
Description: CMK for encrypting and decrypting
KeyPolicy:
Version: '2012-10-17'
Id: key-default-1
Statement:
- Sid: Enable IAM User Permissions
Effect: Allow
Principal:
AWS: !Sub arn:aws:iam::${AWS::AccountId}:root
Action: kms:*
Resource: '*'
- Sid: Allow administration of the key
Effect: Allow
Principal:
AWS: !Sub arn:aws:iam::${AWS::AccountId}:user/${KeyAdmin}
Action:
- kms:Create*
- kms:Describe*
- kms:Enable*
- kms:List*
- kms:Put*
- kms:Update*
- kms:Revoke*
- kms:Disable*
- kms:Get*
- kms:Delete*
- kms:ScheduleKeyDeletion
- kms:CancelKeyDeletion
Resource: '*'
- Sid: Allow use of the key
Effect: Allow
Principal:
AWS: !Sub arn:aws:iam::${AWS::AccountId}:user/${KeyUser}
Action:
- kms:DescribeKey
- kms:Encrypt
- kms:Decrypt
- kms:ReEncrypt*
- kms:GenerateDataKey
- kms:GenerateDataKeyWithoutPlaintext
Resource: '*'
The important thing in above snippet is the KeyPolicy. KMS requires a Key Administrator and Key User. As a best practice your Key Administrator and Key User should be 2 separate user in your Organisation. We are allowing all permissions to the root users.
So if your key Administrator leaves the organisation, the root user will be able to delete this key. As you can see **KeyAdmin **can manage the key but not use it and KeyUser can only use the key. ${KeyAdmin} and **${KeyUser} **are parameters in the SAM template.
You would be asked to provide values for these parameters during SAM Deploy.
#aws #serverless #aws-sam #aws-key-management-service #aws-certification #aws-api-gateway #tutorial-for-beginners #aws-blogs
1656636720
Serverless Framework: Deploy on Scaleway Functions
The Scaleway functions plugin for Serverless Framework allows users to deploy their functions and containers to Scaleway Functions with a simple serverless deploy
.
Serverless Framework handles everything from creating namespaces to function/code deployment by calling APIs endpoint under the hood.
npm install serverless -g
)Let's work into ~/my-srvless-projects
# mkdir ~/my-srvless-projects
# cd ~/my-srvless-projects
The easiest way to create a project is to use one of our templates. The list of templates is here
Let's use python3
serverless create --template-url https://github.com/scaleway/serverless-scaleway-functions/tree/master/examples/python3 --path myService
Once it's done, we can install mandatory node packages used by serverless
cd mypython3functions
npm i
Note: these packages are only used by serverless, they are not shipped with your functions.
Your functions are defined in the serverless.yml
file created:
service: scaleway-python3
configValidationMode: off
useDotenv: true
provider:
name: scaleway
runtime: python310
# Global Environment variables - used in every functions
env:
test: test
# Storing credentials in this file is strongly not recommanded for security concerns, please refer to README.md about best practices
scwToken: <scw-token>
scwProject: <scw-project-id>
# region in which the deployment will happen (default: fr-par)
scwRegion: <scw-region>
plugins:
- serverless-scaleway-functions
package:
patterns:
- '!node_modules/**'
- '!.gitignore'
- '!.git/**'
functions:
first:
handler: handler.py
# Local environment variables - used only in given function
env:
local: local
Note: provider.name
and plugins
MUST NOT be changed, they enable us to use the scaleway provider
This file contains the configuration of one namespace containing one or more functions (in this example, only one) of the same runtime (here python3
)
The different parameters are:
service
: your namespace nameuseDotenv
: Load environment variables from .env files (default: false), read Security and secret managementconfigValidationMode
: Configuration validation: 'error' (fatal error), 'warn' (logged to the output) or 'off' (default: warn)provider.runtime
: the runtime of your functions (check the supported runtimes above)provider.env
: environment variables attached to your namespace are injected to all your namespace functionsprovider.secret
: secret environment variables attached to your namespace are injected to all your namespace functions, see this example projectscwToken
: Scaleway token you got in prerequisitesscwProject
: Scaleway org id you got in prerequisitesscwRegion
: Scaleway region in which the deployment will take place (default: fr-par
)package.patterns
: usually, you don't need to configure it. Enable to include/exclude directories to/from the deploymentfunctions
: Configure of your fonctions. It's a yml dictionary, with the key being the function namehandler
(Required): file or function which will be executed. See the next section for runtime specific handlersenv
(Optional): environment variables specific for the current functionsecret
(Optional): secret environment variables specific for the current function, see this example projectminScale
(Optional): how many function instances we keep running (default: 0)maxScale
(Optional): maximum number of instances this function can scale to (default: 20)memoryLimit
: ram allocated to the function instances. See the introduction for the list of supported valuestimeout
: is the maximum duration in seconds that the request will wait to be served before it times out (default: 300 seconds)runtime
: (Optional) runtime of the function, if you need to deploy multiple functions with different runtimes in your Serverless Project. If absent, provider.runtime
will be used to deploy the function, see this example project.events
(Optional): List of events to trigger your functions (e.g, trigger a function based on a schedule with CRONJobs
). See events
section belowcustom_domains
(Optional): List of custom domains, refer to Custom Domain DocumentationYou configuration file may contains sensitive data, your project ID and your Token must not be shared and must not be commited in VCS.
To keep your informations safe and be able to share or commit your serverles.yml
file you should remove your credentials from the file. Then you can :
.env
file and keep it secretTo use .env
file you can modify your serverless.yml
file as following :
# This will alow the plugin to read your .env file
useDotenv: true
provider:
name: scaleway
runtime: node16
scwToken: ${env:SCW_SECRET_KEY}
scwProject: ${env:SCW_DEFAULT_PROJECT_ID}
scwRegion: ${env:SCW_REGION}
And then create a .env
file next to your serverless.yml
file, containing following values :
SCW_SECRET_KEY=XXX
SCW_DEFAULT_PROJECT_ID=XXX
SCW_REGION=fr-par
You can use this pattern to hide your secrets (for example a connexion string to a database or a S3 bucket).
Based on the chosen runtime, the handler
variable on function might vary.
Node has two module systems: CommonJS
modules and ECMAScript
(ES
) modules. By default, Node treats your code files as CommonJS modules, however ES modules have also been available since the release of node16
runtime on Scaleway Serverless Functions. ES modules give you a more modern way to re-use your code.
According to the official documentation, to use ES modules you can specify the module type in package.json
, as in the following example:
...
"type": "module",
...
This then enables you to write your code for ES modules:
export {handle};
function handle (event, context, cb) {
return {
body: process.version,
statusCode: 200,
};
};
The use of ES modules is encouraged, since they are more efficient and make setup and debugging much easier.
Note that using "type": "module"
or "type": "commonjs"
in your package.json will enable/disable some features in Node runtime. For a comprehensive list of differences, please refer to the official documentation, the following is a summary only:
commonjs
is used as default valuecommonjs
allows you to use require/module.exports
(synchronous code loading, it basically copies all file contents)module
allows you to use import/export
ES6 instructions (asynchronous loading, more optimized as it imports only the pieces of code you need)Path to your handler file (from serverless.yml), omit ./
, ../
, and add the exported function to use as a handler :
- src
- handlers
- firstHandler.js => module.exports.myFirstHandler = ...
- secondHandler.js => module.exports.mySecondHandler = ...
- serverless.yml
In serverless.yml:
provider:
# ...
runtime: node16
functions:
first:
handler: src/handlers/firstHandler.myFirstHandler
second:
handler: src/handlers/secondHandler.mySecondHandler
Similar to node
, path to handler file src/testing/handler.py
:
- src
- handlers
- firstHandler.py => def my_first_handler
- secondHandler.py => def my_second_handler
- serverless.yml
In serverless.yml:
provider:
# ...
runtime: python310 # or python37, python38, python39
functions:
first:
handler: src/handlers/firstHandler.my_first_handler
second:
handler: src/handlers/secondHandler.my_second_handler
Path to your handler's package, for example if I have the following structure:
- src
- testing
- handler.go -> package main in src/testing subdirectory
- second
- handler.go -> package main in src/second subdirectory
- serverless.yml
- handler.go -> package main at the root of project
Your serverless.yml functions
should look something like this:
provider:
# ...
runtime: go118
functions:
main:
handler: "."
testing:
handler: src/testing
second:
handler: src/second
With events
, you may link your functions with specific triggers, which might include CRON Schedule (Time based)
, MQTT Queues
(Publish on a topic will trigger the function), S3 Object update
(Upload an object will trigger the function).
Note that we do not include HTTP triggers in our event types, as a HTTP endpoint is created for every function. Triggers are just a new way to trigger your Function, but you will always be able to execute your code via HTTP.
Here is a list of supported triggers on Scaleway Serverless, and the configuration parameters required to deploy them:
rate
: CRON Schedule (UNIX Format) on which your function will be executedinput
: key-value mapping to define arguments that will be passed into your function's event object during execution.To link a Trigger to your function, you may define a key events
in your function:
functions:
handler: myHandler.handle
events:
# "events" is a list of triggers, the first key being the type of trigger.
- schedule:
# CRON Job Schedule (UNIX Format)
rate: '1 * * * *'
# Input variable are passed in your function's event during execution
input:
key: value
key2: value2
You may link Events to your Containers too (See section Managing containers
below for more informations on how to deploy containers):
custom:
containers:
mycontainer:
directory: my-directory
# Events key
events:
- schedule:
rate: '1 * * * *'
input:
key: value
key2: value2
You may refer to the follow examples:
Custom domains allows users to use their own domains.
For domain configuration please Refer to Scaleway Documentation
Integration with serverless framework example :
functions:
first:
handler: handler.handle
# Local environment variables - used only in given function
env:
local: local
custom_domains:
- func1.scaleway.com
- func2.scaleway.com
Note As your domain must have a record to your function hostname, you should deploy your function once to read its hostname. Custom Domains configurations will be available after the first deploy.
Note: Serverless Framework will consider the configuration file as the source of truth.
If you create a domain with other tools (Scaleway's Console, CLI or API) you must refer created domain into your serverless configuration file. Otherwise it will be deleted as Serverless Framework will give the priority to its configuration.
Requirements: You need to have Docker installed to be able to build and push your image to your Scaleway registry.
You must define your containers inside the custom.containers
field in your serverless.yml manifest. Each container must specify the relative path of its application directory (containing the Dockerfile, and all files related to the application to deploy):
custom:
containers:
mycontainer:
directory: my-container-directory
# port: 8080
# Environment only available in this container
env:
MY_VARIABLE: "my-value"
Here is an example of the files you should have, the directory
containing your Dockerfile and scripts is my-container-directory
.
.
├── my-container-directory
│ ├── Dockerfile
│ ├── requirements.txt
│ ├── server.py
│ └── (...)
├── node_modules
│ ├── serverless-scaleway-functions
│ └── (...)
├── package-lock.json
├── package.json
└── serverless.yml
Scaleway's platform will automatically inject a PORT environment variable on which your server should be listening for incoming traffic. By default, this PORT is 8080. You may change the port
in your serverless.yml
.
You may use the container example to getting started.
The serverless logs
command lets you watch the logs of a specific function or container.
Pass the function or container name you want to fetch the logs for with --function
:
serverless logs --function <function_or_container_name>
serverless info
command gives you informations your current deployement state in JSON format.
MUST
use this library if you plan to develop with Golang).This plugin is mainly developed and maintained by Scaleway Serverless Team
but you are free to open issues or discuss with us on our Community Slack Channels #serverless-containers and #serverless-functions.
Author: Scaleway
Source Code: https://github.com/scaleway/serverless-scaleway-functions
License: MIT license
1671284880
While I haven’t done too much with Serverless Framework and Functions as a Service (Faas) recently, I did in the past and it isn’t something that I’ve forgotten. In the past I demonstrated how to deploy Node.js functions to Amazon Web Services (AWS) Lambda that contain native dependencies. While not a necessity for all Lambda functions, it is for functions that use libraries for specific operating systems and architectures. For example, my previous article titled, Use AWS Lambda and API Gateway with Node.js and Couchbase NoSQL, fell into this situation. Making use of an EC2 instance or a Docker container with Amazon Linux will help most of the time, but there are scenarios where a little bit extra must be done to accomplish the task.
In certain circumstances everything may package and deploy correctly, but still throw errors. For example, a common error is around libstdc++ and a version of GLIBCXX not being found.
In this tutorial we’re going to see how to resolve library errors that might not be caught in a typical packaging and deployment scenario with Serverless Framework and AWS Lambda.
For this example we’re going to reference my previous Couchbase example, but we’re not going to think too hard on the code. Instead, lets imagine that we’ve packaged the functions and deployed them as I recommended previously.
When we try to run our functions, either from an API Gateway endpoint or directly as an invocation, we might come across this error in our CloudWatch logs:
module initialization error: Error
at Object.Module._extensions..node (module.js:681:18)
at Module.load (module.js:565:32)
at tryModuleLoad (module.js:505:12)
at Function.Module._load (module.js:497:3)
at Module.require (module.js:596:17)
at require (internal/module.js:11:18)
at bindings (/var/task/node_modules/bindings/bindings.js:84:48)
at Object.<anonymous> (/var/task/node_modules/couchbase/lib/binding.js:213:36)
at Module._compile (module.js:652:30)
at Object.Module._extensions..js (module.js:663:10)
Great, the above error is not at all helpful to us. For whatever reason a module didn’t initialize and all we can see is that Couchbase was involved somehow. To be fair, I was getting this error when I looked at my CloudWatch dashboard, not when I tried to invoke the functions directly.
If you search the web for the error in question, you get a lot of results that indicate that you probably have native dependencies and you just need to build them using the same flavor and architecture of Linux that AWS Lambda uses. Well, that is great, but if you followed my previous tutorials you’ll know that we downloaded the amazonlinux image on Docker and downloaded our Node.js modules through that container deployment.
So we have to dig a bit deeper into the problem.
If we invoke the function directly, without API Gateway or some other trigger, we get a new error:
{
"errorMessage": "/var/lang/bin/../lib/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /var/task/node_modules/couchbase/build/Release/couchbase_impl.node)",
"errorType": "Error",
"stackTrace": [
"Module.load (module.js:565:32)",
"tryModuleLoad (module.js:505:12)",
"Function.Module._load (module.js:497:3)",
"Module.require (module.js:596:17)",
"require (internal/module.js:11:18)",
"bindings (/var/task/node_modules/bindings/bindings.js:84:48)",
"Object.<anonymous> (/var/task/node_modules/couchbase/lib/binding.js:213:36)",
"Module._compile (module.js:652:30)",
"Object.Module._extensions..js (module.js:663:10)"
]
}
To invoke your function you can either use serverless invoke -f <function-name>
with the Serverless CLI, or through the AWS Lambda dashboard. In this new error, we have information regarding libstdc++ and GLIBCXX. If you’re like me, you have absolutely no idea what that is. While I develop using numerous technologies, C and C++ is not one of them.
After doing some searching around the web, people are saying numerous things about what the problem could be. Here is an aggregate of some of the things that came back in my search:
So what do I do with this information?
In the first scenario, I’m pretty sure my dependencies were installed using the correct Amazon Linux flavor. I had mapped my project directory with Docker and installed my Node.js modules from within my Docker container. Life should be good because it worked the last time I tried about a year ago. In the second scenario, I have no idea how to check what version of libstdc++ I’m using in my Docker container or on AWS Lambda. I’m only hoping that the versions are matched up. In the third scenario, I did find this file in the container, so I was hoping that was enough.
If you can believe it, my Couchbase Node.js SDK was not compiled using the correct Amazon Linux flavor. When I downloaded this dependency from within my Docker container, it was using a pre-built version of the SDK which didn’t match what I truly needed. To resolve this, I needed to build the SDK from source.
To build my Node.js module from source so that the GLIBCXX errors went away, I needed to first download the build essentials to my Amazon Linux container. From within my container, I could execute the following:
yum -y install git
yum -y groupinstall "Development Tools"
The source to the Couchbase Node.js SDK was on GitHub so I needed to install the NPM dependency from GitHub rather than the standard NPM repository. For this to work git
needed to be available. Also, since the Amazon Linux instance is very basic, the appropriate compilers and build tools needed to be available. These tools were in the “Developer Tools” group.
After installing the proper OS tools, we could run the following:
npm install git+https://github.com/couchbase/couchnode.git#v2.6.2 --save
The above command will install the 2.6.2 release of the SDK, but it will also build it from source. By building it from source we can be sure that the proper operating system and architecture were used. After deploying this to AWS Lambda and AWS API Gateway, everything worked fine.
In case you’d like to create your own Docker image and save some of the hassle, you could use the following Dockerfile file:
FROM amazonlinux:1
RUN yum -y install tar
RUN yum -y install gzip
RUN yum -y install git
RUN yum -y groupinstall "Development Tools"
RUN curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.34.0/install.sh | bash
RUN /bin/bash -c "source /root/.nvm/nvm.sh; nvm install 8.10.0"
CMD /bin/bash -c "source /root/.nvm/nvm.sh; nvm use 8.10.0; bash"
The above blueprint will install the necessary applications, and choose the correct version of Node.js to match what AWS Lambda uses. Using Docker, you could run something like the following:
docker build -t custom-amazon-linux /path/to/Dockerfile
With the custom image available, you could deploy it as a container and map your project directory on the host, similar to how we did it in the previous tutorial.
You just saw how to solve GLIBCXX version errors related to the libstdc++ package, something that is a surprisingly common issue when it comes to function deployment on AWS Lambda. To be clear, these issues are not specific to the Couchbase Node.js SDK. These issues come up due to native dependencies being deployed after having been built using the incorrect operating system or architecture. Some Node.js and Python dependencies use native libraries and these native libraries need to match what Amazon Linux uses on AWS Lambda.
Original article source at: https://www.thepolyglotdeveloper.com/
1617016800
In this post, I will show you how to use Amazon S3 Object Lambda to resize images on the fly. The Serverless Framework will be used to define the Infrastructure as Code and to simplify the deployment. Sharp will be used to resize the images. Lambda will be written using the Node.js 14x Lambda runtime
One of the most common Lambda patterns is to transform data stored inside Amazon S3. Generally, a lambda function is invoked after a file has been uploaded. Lambda would retrieve that file, apply any needed transformation (e.g. converting type of file) and store the result in S3.
That pattern was working well, however, it would require some work done onto a file despite that being accessed in the future or not.
If you needed to convert a file on the fly you should have created a Lambda function, invoke it via Amazon API GW and wait for the lambda to perform the transformation.
AWS has recently introduced Amazon S3 Object Lambda in a good post by Danilo Poccia. S3 Object Lambda allows creating a Lambda directly connected to the S3 bucket (using S3 Access Points) that is automatically invoked when retrieving the object from S3!
That means that our application needs only to send an S3 Get Object request to retrieve the original or transformed data
Also, a very important peculiarity of using Amazon S3 Object Lambda it’s that the file you want to retrieve doesn’t need to exist on S3! We will make use of this for our scenario
_Note: High-level AWS CLI S3 commands (e.g.
_aws s3 cp_
) don’t currently support S3 Object Lambda, instead we need to use low-level S3 API commands (e.g. __aws s3api get-object)_
In his post, Danilo highlighted the most common use cases for Amazon S3 Object Lambda:
#aws-lambda #serverless #aws-s3 #aws