Hermann  Frami

Hermann Frami

1655426640

Serverless Plugin for Microservice Code Management and Deployment

Serverless M

Serverless M (or Serverless Modular) is a plugin for the serverless framework. This plugins helps you in managing multiple serverless projects with a single serverless.yml file. This plugin gives you a super charged CLI options that you can use to create new features, build them in a single file and deploy them all in parallel

splash.gif

Currently this plugin is tested for the below stack only

  • AWS
  • NodeJS λ
  • Rest API (You can use other events as well)

Prerequisites

Make sure you have the serverless CLI installed

# Install serverless globally
$ npm install serverless -g

Getting Started

To start the serverless modular project locally you can either start with es5 or es6 templates or add it as a plugin

ES6 Template install

# Step 1. Download the template
$ sls create --template-url https://github.com/aa2kb/serverless-modular/tree/master/template/modular-es6 --path myModularService

# Step 2. Change directory
$ cd myModularService

# Step 3. Create a package.json file
$ npm init

# Step 3. Install dependencies
$ npm i serverless-modular serverless-webpack webpack --save-dev

ES5 Template install

# Step 1. Download the template
$ sls create --template-url https://github.com/aa2kb/serverless-modular/tree/master/template/modular-es5 --path myModularService

# Step 2. Change directory
$ cd myModularService

# Step 3. Create a package.json file
$ npm init

# Step 3. Install dependencies
$ npm i serverless-modular --save-dev

If you dont want to use the templates above you can just add in your existing project

Adding it as plugin

plugins:
  - serverless-modular

Now you are all done to start building your serverless modular functions

API Reference

The serverless CLI can be accessed by

# Serverless Modular CLI
$ serverless modular

# shorthand
$ sls m

Serverless Modular CLI is based on 4 main commands

  • sls m init
  • sls m feature
  • sls m function
  • sls m build
  • sls m deploy

init command

sls m init

The serverless init command helps in creating a basic .gitignore that is useful for serverless modular.

The basic .gitignore for serverless modular looks like this

#node_modules
node_modules

#sm main functions
sm.functions.yml

#serverless file generated by build
src/**/serverless.yml

#main serverless directories generated for sls deploy
.serverless

#feature serverless directories generated sls deploy
src/**/.serverless

#serverless logs file generated for main sls deploy
.sm.log

#serverless logs file generated for feature sls deploy
src/**/.sm.log

#Webpack config copied in each feature
src/**/webpack.config.js

feature command

The feature command helps in building new features for your project

options (feature Command)

This command comes with three options

--name: Specify the name you want for your feature

--remove: set value to true if you want to remove the feature

--basePath: Specify the basepath you want for your feature, this base path should be unique for all features. helps in running offline with offline plugin and for API Gateway

optionsshortcutrequiredvaluesdefault value
--name-nstringN/A
--remove-rtrue, falsefalse
--basePath-pstringsame as name

Examples (feature Command)

Creating a basic feature

# Creating a jedi feature
$ sls m feature -n jedi

Creating a feature with different base path

# A feature with different base path
$ sls m feature -n jedi -p tatooine

Deleting a feature

# Anakin is going to delete the jedi feature
$ sls m feature -n jedi -r true

function command

The function command helps in adding new function to a feature

options (function Command)

This command comes with four options

--name: Specify the name you want for your function

--feature: Specify the name of the existing feature

--path: Specify the path for HTTP endpoint helps in running offline with offline plugin and for API Gateway

--method: Specify the path for HTTP method helps in running offline with offline plugin and for API Gateway

optionsshortcutrequiredvaluesdefault value
--name-nstringN/A
--feature-fstringN/A
--path-pstringsame as name
--method-mstring'GET'

Examples (function Command)

Creating a basic function

# Creating a cloak function for jedi feature
$ sls m function -n cloak -f jedi

Creating a basic function with different path and method

# Creating a cloak function for jedi feature with custom path and HTTP method
$ sls m function -n cloak -f jedi -p powers -m POST

build command

The build command helps in building the project for local or global scope

options (build Command)

This command comes with four options

--scope: Specify the scope of the build, use this with "--feature" tag

--feature: Specify the name of the existing feature you want to build

optionsshortcutrequiredvaluesdefault value
--scope-sstringlocal
--feature-fstringN/A

Saving build Config in serverless.yml

You can also save config in serverless.yml file

custom:
  smConfig:
    build:
      scope: local

Examples (build Command)

all feature build (local scope)

# Building all local features
$ sls m build

Single feature build (local scope)

# Building a single feature
$ sls m build -f jedi -s local

All features build global scope

# Building all features with global scope
$ sls m build -s global

deploy command

The deploy command helps in deploying serverless projects to AWS (it uses sls deploy command)

options (deploy Command)

This command comes with four options

--sm-parallel: Specify if you want to deploy parallel (will only run in parallel when doing multiple deployments)

--sm-scope: Specify if you want to deploy local features or global

--sm-features: Specify the local features you want to deploy (comma separated if multiple)

optionsshortcutrequiredvaluesdefault value
--sm-paralleltrue, falsetrue
--sm-scopelocal, globallocal
--sm-featuresstringN/A
--sm-ignore-buildstringfalse

Saving deploy Config in serverless.yml

You can also save config in serverless.yml file

custom:
  smConfig:
    deploy:
      scope: local
      parallel: true
      ignoreBuild: true

Examples (deploy Command)

Deploy all features locally

# deploy all local features
$ sls m deploy

Deploy all features globally

# deploy all global features
$ sls m deploy --sm-scope global

Deploy single feature

# deploy all global features
$ sls m deploy --sm-features jedi

Deploy Multiple features

# deploy all global features
$ sls m deploy --sm-features jedi,sith,dark_side

Deploy Multiple features in sequence

# deploy all global features
$ sls m deploy  --sm-features jedi,sith,dark_side --sm-parallel false

Author: aa2kb
Source Code: https://github.com/aa2kb/serverless-modular 
License: MIT license

#serverless #aws #node #lambda 

What is GEEK

Buddha Community

Serverless Plugin for Microservice Code Management and Deployment
Hermann  Frami

Hermann Frami

1655426640

Serverless Plugin for Microservice Code Management and Deployment

Serverless M

Serverless M (or Serverless Modular) is a plugin for the serverless framework. This plugins helps you in managing multiple serverless projects with a single serverless.yml file. This plugin gives you a super charged CLI options that you can use to create new features, build them in a single file and deploy them all in parallel

splash.gif

Currently this plugin is tested for the below stack only

  • AWS
  • NodeJS λ
  • Rest API (You can use other events as well)

Prerequisites

Make sure you have the serverless CLI installed

# Install serverless globally
$ npm install serverless -g

Getting Started

To start the serverless modular project locally you can either start with es5 or es6 templates or add it as a plugin

ES6 Template install

# Step 1. Download the template
$ sls create --template-url https://github.com/aa2kb/serverless-modular/tree/master/template/modular-es6 --path myModularService

# Step 2. Change directory
$ cd myModularService

# Step 3. Create a package.json file
$ npm init

# Step 3. Install dependencies
$ npm i serverless-modular serverless-webpack webpack --save-dev

ES5 Template install

# Step 1. Download the template
$ sls create --template-url https://github.com/aa2kb/serverless-modular/tree/master/template/modular-es5 --path myModularService

# Step 2. Change directory
$ cd myModularService

# Step 3. Create a package.json file
$ npm init

# Step 3. Install dependencies
$ npm i serverless-modular --save-dev

If you dont want to use the templates above you can just add in your existing project

Adding it as plugin

plugins:
  - serverless-modular

Now you are all done to start building your serverless modular functions

API Reference

The serverless CLI can be accessed by

# Serverless Modular CLI
$ serverless modular

# shorthand
$ sls m

Serverless Modular CLI is based on 4 main commands

  • sls m init
  • sls m feature
  • sls m function
  • sls m build
  • sls m deploy

init command

sls m init

The serverless init command helps in creating a basic .gitignore that is useful for serverless modular.

The basic .gitignore for serverless modular looks like this

#node_modules
node_modules

#sm main functions
sm.functions.yml

#serverless file generated by build
src/**/serverless.yml

#main serverless directories generated for sls deploy
.serverless

#feature serverless directories generated sls deploy
src/**/.serverless

#serverless logs file generated for main sls deploy
.sm.log

#serverless logs file generated for feature sls deploy
src/**/.sm.log

#Webpack config copied in each feature
src/**/webpack.config.js

feature command

The feature command helps in building new features for your project

options (feature Command)

This command comes with three options

--name: Specify the name you want for your feature

--remove: set value to true if you want to remove the feature

--basePath: Specify the basepath you want for your feature, this base path should be unique for all features. helps in running offline with offline plugin and for API Gateway

optionsshortcutrequiredvaluesdefault value
--name-nstringN/A
--remove-rtrue, falsefalse
--basePath-pstringsame as name

Examples (feature Command)

Creating a basic feature

# Creating a jedi feature
$ sls m feature -n jedi

Creating a feature with different base path

# A feature with different base path
$ sls m feature -n jedi -p tatooine

Deleting a feature

# Anakin is going to delete the jedi feature
$ sls m feature -n jedi -r true

function command

The function command helps in adding new function to a feature

options (function Command)

This command comes with four options

--name: Specify the name you want for your function

--feature: Specify the name of the existing feature

--path: Specify the path for HTTP endpoint helps in running offline with offline plugin and for API Gateway

--method: Specify the path for HTTP method helps in running offline with offline plugin and for API Gateway

optionsshortcutrequiredvaluesdefault value
--name-nstringN/A
--feature-fstringN/A
--path-pstringsame as name
--method-mstring'GET'

Examples (function Command)

Creating a basic function

# Creating a cloak function for jedi feature
$ sls m function -n cloak -f jedi

Creating a basic function with different path and method

# Creating a cloak function for jedi feature with custom path and HTTP method
$ sls m function -n cloak -f jedi -p powers -m POST

build command

The build command helps in building the project for local or global scope

options (build Command)

This command comes with four options

--scope: Specify the scope of the build, use this with "--feature" tag

--feature: Specify the name of the existing feature you want to build

optionsshortcutrequiredvaluesdefault value
--scope-sstringlocal
--feature-fstringN/A

Saving build Config in serverless.yml

You can also save config in serverless.yml file

custom:
  smConfig:
    build:
      scope: local

Examples (build Command)

all feature build (local scope)

# Building all local features
$ sls m build

Single feature build (local scope)

# Building a single feature
$ sls m build -f jedi -s local

All features build global scope

# Building all features with global scope
$ sls m build -s global

deploy command

The deploy command helps in deploying serverless projects to AWS (it uses sls deploy command)

options (deploy Command)

This command comes with four options

--sm-parallel: Specify if you want to deploy parallel (will only run in parallel when doing multiple deployments)

--sm-scope: Specify if you want to deploy local features or global

--sm-features: Specify the local features you want to deploy (comma separated if multiple)

optionsshortcutrequiredvaluesdefault value
--sm-paralleltrue, falsetrue
--sm-scopelocal, globallocal
--sm-featuresstringN/A
--sm-ignore-buildstringfalse

Saving deploy Config in serverless.yml

You can also save config in serverless.yml file

custom:
  smConfig:
    deploy:
      scope: local
      parallel: true
      ignoreBuild: true

Examples (deploy Command)

Deploy all features locally

# deploy all local features
$ sls m deploy

Deploy all features globally

# deploy all global features
$ sls m deploy --sm-scope global

Deploy single feature

# deploy all global features
$ sls m deploy --sm-features jedi

Deploy Multiple features

# deploy all global features
$ sls m deploy --sm-features jedi,sith,dark_side

Deploy Multiple features in sequence

# deploy all global features
$ sls m deploy  --sm-features jedi,sith,dark_side --sm-parallel false

Author: aa2kb
Source Code: https://github.com/aa2kb/serverless-modular 
License: MIT license

#serverless #aws #node #lambda 

Serverless Vs Microservices Architecture - A Deep Dive

Companies need to be thinking long-term before even starting a software development project. These needs are solved at the level of architecture: business owners want to assure agility, scalability, and performance.

The top contenders for scalable solutions are serverless and microservices. Both architectures prioritize security but approach it in their own ways. Let’s take a look at how businesses can benefit from the adoption of serverless architecture vs microservices, examine their differences, advantages, and use cases.

#serverless #microservices #architecture #software-architecture #serverless-architecture #microservice-architecture #serverless-vs-microservices #hackernoon-top-story

Hermann  Frami

Hermann Frami

1654345320

Serverless Fast Deploy Plugin: Lightening Fast Serverless Deployments

Serverless Fast Deploy Plugin  

Fast Serverless deployments for large packages

Requirements:

  • Serverless v1.12.x or higher.
  • AWS provider

How it works

I found that while working with Python libraries such Numpy and Pandas, my deploys became very slow and expensive (I work off a mobile data plan) due to the increased package size. This plugin deploys a specialized Lambda always you to only deploy the files that are most likely to change. It does this by merging the incoming files with the latest existing package on S3. So now when I deploy a change, I am sending a few KB across the wire each time, not 50 MB.

Caveats

A note about merging the update package with the base package

y first attempt was to just use the latest existing deployment package on S3, unpack that and create a new package with the update files. This was a bit "slow", so now I create a base package which is the full previous deployment package without the files described by the custom.fastDeploy.include property. This means that I can simply append the new files, resulting in an even faster deploy. The unfortunately side effect being that if you change the custom.fastDeploy.include property, you need to do a full deployment before doing your next FastDeploy.

The creation of the base deployment package also means that the first FastDeploy will be slightly slower than subsequent deployments.

Custom deployment bucket

At the moment this plugin bypasses all of the standard deployment lifecycle stages, so I am not yet able to get hold of the auto generated deployment bucket. As such this plugin only works if you have created a custom deployment bucket and configured it via the provider.deploymentBucket property.

IAM Role

The FastDeploy Lambda requires the following permissions on the deployment bucket. Either this can be added to the services default role, or you can create a new role and configure it via the custom.fastDeploy.role property.

Updates to CloudFormation configuration requires a full deployment

Much like Serverless's function deployment feature, any updates to the CloudFormation stack requires a full deployment.

- Effect: Allow
  Action:
    - s3:GetObject
    - s3:PutObject
  Resource: arn:aws:s3:::aronim-serverless/*
- Effect: Allow
  Action:
    - s3:ListBucket
  Resource: arn:aws:s3:::aronim-serverless     

Setup

Install via npm in the root of your Serverless service:

npm install serverless-plugin-fastdeploy --save-dev
  • Add the plugin to the plugins array in your Serverless serverless.yml:
plugins:
  - serverless-plugin-fastdeploy

Run

sls fastdeploy

Configuration

The custom.fastDeploy.include property describes which files to include in the update package, and exclude from the base package. This can be an array if you are just working in single module project, or an object if you are working with a multi-module project.

Available custom properties:


custom:
  fastDeploy:
    memorySize: 512    # Optional. Default: 512MB
    timeout: 30        # Optional. Default: 30sec
    include:           # Required. No Default
      - src/*.js       # Example
    role:              # Optional. Uses service default role if one is provided
      - FastDeployRole # Example
service: ServerlessFastDeployExample

plugins:
  - serverless-plugin-fastdeploy

provider:
  ...
  role: DefaultRole
  deploymentBucket: aronim-serverless

custom:
  fastDeploy:
    include:
      - package_one/**
      - package_two/**

######      
# OR #      
###### 
 
custom:
  fastDeploy:
    include:
      ".": service_one/**
      "../../modules/module-two": module_two/**     

resources:
  Resources:
    DefaultRole:
      Type: AWS::IAM::Role
      Properties:
        Path: /
        RoleName: ${self:service}-${self:provider.stage}
        AssumeRolePolicyDocument:
          Version: "2012-10-17"
          Statement:
            - Effect: Allow
              Principal:
                Service:
                  - lambda.amazonaws.com
              Action: sts:AssumeRole
        Policies:
          - PolicyName: ${self:service}-${self:provider.stage}
            PolicyDocument:
              Version: "2012-10-17"
              Statement:
                - Effect: Allow
                  Action:
                    - logs:CreateLogGroup
                    - logs:CreateLogStream
                    - logs:PutLogEvents
                  Resource: arn:aws:logs:${self:provider.region}:*:log-group:/aws/lambda/*:*:*
                - Effect: Allow
                  Action:
                    - s3:GetObject
                    - s3:PutObject
                  Resource: arn:aws:s3:::aronim-serverless/*
                - Effect: Allow
                  Action:
                    - s3:ListBucket
                  Resource: arn:aws:s3:::aronim-serverless     

Cost

Since we are deploying an additional Lambda, there are some neglible cost implications. The default memory allocated to the FastDeploy Lambda is 512MB, but this can be increased or decreased using the custom.fastDeploy.memory property.

Acknowledgements

A big thank you to FidelLimited, I blatently plagiarized their WarmUp plugin for the basis of the FastDeploy Lambda :-) As they say "Mimicry is the highest form of flattery".

Contribute

Help us making this plugin better and future proof.

  • Clone the code
  • Install the dependencies with npm install
  • Create a feature branch git checkout -b new_feature
  • Lint with standard npm run lint

Author: Aronim
Source Code: https://github.com/aronim/serverless-plugin-fastdeploy 
License: MIT license

#serverless #plugin #deployment 

Maud  Rosenbaum

Maud Rosenbaum

1601096160

Microservice and Serverless With Micronaut + GraalVM

Micronaut® framework is creating a buzz around cloud-native (microservice, serverless ) application development due to its enriched features and optimizations based out of modern Polyglot JVM — GraalVMoptimizers.

For GraalVM ** optimizers go thought Part1 of this series GraalVM — Byte Code to Bit Code.**

Why Micronaut® and what makes it the next level stuff as a framework for Cloud-Native development :

  • Natively Cloud Native
  • Micronaut®’s cloud support is built right in, including support for common discovery services, distributed tracing tools, cloud runtimes and leading vendors ( AWS, AZURE, GCP).
  • Ready for Serverless Development
  • Micronaut® low overhead compile-time DI and AOP make it perfect for writing functions for serverless environments like AWS Lambda, Azure Functions etc.
  • Fast Start Time — Low Memory Consumption
  • Reflection-based IoC framework load and cache reflection data for every single field, method, and constructor in your code. Whereas with Micronaut®, your application startup time and memory consumption are not bound to the size of your codebase. Micronaut® build around GraalVM. Micronaut features a dependency injection and aspect-oriented programming runtime that uses no reflection. This makes it easier for Micronaut applications to run on GraalVM.
  • Micronaut apps start in a tenth of millisecond with GraalVM!
  • Built with Non-Blocking Http Server on Netty
  • With a smooth learning curve, Micronaut’s HTTP server makes it as easy as possible to expose APIs that can be consumed by HTTP clients.
  • Design for Building Resilient Microservices
  • Distributed environments require planning for failure. Micronaut’s built-in support for retry, circuit breaker, and fallbacks help you plan.
  • Faster Data Layer Configuration
  • Micronaut provides sensible defaults that automatically configure your favorite data access toolkit and APIs to make it easy to write your own integrations. Supports like Caching, SQL, NoSQL, and different DB vendor products are readily available.
  • Fast and Easy Testing
  • Easily spin up servers and clients in your unit tests, and run them instantaneously with mostly auto-generated test stuff for main source code.
  • Reloading (or “hot-loading”)
  • Out of box excellent support of hot reloading, restart, and reinitialization with build-in JVM agent support rather than relying on classloader reloading or proprietary JVM agents like JRebel etc. It is an important feature to make the development cycle efficient and fast.
  • Refreshable feature
  • @Refreshable is another interesting scope offered by Micronaut. You can refresh the state of a bean by calling the HTTP endpoint /refresh or by publishing RefreshEvent to the application context.

How to Get Started Along Microservice With Micronaut

Create an app using the Micronaut Command Line Interface.

Installing Micronaut® manually comes with a super-rich & intuitive CLI that we will be using for the rest of the project lifecycle ( its power-pack CLI features add to developers productivity). We will be doing 3case studies with Micronaut® to compare and benchmark its potential relative to today’s Cloud-Native de-facto framework — SpringBoot.

_Case A : _ Micronaut ® _ app on GraalVM optimised JIT_

_Case B : _ Micronaut ® _ app on GraalVM “Native-Image”_

Case C : SpringBoot on openJDK HotSpotVM JIT — conventional way

Let’s get to the suspense straight-in with below depiction of different benchmarking results :

Benchmarking Micronaut

Micronaut application benchmarking

#kubernetes #serverless #microservice architecture #microservice #aws lambda #cloud native applications #microservice best practices #cloud application #serverless adoption #azure function

Tyrique  Littel

Tyrique Littel

1604008800

Static Code Analysis: What It Is? How to Use It?

Static code analysis refers to the technique of approximating the runtime behavior of a program. In other words, it is the process of predicting the output of a program without actually executing it.

Lately, however, the term “Static Code Analysis” is more commonly used to refer to one of the applications of this technique rather than the technique itself — program comprehension — understanding the program and detecting issues in it (anything from syntax errors to type mismatches, performance hogs likely bugs, security loopholes, etc.). This is the usage we’d be referring to throughout this post.

“The refinement of techniques for the prompt discovery of error serves as well as any other as a hallmark of what we mean by science.”

  • J. Robert Oppenheimer

Outline

We cover a lot of ground in this post. The aim is to build an understanding of static code analysis and to equip you with the basic theory, and the right tools so that you can write analyzers on your own.

We start our journey with laying down the essential parts of the pipeline which a compiler follows to understand what a piece of code does. We learn where to tap points in this pipeline to plug in our analyzers and extract meaningful information. In the latter half, we get our feet wet, and write four such static analyzers, completely from scratch, in Python.

Note that although the ideas here are discussed in light of Python, static code analyzers across all programming languages are carved out along similar lines. We chose Python because of the availability of an easy to use ast module, and wide adoption of the language itself.

How does it all work?

Before a computer can finally “understand” and execute a piece of code, it goes through a series of complicated transformations:

static analysis workflow

As you can see in the diagram (go ahead, zoom it!), the static analyzers feed on the output of these stages. To be able to better understand the static analysis techniques, let’s look at each of these steps in some more detail:

Scanning

The first thing that a compiler does when trying to understand a piece of code is to break it down into smaller chunks, also known as tokens. Tokens are akin to what words are in a language.

A token might consist of either a single character, like (, or literals (like integers, strings, e.g., 7Bob, etc.), or reserved keywords of that language (e.g, def in Python). Characters which do not contribute towards the semantics of a program, like trailing whitespace, comments, etc. are often discarded by the scanner.

Python provides the tokenize module in its standard library to let you play around with tokens:

Python

1

import io

2

import tokenize

3

4

code = b"color = input('Enter your favourite color: ')"

5

6

for token in tokenize.tokenize(io.BytesIO(code).readline):

7

    print(token)

Python

1

TokenInfo(type=62 (ENCODING),  string='utf-8')

2

TokenInfo(type=1  (NAME),      string='color')

3

TokenInfo(type=54 (OP),        string='=')

4

TokenInfo(type=1  (NAME),      string='input')

5

TokenInfo(type=54 (OP),        string='(')

6

TokenInfo(type=3  (STRING),    string="'Enter your favourite color: '")

7

TokenInfo(type=54 (OP),        string=')')

8

TokenInfo(type=4  (NEWLINE),   string='')

9

TokenInfo(type=0  (ENDMARKER), string='')

(Note that for the sake of readability, I’ve omitted a few columns from the result above — metadata like starting index, ending index, a copy of the line on which a token occurs, etc.)

#code quality #code review #static analysis #static code analysis #code analysis #static analysis tools #code review tips #static code analyzer #static code analysis tool #static analyzer