Hermann  Frami

Hermann Frami

1656614460

Serverless Plugin for S3 Sync

⚑️ Serverless Plugin for S3 Sync   

With this plugin for serverless, you can sync local folders to S3 buckets after your service is deployed.

Usage

Add the NPM package to your project:

# Via yarn
$ yarn add serverless-s3bucket-sync

# Via npm
$ npm install serverless-s3bucket-sync

Add the plugin to your serverless.yml:

plugins:
  - serverless-s3bucket-sync

Configuration

Configure S3 Bucket syncing Auto Scaling in serverless.yml with references to your local folder and the name of the S3 bucket.

custom:
  s3-sync:
    - folder: relative/folder
      bucket: bucket-name

That's it! With the next deployment, serverless will sync your local folder relative/folder with the S3 bucket named bucket-name.

Sync

You can use sls sync to synchornize all buckets without deploying your serverless stack.

Contribution

You are welcome to contribute to this project! 😘

To make sure you have a pleasant experience, please read the code of conduct. It outlines core values and beliefs and will make working together a happier experience.

Author: sbstjn
Source Code: https://github.com/sbstjn/serverless-s3bucket-sync 
License: MIT license

#serverless #s3 #sync #aws 

What is GEEK

Buddha Community

Serverless Plugin for S3 Sync
Hermann  Frami

Hermann Frami

1656614460

Serverless Plugin for S3 Sync

⚑️ Serverless Plugin for S3 Sync   

With this plugin for serverless, you can sync local folders to S3 buckets after your service is deployed.

Usage

Add the NPM package to your project:

# Via yarn
$ yarn add serverless-s3bucket-sync

# Via npm
$ npm install serverless-s3bucket-sync

Add the plugin to your serverless.yml:

plugins:
  - serverless-s3bucket-sync

Configuration

Configure S3 Bucket syncing Auto Scaling in serverless.yml with references to your local folder and the name of the S3 bucket.

custom:
  s3-sync:
    - folder: relative/folder
      bucket: bucket-name

That's it! With the next deployment, serverless will sync your local folder relative/folder with the S3 bucket named bucket-name.

Sync

You can use sls sync to synchornize all buckets without deploying your serverless stack.

Contribution

You are welcome to contribute to this project! 😘

To make sure you have a pleasant experience, please read the code of conduct. It outlines core values and beliefs and will make working together a happier experience.

Author: sbstjn
Source Code: https://github.com/sbstjn/serverless-s3bucket-sync 
License: MIT license

#serverless #s3 #sync #aws 

Hermann  Frami

Hermann Frami

1655426640

Serverless Plugin for Microservice Code Management and Deployment

Serverless M

Serverless M (or Serverless Modular) is a plugin for the serverless framework. This plugins helps you in managing multiple serverless projects with a single serverless.yml file. This plugin gives you a super charged CLI options that you can use to create new features, build them in a single file and deploy them all in parallel

splash.gif

Currently this plugin is tested for the below stack only

  • AWS
  • NodeJS Ξ»
  • Rest API (You can use other events as well)

Prerequisites

Make sure you have the serverless CLI installed

# Install serverless globally
$ npm install serverless -g

Getting Started

To start the serverless modular project locally you can either start with es5 or es6 templates or add it as a plugin

ES6 Template install

# Step 1. Download the template
$ sls create --template-url https://github.com/aa2kb/serverless-modular/tree/master/template/modular-es6 --path myModularService

# Step 2. Change directory
$ cd myModularService

# Step 3. Create a package.json file
$ npm init

# Step 3. Install dependencies
$ npm i serverless-modular serverless-webpack webpack --save-dev

ES5 Template install

# Step 1. Download the template
$ sls create --template-url https://github.com/aa2kb/serverless-modular/tree/master/template/modular-es5 --path myModularService

# Step 2. Change directory
$ cd myModularService

# Step 3. Create a package.json file
$ npm init

# Step 3. Install dependencies
$ npm i serverless-modular --save-dev

If you dont want to use the templates above you can just add in your existing project

Adding it as plugin

plugins:
  - serverless-modular

Now you are all done to start building your serverless modular functions

API Reference

The serverless CLI can be accessed by

# Serverless Modular CLI
$ serverless modular

# shorthand
$ sls m

Serverless Modular CLI is based on 4 main commands

  • sls m init
  • sls m feature
  • sls m function
  • sls m build
  • sls m deploy

init command

sls m init

The serverless init command helps in creating a basic .gitignore that is useful for serverless modular.

The basic .gitignore for serverless modular looks like this

#node_modules
node_modules

#sm main functions
sm.functions.yml

#serverless file generated by build
src/**/serverless.yml

#main serverless directories generated for sls deploy
.serverless

#feature serverless directories generated sls deploy
src/**/.serverless

#serverless logs file generated for main sls deploy
.sm.log

#serverless logs file generated for feature sls deploy
src/**/.sm.log

#Webpack config copied in each feature
src/**/webpack.config.js

feature command

The feature command helps in building new features for your project

options (feature Command)

This command comes with three options

--name: Specify the name you want for your feature

--remove: set value to true if you want to remove the feature

--basePath: Specify the basepath you want for your feature, this base path should be unique for all features. helps in running offline with offline plugin and for API Gateway

optionsshortcutrequiredvaluesdefault value
--name-nβœ…stringN/A
--remove-r❎true, falsefalse
--basePath-p❎stringsame as name

Examples (feature Command)

Creating a basic feature

# Creating a jedi feature
$ sls m feature -n jedi

Creating a feature with different base path

# A feature with different base path
$ sls m feature -n jedi -p tatooine

Deleting a feature

# Anakin is going to delete the jedi feature
$ sls m feature -n jedi -r true

function command

The function command helps in adding new function to a feature

options (function Command)

This command comes with four options

--name: Specify the name you want for your function

--feature: Specify the name of the existing feature

--path: Specify the path for HTTP endpoint helps in running offline with offline plugin and for API Gateway

--method: Specify the path for HTTP method helps in running offline with offline plugin and for API Gateway

optionsshortcutrequiredvaluesdefault value
--name-nβœ…stringN/A
--feature-fβœ…stringN/A
--path-p❎stringsame as name
--method-m❎string'GET'

Examples (function Command)

Creating a basic function

# Creating a cloak function for jedi feature
$ sls m function -n cloak -f jedi

Creating a basic function with different path and method

# Creating a cloak function for jedi feature with custom path and HTTP method
$ sls m function -n cloak -f jedi -p powers -m POST

build command

The build command helps in building the project for local or global scope

options (build Command)

This command comes with four options

--scope: Specify the scope of the build, use this with "--feature" tag

--feature: Specify the name of the existing feature you want to build

optionsshortcutrequiredvaluesdefault value
--scope-s❎stringlocal
--feature-f❎stringN/A

Saving build Config in serverless.yml

You can also save config in serverless.yml file

custom:
  smConfig:
    build:
      scope: local

Examples (build Command)

all feature build (local scope)

# Building all local features
$ sls m build

Single feature build (local scope)

# Building a single feature
$ sls m build -f jedi -s local

All features build global scope

# Building all features with global scope
$ sls m build -s global

deploy command

The deploy command helps in deploying serverless projects to AWS (it uses sls deploy command)

options (deploy Command)

This command comes with four options

--sm-parallel: Specify if you want to deploy parallel (will only run in parallel when doing multiple deployments)

--sm-scope: Specify if you want to deploy local features or global

--sm-features: Specify the local features you want to deploy (comma separated if multiple)

optionsshortcutrequiredvaluesdefault value
--sm-parallel❎❎true, falsetrue
--sm-scope❎❎local, globallocal
--sm-features❎❎stringN/A
--sm-ignore-build❎❎stringfalse

Saving deploy Config in serverless.yml

You can also save config in serverless.yml file

custom:
  smConfig:
    deploy:
      scope: local
      parallel: true
      ignoreBuild: true

Examples (deploy Command)

Deploy all features locally

# deploy all local features
$ sls m deploy

Deploy all features globally

# deploy all global features
$ sls m deploy --sm-scope global

Deploy single feature

# deploy all global features
$ sls m deploy --sm-features jedi

Deploy Multiple features

# deploy all global features
$ sls m deploy --sm-features jedi,sith,dark_side

Deploy Multiple features in sequence

# deploy all global features
$ sls m deploy  --sm-features jedi,sith,dark_side --sm-parallel false

Author: aa2kb
Source Code: https://github.com/aa2kb/serverless-modular 
License: MIT license

#serverless #aws #node #lambda 

Hermann  Frami

Hermann Frami

1656599580

Serverless S3 Remover

serverless-s3-remover

plugin for serverless to make buckets empty before remove

Usage

Run next command.

$ npm install serverless-s3-remover

Add to your serverless.yml

plugins:
  - serverless-s3-remover

custom:
  remover:
     buckets:
       - my-bucket-1
       - my-bucket-2

You can specify any number of buckets that you want.

Now you can make all buckets empty by running:

$ sls s3remove

When removing

When removing serverless stack, this plugin automatically make buckets empty before removing stack.

$ sls remove

Using Prompt

You can use prompt before deleting bucket.

custom:
  remover:
    prompt: true # default value is `false`
    buckets:
      - remover-bucket-a
      - remover-bucket-b

terminal.png

Populating the configuration object before using it

custom:
  boolean:
    true: true
    false: false
  remover:
    prompt: ${self:custom.boolean.${opt:s3-remover-prompt, 'true'}}

I can use the command line argument --s3-remover-prompt false to disable the prompt feature.

Author: Sinofseven
Source Code: https://github.com/sinofseven/serverless-s3-remover 
License: MIT license

#serverless #s3 #remove #plugin 

Hermann  Frami

Hermann Frami

1656577200

Serverless S3 Deploy

serverless-s3-deploy

Plugin for serverless to deploy files to a variety of S3 Buckets

Note: This project is currently not maintained.

Installation

 npm install --save-dev serverless-s3-deploy

Usage

Add to your serverless.yml:

  plugins:
    - serverless-s3-deploy

  custom:
    assets:
      targets:
       - bucket: my-bucket
         files:
          - source: ../assets/
            globs: '**/*.css'
          - source: ../app/
            globs:
              - '**/*.js'
              - '**/*.map'
       - bucket: my-other-bucket
         empty: true
         prefix: subdir
         files:
          - source: ../email-templates/
            globs: '**/*.html'

You can specify any number of targets that you want. Each target has a bucket and a prefix.

bucket is either the name of your S3 bucket or a reference to a CloudFormation resources created in the same serverless configuration file. See below for additional details.

You can specify source relative to the current directory.

Each source has its own list of globs, which can be either a single glob, or a list of globs.

Setting empty to true will delete all files inside the bucket before uploading the new content to S3 bucket. The prefix value is respected and files outside will not be deleted.

Now you can upload all of these assets to your bucket by running:

$ sls s3deploy

If you have defined multiple buckets, you can limit your deployment to a single bucket with the --bucket option:

$ sls s3deploy --bucket my-bucket

ACL

You can optionally specificy an ACL for the files uploaded on a per target basis:

  custom:
    assets:
      targets:
        - bucket: my-bucket
          acl: private
          files:

The default value is private. Options are defined here.

Content Type

The appropriate Content Type for each file will attempt to be determined using mime-types. If one can't be determined, a default fallback of 'application/octet-stream' will be used.

You can override this fallback per-source by setting defaultContentType.

  custom:
    assets:
      targets:
        - bucket: my-bucket
          files:
            - source: html/
              defaultContentType: text/html
              ...

Other Headers

Additional headers can be included per target by providing a headers object.

See http://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectPUT.html for more details.

  custom:
    assets:
      targets:
        - bucket: my-bucket
          files:
            - source: html/
              headers:
                CacheControl: max-age=31104000 # 1 year

Resolving References

A common use case is to create the S3 buckets in the resources section of your serverless configuration and then reference it in your S3 plugin settings:

  custom:
    assets:
      targets:
        - bucket:
            Ref: MyBucket
          files:
            - source: html/

  resources:
    # AWS CloudFormation Template
    Resources:
      MyBucket:
        Type: AWS::S3::Bucket
        Properties:
          AccessControl: PublicRead
          WebsiteConfiguration:
            IndexDocument: index.html
            ErrorDocument: index.html

You can disable the resolving with the following flag:

  custom:
    assets:
      resolveReferences: false

Auto-deploy

If you want s3deploy to run automatically after a deploy, set the auto flag:

  custom:
    assets:
      auto: true

IAM Configuration

You're going to need an IAM policy that supports this deployment. This might be a good starting point:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket"
            ],
            "Resource": [
                "arn:aws:s3:::${bucket}"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:PutObjectAcl",
                "s3:GetObject",
                "s3:DeleteObject"
            ],
            "Resource": [
                "arn:aws:s3:::${bucket}/*"
            ]
        }
    ]
}

Upload concurrency

If you want to tweak the upload concurrency, change uploadConcurrency config:

config:
  assets:
    # defaults to 3
    uploadConcurrency: 1

Verbosity

Verbosity cloud be enabled using either of these methods:

Configuration:

  custom:
    assets:
      verbose: true

Cli:

  sls s3deploy -v

Author: Funkybob
Source Code: https://github.com/funkybob/serverless-s3-deploy 
License: MIT license

#serverless #deploy #s3 #plugin 

Hermann  Frami

Hermann Frami

1653578100

Serverless Plugin Datadog

Datadog recommends the Serverless Framework Plugin for developers using the Serverless Framework to deploy their serverless applications. The plugin automatically enables instrumentation for applications to collect metrics, traces, and logs by:

  • Installing the Datadog Lambda library to your Lambda functions as a Lambda layer.
  • Installing the Datadog Lambda Extension to your Lambda functions as a Lambda layer (addExtension) or subscribing the Datadog Forwarder to your Lambda functions' log groups (forwarderArn).
  • Making the required configuration changes, such as adding environment variables or additional tracing layers, to your Lambda functions.

Getting started

To quickly get started, follow the installation instructions for Python, Node.js, Ruby, Java, Go, or .NET and view your function's enhanced metrics, traces, and logs in Datadog.

After installation is complete, configure the advanced options to suit your monitoring needs.

Upgrade

Each version of the plugin is published with a specific set of versions of the Datadog Lambda layers. To pick up new features and bug fixes provided by the latest versions of Datadog Lambda layers, upgrade the serverless framework plugin. Test the new version before applying it on your production applications.

Configuration parameters

To further configure your plugin, use the following custom parameters in your serverless.yml:

ParameterDescription
siteSet which Datadog site to send data to, such as datadoghq.com (default), datadoghq.eu, us3.datadoghq.com, us5.datadoghq.com, or ddog-gov.com. This parameter is required when collecting telemtry using the Datadog Lambda Extension.
apiKey[Datadog API key][7]. This parameter is required when collecting telemetry using the Datadog Lambda Extension. Alternatively, you can also set the DATADOG_API_KEY environment variable in your deployment environment.
appKeyDatadog app key. Only needed when the monitors field is defined. Alternatively, you can also set the DATADOG_APP_KEY environment variable in your deployment environment.
apiKeySecretArnAn alternative to using the apiKey field. The ARN of the secret that is storing the Datadog API key in AWS Secrets Manager. Remember to add the secretsmanager:GetSecretValue permission to the Lambda execution role.
apiKMSKeyAn alternative to using the apiKey field. Datadog API key encrypted using KMS. Remember to add the kms:Decrypt permission to the Lambda execution role.
envWhen set along with addExtension, a DD_ENV environment variable is added to all Lambda functions with the provided value. Otherwise, an env tag is added to all Lambda functions with the provided value. Defaults to the stage value of the serverless deployment.
serviceWhen set along with addExtension, a DD_SERVICE environment variable is added to all Lambda functions with the provided value. Otherwise, a service tag is added to all Lambda functions with the provided value. Defaults to the service value of the serverless project.
versionWhen set along with addExtension, a DD_VERSION environment variable is added to all Lambda functions with the provided value. When set along with forwarderArn, a version tag is added to all Lambda functions with the provided value.
tagsA comma separated list of key:value pairs as a single string. When set along with extensionLayerVersion, a DD_TAGS environment variable is added to all Lambda functions with the provided value. When set along with forwarderArn, the plugin parses the string and sets each key:value pair as a tag on all Lambda functions.
enableXrayTracingSet true to enable X-Ray tracing on the Lambda functions and API Gateway integrations. Defaults to false.
enableDDTracingEnable Datadog tracing on the Lambda function. Defaults to true.
enableDDLogsEnable Datadog log collection using the Lambda Extension. Defaults to true. Note: This setting has no effect on logs sent by the Datadog Forwarder.
monitorsWhen defined, the Datadog plugin configures monitors for the deployed function. Requires setting DATADOG_API_KEY and DATADOG_APP_KEY in your environment. To learn how to define monitors, see To Enable and Configure a Recommended Serverless Monitor.
captureLambdaPayload[Captures incoming and outgoing AWS Lambda payloads][17] in the Datadog APM spans for Lambda invocations. Defaults to false.
enableSourceCodeIntegrationEnable [Datadog source code integration][18] for the function. Defaults to true.
subscribeToApiGatewayLogsEnable automatic subscription of the Datadog Forwarder to API Gateway log groups. Requires setting forwarderArn. Defaults to true.
subscribeToHttpApiLogsEnable automatic subscription of the Datadog Forwarder to HTTP API log groups. Requires setting forwarderArn. Defaults to true.
subscribeToWebsocketLogsEnable automatic subscription of the Datadog Forwarder to WebSocket log groups. Requires setting forwarderArn. Defaults to true.
forwarderArnThe ARN of the Datadog Forwarder to be subscribed to the Lambda or API Gateway log groups.
addLayersWhether to install the Datadog Lambda library as a layer. Defaults to true. Set to false when you plan to package the Datadog Lambda library to your function's deployment package on your own so that you can install a specific version of the Datadog Lambda library ([Python][8] or [Node.js][9]).
addExtensionWhether to install the Datadog Lambda Extension as a layer. Defaults to true. When enabled, it's required to set the apiKey and site.
excludeWhen set, this plugin ignores all specified functions. Use this parameter if you have any functions that should not include Datadog functionality. Defaults to [].
enabledWhen set to false, the Datadog plugin stays inactive. Defaults to true. You can control this option using an environment variable. For example, use enabled: ${strToBool(${env:DD_PLUGIN_ENABLED, true})} to activate/deactivate the plugin during deployment. Alternatively, you can also use the value passed in through --stage to control this optionβ€”see example.
customHandlerWhen set, the specified handler is set as the handler for all the functions.
failOnErrorWhen set, this plugin throws an error if any custom Datadog monitors fail to create or update. This occurs after deploy, but will cause the result of serverless deploy to return a nonzero exit code (to fail user CI). Defaults to false.
integrationTestingSet true when running integration tests. This bypasses the validation of the Forwarder ARN and the addition of Datadog Monitor output links. Defaults to false.
logLevelThe log level, set to DEBUG for extended logging.

To use any of these parameters, add a custom > datadog section to your serverless.yml similar to this example:

custom:
  datadog:
    apiKeySecretArn: "{Datadog_API_Key_Secret_ARN}"
    enableXrayTracing: false
    enableDDTracing: true
    enableDDLogs: true
    subscribeToAccessLogs: true
    forwarderArn: arn:aws:lambda:us-east-1:000000000000:function:datadog-forwarder
    exclude:
      - dd-excluded-function

Webpack

If you are using a bundler, such as webpack, see Serverless Tracing and Webpack.

TypeScript

You may encounter the error of missing type definitions. To resolve the error, add datadog-lambda-js and dd-trace to the devDependencies list of your project's package.json.

If you are using serverless-typescript, make sure that serverless-datadog is above the serverless-typescript entry in your serverless.yml. The plugin will automatically detect .ts files.

plugins:
  - serverless-plugin-datadog
  - serverless-typescript

Disable Plugin for Particular Environment

If you'd like to turn off the plugin based on the environment (passed via --stage), you can use something similar to the example below.

provider:
  stage: ${self:opt.stage, 'dev'}

custom:
  staged: ${self:custom.stageVars.${self:provider.stage}, {}}

  stageVars:
    dev:
      dd_enabled: false

  datadog:
    enabled: ${self:custom.staged.dd_enabled, true}

Serverless Monitors

There are seven recommended monitors with default values pre-configured.

MonitorMetricsThresholdServerless Monitor ID
High Error Rateaws.lambda.errors/aws.lambda.invocations>= 10%high_error_rate
Timeoutaws.lambda.duration.max/aws.lambda.timeout>= 1timeout
Out of Memoryaws.lambda.enhanced.out_of_memory> 0out_of_memory
High Iterator Ageaws.lambda.iterator_age.maximum>= 24 hrshigh_iterator_age
High Cold Start Rateaws.lambda.enhanced.invocations(cold_start:true)/
aws.lambda.enhanced.invocations
>= 20%high_cold_start_rate
High Throttlesaws.lambda.throttles/aws.lambda.invocations>= 20%high_throttles
Increased Costaws.lambda.enhanced.estimated_cost↑20%increased_cost

To Enable and Configure a Recommended Serverless Monitor

To create a recommended monitor, you must use its respective serverless monitor ID. Note that you must also set the DATADOG_API_KEY and DATADOG_APP_KEY in your environment.

If you’d like to further configure the parameters for a recommended monitor, you can directly define the parameter values below the serverless monitor ID. Parameters not specified under a recommended monitor will use the default recommended value. The query parameter for recommended monitors cannot be directly modified and will default to using the query valued as defined above; however, you may change the threshold value in query by re-defining it within the options parameter. To delete a monitor, remove the monitor from the serverless.yml template. For further documentation on how to define monitor parameters, see the Datadog Monitors API.

Monitor creation occurs after the function is deployed. In the event that a monitor is unsuccessfully created, the function will still be successfully deployed.

To create a recommended monitor with the default values

Define the appropriate serverless monitor ID without specifying any parameter values

custom:
  datadog:
    addLayers: true
    monitors:
      - high_error_rate:

To configure a recommended monitor

custom:
  datadog:
    addLayers: true
    monitors:
      - high_error_rate:
          name: "High Error Rate with Modified Warning Threshold"
          message: "More than 10% of the function’s invocations were errors in the selected time range. Notify @data.dog@datadoghq.com @slack-serverless-monitors"
          tags: ["modified_error_rate", "serverless", "error_rate"]
          require_full_window: true
          priority: 2
          options:
            include_tags: true
            notify_audit: true
            thresholds:
              ok: 0.025
              warning: 0.05

To delete a monitor

Removing the serverless monitor ID and its parameters will delete the monitor.

To Enable and Configure a Custom Monitor

To define a custom monitor, you must define a unique serverless monitor ID string in addition to passing in the API key and Application key, DATADOG_API_KEY and DATADOG_APP_KEY, in your environment. The query parameter is required but every other parameter is optional. Define a unique serverless monitor ID string and specify the necessary parameters below. For further documentation on monitor parameters, see the Datadog Monitors API.

custom:
  datadog:
    addLayers: true
    monitors:
      - custom_monitor_id:
          name: "Custom Monitor"
          query: "max(next_1w):forecast(avg:system.load.1{*}, 'linear', 1, interval='60m', history='1w', model='default') >= 3"
          message: "Custom message for custom monitor. Notify @data.dog@datadoghq.com @slack-serverless-monitors"
          tags: ["custom_monitor", "serverless"]
          priority: 3
          options:
            enable_logs_sample: true
            require_full_window: true
            include_tags: false
            notify_audit: true
            notify_no_data: false
            thresholds:
              ok: 1
              warning: 2

Breaking Changes

v5.0.0

  • When used in conjunction with the Datadog Extension, this plugin sets service and env tags through environment variables instead of Lambda resource tags.
  • The enableTags parameter was replaced by the new service, env parameters.

v4.0.0

  • The Datadog Lambda Extension is now the default mechanism for transmitting telemetry to Datadog.

Opening Issues

If you encounter a bug with this package, let us know by filing an issue! Before opening a new issue, please search the existing issues to avoid duplicates.

When opening an issue, include your Serverless Framework version, Python/Node.js version, and stack trace if available. Also, please include the steps to reproduce when appropriate.

You can also open an issue for a feature request.

Contributing

If you find an issue with this package and have a fix, open a pull request following the procedures.

Community

For product feedback and questions, join the #serverless channel in the Datadog community on Slack.

Author: DataDog
Source Code: https://github.com/DataDog/serverless-plugin-datadog 
License: View license

#serverless #datadog #plugin