A Bug in Stripe Caused by AWS Lambda Serverless Design (Container re-use)

From time to time I like to loiter on people’s GitHub Repos look through issues submitted and see if there are interesting hidden gems and bugs that would make a good lesson or learning experience and boy did I find one for you. This bug is caused in stripe-node code in AWS Lambda serverless environment where requests are failing intermittently. We discuss how AWS serverless container re-use can cause this and how stripe solved it.

Resources
https://github.com/stripe/stripe-node/issues/1040
Intermittent Error: write EPIPE when running stripe client in AWS Lambda · Issue #1040 · stripe/stripe-node · GitHub
https://aws.amazon.com/blogs/compute/container-reuse-in-lambda/

#aws #serverless #cloud #developer

What is GEEK

Buddha Community

A Bug in Stripe Caused by AWS Lambda Serverless Design (Container re-use)

A Bug in Stripe Caused by AWS Lambda Serverless Design (Container re-use)

From time to time I like to loiter on people’s GitHub Repos look through issues submitted and see if there are interesting hidden gems and bugs that would make a good lesson or learning experience and boy did I find one for you. This bug is caused in stripe-node code in AWS Lambda serverless environment where requests are failing intermittently. We discuss how AWS serverless container re-use can cause this and how stripe solved it.

Resources
https://github.com/stripe/stripe-node/issues/1040
Intermittent Error: write EPIPE when running stripe client in AWS Lambda · Issue #1040 · stripe/stripe-node · GitHub
https://aws.amazon.com/blogs/compute/container-reuse-in-lambda/

#aws #serverless #cloud #developer

Christa  Stehr

Christa Stehr

1602964260

50+ Useful Kubernetes Tools for 2020 - Part 2

Introduction

Last year, we provided a list of Kubernetes tools that proved so popular we have decided to curate another list of some useful additions for working with the platform—among which are many tools that we personally use here at Caylent. Check out the original tools list here in case you missed it.

According to a recent survey done by Stackrox, the dominance Kubernetes enjoys in the market continues to be reinforced, with 86% of respondents using it for container orchestration.

(State of Kubernetes and Container Security, 2020)

And as you can see below, more and more companies are jumping into containerization for their apps. If you’re among them, here are some tools to aid you going forward as Kubernetes continues its rapid growth.

(State of Kubernetes and Container Security, 2020)

#blog #tools #amazon elastic kubernetes service #application security #aws kms #botkube #caylent #cli #container monitoring #container orchestration tools #container security #containers #continuous delivery #continuous deployment #continuous integration #contour #developers #development #developments #draft #eksctl #firewall #gcp #github #harbor #helm #helm charts #helm-2to3 #helm-aws-secret-plugin #helm-docs #helm-operator-get-started #helm-secrets #iam #json #k-rail #k3s #k3sup #k8s #keel.sh #keycloak #kiali #kiam #klum #knative #krew #ksniff #kube #kube-prod-runtime #kube-ps1 #kube-scan #kube-state-metrics #kube2iam #kubeapps #kubebuilder #kubeconfig #kubectl #kubectl-aws-secrets #kubefwd #kubernetes #kubernetes command line tool #kubernetes configuration #kubernetes deployment #kubernetes in development #kubernetes in production #kubernetes ingress #kubernetes interfaces #kubernetes monitoring #kubernetes networking #kubernetes observability #kubernetes plugins #kubernetes secrets #kubernetes security #kubernetes security best practices #kubernetes security vendors #kubernetes service discovery #kubernetic #kubesec #kubeterminal #kubeval #kudo #kuma #microsoft azure key vault #mozilla sops #octant #octarine #open source #palo alto kubernetes security #permission-manager #pgp #rafay #rakess #rancher #rook #secrets operations #serverless function #service mesh #shell-operator #snyk #snyk container #sonobuoy #strongdm #tcpdump #tenkai #testing #tigera #tilt #vert.x #wireshark #yaml

Gordon  Matlala

Gordon Matlala

1617875400

Adding Code to AWS Lambda, Lambda Layers, and Lambda Extensions Using Docker

2020 was a difficult year for all of us, and it was no different for engineering teams. Many software releases were postponed, and the industry slowed its development speed quite a bit.

But at least at AWS, some teams released updates out of the door at the end of the year. AWS Lambda received two significant improvements:

  • AWS Lambda Extensions; and
  • Support of Docker images for your functions.

With these two new features and Lambda Layers, we now have three ways to add code to Lambda that isn’t directly part of our Lambda function.

The question is now: when should we use what?

In this article, I try to shine some light on the Lambda Layers, Lambda Extensions, and Docker image for Lambda.

First things first. All these Lambda features can be used together. So if you think about where to put your code, at least your decisions aren’t mutually exclusive. You can upload a Docker image and attach a regular Lambda Layer and a Lambda Extension. The same is possible if your Lambda function is based on a ZIP archive.

What does this all mean? Keep reading and find out.

#aws #aws-lambda #serverless #devops #docker #lambda

Christa  Stehr

Christa Stehr

1598408880

How To Unite AWS KMS with Serverless Application Model (SAM)

The Basics

AWS KMS is a Key Management Service that let you create Cryptographic keys that you can use to encrypt and decrypt data and also other keys. You can read more about it here.

Important points about Keys

Please note that the customer master keys(CMK) generated can only be used to encrypt small amount of data like passwords, RSA key. You can use AWS KMS CMKs to generate, encrypt, and decrypt data keys. However, AWS KMS does not store, manage, or track your data keys, or perform cryptographic operations with data keys.

You must use and manage data keys outside of AWS KMS. KMS API uses AWS KMS CMK in the encryption operations and they cannot accept more than 4 KB (4096 bytes) of data. To encrypt application data, use the server-side encryption features of an AWS service, or a client-side encryption library, such as the AWS Encryption SDK or the Amazon S3 encryption client.

Scenario

We want to create signup and login forms for a website.

Passwords should be encrypted and stored in DynamoDB database.

What do we need?

  1. KMS key to encrypt and decrypt data
  2. DynamoDB table to store password.
  3. Lambda functions & APIs to process Login and Sign up forms.
  4. Sign up/ Login forms in HTML.

Lets Implement it as Serverless Application Model (SAM)!

Lets first create the Key that we will use to encrypt and decrypt password.

KmsKey:
    Type: AWS::KMS::Key
    Properties: 
      Description: CMK for encrypting and decrypting
      KeyPolicy:
        Version: '2012-10-17'
        Id: key-default-1
        Statement:
        - Sid: Enable IAM User Permissions
          Effect: Allow
          Principal:
            AWS: !Sub arn:aws:iam::${AWS::AccountId}:root
          Action: kms:*
          Resource: '*'
        - Sid: Allow administration of the key
          Effect: Allow
          Principal:
            AWS: !Sub arn:aws:iam::${AWS::AccountId}:user/${KeyAdmin}
          Action:
          - kms:Create*
          - kms:Describe*
          - kms:Enable*
          - kms:List*
          - kms:Put*
          - kms:Update*
          - kms:Revoke*
          - kms:Disable*
          - kms:Get*
          - kms:Delete*
          - kms:ScheduleKeyDeletion
          - kms:CancelKeyDeletion
          Resource: '*'
        - Sid: Allow use of the key
          Effect: Allow
          Principal:
            AWS: !Sub arn:aws:iam::${AWS::AccountId}:user/${KeyUser}
          Action:
          - kms:DescribeKey
          - kms:Encrypt
          - kms:Decrypt
          - kms:ReEncrypt*
          - kms:GenerateDataKey
          - kms:GenerateDataKeyWithoutPlaintext
          Resource: '*'

The important thing in above snippet is the KeyPolicy. KMS requires a Key Administrator and Key User. As a best practice your Key Administrator and Key User should be 2 separate user in your Organisation. We are allowing all permissions to the root users.

So if your key Administrator leaves the organisation, the root user will be able to delete this key. As you can see **KeyAdmin **can manage the key but not use it and KeyUser can only use the key. ${KeyAdmin} and **${KeyUser} **are parameters in the SAM template.

You would be asked to provide values for these parameters during SAM Deploy.

#aws #serverless #aws-sam #aws-key-management-service #aws-certification #aws-api-gateway #tutorial-for-beginners #aws-blogs

Hermann  Frami

Hermann Frami

1617016800

Serverless Framework: Use AWS S3 Object Lambda to Resize Images on The Fly

Goal

In this post, I will show you how to use Amazon S3 Object Lambda to resize images on the fly. The  Serverless Framework will be used to define the Infrastructure as Code and to simplify the deployment. Sharp will be used to resize the images. Lambda will be written using the Node.js 14x  Lambda runtime

What’s Amazon S3 Object Lambda

One of the most common Lambda patterns is to transform data stored inside Amazon S3. Generally, a lambda function is invoked after a file has been uploaded. Lambda would retrieve that file, apply any needed transformation (e.g. converting type of file) and store the result in S3.

That pattern was working well, however, it would require some work done onto a file despite that being accessed in the future or not.

If you needed to convert a file on the fly you should have created a Lambda function, invoke it via Amazon API GW and wait for the lambda to perform the transformation.

AWS has recently introduced  Amazon S3 Object Lambda in a good post by Danilo Poccia. S3 Object Lambda allows creating a Lambda directly connected to the S3 bucket (using  S3 Access Points) that is automatically invoked when retrieving the object from S3!

That means that our application needs only to send an S3 Get Object request to retrieve the original or transformed data

Also, a very important peculiarity of using Amazon S3 Object Lambda it’s that the file you want to retrieve doesn’t need to exist on S3! We will make use of this for our scenario

_Note: High-level AWS CLI S3 commands (e.g. _aws s3 cp_) don’t currently support S3 Object Lambda, instead we need to use low-level S3 API commands (e.g. __aws s3api get-object)_

Common Use Cases

In his post, Danilo highlighted the most common use cases for Amazon S3 Object Lambda:

  • Converting across data formats (e.g. XML to JSON)
  • Compressing or decompressing files on the fly
  • Resizing images on the fly (our use case!)
  • Many more (left to the devs’ creativity)

#aws-lambda #serverless #aws-s3 #aws