Serverless Diary: What you need to know about NoSQL AWS Dynamo Database

I. Introduction

Having shared my experience of implementing microservices in a serverless style in my previous blogs, let me dive into a level deeper sharing design considerations to be aware of for each component of our microservices. The focus of this blog will be understanding the right way of designing NoSQL Dynamo tables, which in my opinion is a pre-requisite before I share my learnings on how to approach event-driven architecture. More on that in my future blogs. Let’s use the below simple diagram as a starting point to discuss DynamoDb. I will be extending this design in my future blogs as I touch one service at a time and the role it plays in an event-driven serverless architecture.

Microservice: API G -> Lambda -> Dynamodb

II. Isn’t DynamoDB just another database?

Why even bother going through a blog to understand design considerations for the NoSQL Dynamo database? Why not just crack on with the build and modify/extend as we go along? Here’s why:

Even if you have several implementations under your belt which used either a SQL or NoSQL database, recognize the fact that this is still a very much vendor (AWS) specific service and you really need to understand how to approach it if you wish to leverage its true benefits and power.
Winging it as you go approach, will prove both painful and expensive. More on that in a bit.
It’s a database, but in AWS serverless world, this is THE database that provides a very powerful way of implementing event-driven architectures.

#serverless #microservices #aws #dynamodb

What is GEEK

Buddha Community

Serverless Diary: What you need to know about NoSQL AWS Dynamo Database

Christa Stehr

1598408880

How To Unite AWS KMS with Serverless Application Model (SAM)

The Basics

AWS KMS is a Key Management Service that let you create Cryptographic keys that you can use to encrypt and decrypt data and also other keys. You can read more about it here.

Important points about Keys

Please note that the customer master keys(CMK) generated can only be used to encrypt small amount of data like passwords, RSA key. You can use AWS KMS CMKs to generate, encrypt, and decrypt data keys. However, AWS KMS does not store, manage, or track your data keys, or perform cryptographic operations with data keys.

You must use and manage data keys outside of AWS KMS. KMS API uses AWS KMS CMK in the encryption operations and they cannot accept more than 4 KB (4096 bytes) of data. To encrypt application data, use the server-side encryption features of an AWS service, or a client-side encryption library, such as the AWS Encryption SDK or the Amazon S3 encryption client.

Scenario

We want to create signup and login forms for a website.

Passwords should be encrypted and stored in DynamoDB database.

What do we need?

KMS key to encrypt and decrypt data
DynamoDB table to store password.
Lambda functions & APIs to process Login and Sign up forms.
Sign up/ Login forms in HTML.

Lets Implement it as Serverless Application Model (SAM)!

Lets first create the Key that we will use to encrypt and decrypt password.

KmsKey:
    Type: AWS::KMS::Key
    Properties: 
      Description: CMK for encrypting and decrypting
      KeyPolicy:
        Version: '2012-10-17'
        Id: key-default-1
        Statement:
        - Sid: Enable IAM User Permissions
          Effect: Allow
          Principal:
            AWS: !Sub arn:aws:iam::${AWS::AccountId}:root
          Action: kms:*
          Resource: '*'
        - Sid: Allow administration of the key
          Effect: Allow
          Principal:
            AWS: !Sub arn:aws:iam::${AWS::AccountId}:user/${KeyAdmin}
          Action:
          - kms:Create*
          - kms:Describe*
          - kms:Enable*
          - kms:List*
          - kms:Put*
          - kms:Update*
          - kms:Revoke*
          - kms:Disable*
          - kms:Get*
          - kms:Delete*
          - kms:ScheduleKeyDeletion
          - kms:CancelKeyDeletion
          Resource: '*'
        - Sid: Allow use of the key
          Effect: Allow
          Principal:
            AWS: !Sub arn:aws:iam::${AWS::AccountId}:user/${KeyUser}
          Action:
          - kms:DescribeKey
          - kms:Encrypt
          - kms:Decrypt
          - kms:ReEncrypt*
          - kms:GenerateDataKey
          - kms:GenerateDataKeyWithoutPlaintext
          Resource: '*'

The important thing in above snippet is the KeyPolicy. KMS requires a Key Administrator and Key User. As a best practice your Key Administrator and Key User should be 2 separate user in your Organisation. We are allowing all permissions to the root users.

So if your key Administrator leaves the organisation, the root user will be able to delete this key. As you can see **KeyAdmin **can manage the key but not use it and KeyUser can only use the key. ${KeyAdmin} and **${KeyUser} **are parameters in the SAM template.

You would be asked to provide values for these parameters during SAM Deploy.

#aws #serverless #aws-sam #aws-key-management-service #aws-certification #aws-api-gateway #tutorial-for-beginners #aws-blogs

Sheldon Grant

1616070300

Serverless Diary: What you need to know about NoSQL AWS Dynamo Database

I. Introduction

Having shared my experience of implementing microservices in a serverless style in my previous blogs, let me dive into a level deeper sharing design considerations to be aware of for each component of our microservices. The focus of this blog will be understanding the right way of designing NoSQL Dynamo tables, which in my opinion is a pre-requisite before I share my learnings on how to approach event-driven architecture. More on that in my future blogs. Let’s use the below simple diagram as a starting point to discuss DynamoDb. I will be extending this design in my future blogs as I touch one service at a time and the role it plays in an event-driven serverless architecture.

Microservice: API G -> Lambda -> Dynamodb

II. Isn’t DynamoDB just another database?

Why even bother going through a blog to understand design considerations for the NoSQL Dynamo database? Why not just crack on with the build and modify/extend as we go along? Here’s why:

Even if you have several implementations under your belt which used either a SQL or NoSQL database, recognize the fact that this is still a very much vendor (AWS) specific service and you really need to understand how to approach it if you wish to leverage its true benefits and power.
Winging it as you go approach, will prove both painful and expensive. More on that in a bit.
It’s a database, but in AWS serverless world, this is THE database that provides a very powerful way of implementing event-driven architectures.

#serverless #microservices #aws #dynamodb

Hermann Frami

1655426640

Serverless Plugin for Microservice Code Management and Deployment

Serverless M

Serverless M (or Serverless Modular) is a plugin for the serverless framework. This plugins helps you in managing multiple serverless projects with a single serverless.yml file. This plugin gives you a super charged CLI options that you can use to create new features, build them in a single file and deploy them all in parallel

Currently this plugin is tested for the below stack only

AWS
NodeJS λ
Rest API (You can use other events as well)

Prerequisites

Make sure you have the serverless CLI installed

# Install serverless globally
$ npm install serverless -g

Getting Started

To start the serverless modular project locally you can either start with es5 or es6 templates or add it as a plugin

ES6 Template install

# Step 1. Download the template
$ sls create --template-url https://github.com/aa2kb/serverless-modular/tree/master/template/modular-es6 --path myModularService

# Step 2. Change directory
$ cd myModularService

# Step 3. Create a package.json file
$ npm init

# Step 3. Install dependencies
$ npm i serverless-modular serverless-webpack webpack --save-dev

ES5 Template install

# Step 1. Download the template
$ sls create --template-url https://github.com/aa2kb/serverless-modular/tree/master/template/modular-es5 --path myModularService

# Step 2. Change directory
$ cd myModularService

# Step 3. Create a package.json file
$ npm init

# Step 3. Install dependencies
$ npm i serverless-modular --save-dev

If you dont want to use the templates above you can just add in your existing project

Adding it as plugin

plugins:
  - serverless-modular

Now you are all done to start building your serverless modular functions

API Reference

The serverless CLI can be accessed by

# Serverless Modular CLI
$ serverless modular

# shorthand
$ sls m

Serverless Modular CLI is based on 4 main commands

sls m init
sls m feature
sls m function
sls m build
sls m deploy

init command

sls m init

The serverless init command helps in creating a basic .gitignore that is useful for serverless modular.

The basic .gitignore for serverless modular looks like this

#node_modules
node_modules

#sm main functions
sm.functions.yml

#serverless file generated by build
src/**/serverless.yml

#main serverless directories generated for sls deploy
.serverless

#feature serverless directories generated sls deploy
src/**/.serverless

#serverless logs file generated for main sls deploy
.sm.log

#serverless logs file generated for feature sls deploy
src/**/.sm.log

#Webpack config copied in each feature
src/**/webpack.config.js

feature command

The feature command helps in building new features for your project

options (feature Command)

This command comes with three options

--name: Specify the name you want for your feature

--remove: set value to true if you want to remove the feature

--basePath: Specify the basepath you want for your feature, this base path should be unique for all features. helps in running offline with offline plugin and for API Gateway

options	shortcut	required	values	default value
--name	-n	✅	string	N/A
--remove	-r	❎	true, false	false
--basePath	-p	❎	string	same as name

Examples (feature Command)

Creating a basic feature

# Creating a jedi feature
$ sls m feature -n jedi

Creating a feature with different base path

# A feature with different base path
$ sls m feature -n jedi -p tatooine

Deleting a feature

# Anakin is going to delete the jedi feature
$ sls m feature -n jedi -r true

function command

The function command helps in adding new function to a feature

options (function Command)

This command comes with four options

--name: Specify the name you want for your function

--feature: Specify the name of the existing feature

--path: Specify the path for HTTP endpoint helps in running offline with offline plugin and for API Gateway

--method: Specify the path for HTTP method helps in running offline with offline plugin and for API Gateway

options	shortcut	required	values	default value
--name	-n	✅	string	N/A
--feature	-f	✅	string	N/A
--path	-p	❎	string	same as name
--method	-m	❎	string	'GET'

Examples (function Command)

Creating a basic function

# Creating a cloak function for jedi feature
$ sls m function -n cloak -f jedi

Creating a basic function with different path and method

# Creating a cloak function for jedi feature with custom path and HTTP method
$ sls m function -n cloak -f jedi -p powers -m POST

build command

The build command helps in building the project for local or global scope

options (build Command)

This command comes with four options

--scope: Specify the scope of the build, use this with "--feature" tag

--feature: Specify the name of the existing feature you want to build

options	shortcut	required	values	default value
--scope	-s	❎	string	local
--feature	-f	❎	string	N/A

Saving build Config in serverless.yml

You can also save config in serverless.yml file

custom:
  smConfig:
    build:
      scope: local

Examples (build Command)

all feature build (local scope)

# Building all local features
$ sls m build

Single feature build (local scope)

# Building a single feature
$ sls m build -f jedi -s local

All features build global scope

# Building all features with global scope
$ sls m build -s global

deploy command

The deploy command helps in deploying serverless projects to AWS (it uses sls deploy command)

options (deploy Command)

This command comes with four options

--sm-parallel: Specify if you want to deploy parallel (will only run in parallel when doing multiple deployments)

--sm-scope: Specify if you want to deploy local features or global

--sm-features: Specify the local features you want to deploy (comma separated if multiple)

options	shortcut	required	values	default value
--sm-parallel	❎	❎	true, false	true
--sm-scope	❎	❎	local, global	local
--sm-features	❎	❎	string	N/A
--sm-ignore-build	❎	❎	string	false

Saving deploy Config in serverless.yml

You can also save config in serverless.yml file

custom:
  smConfig:
    deploy:
      scope: local
      parallel: true
      ignoreBuild: true

Examples (deploy Command)

Deploy all features locally

# deploy all local features
$ sls m deploy

Deploy all features globally

# deploy all global features
$ sls m deploy --sm-scope global

Deploy single feature

# deploy all global features
$ sls m deploy --sm-features jedi

Deploy Multiple features

# deploy all global features
$ sls m deploy --sm-features jedi,sith,dark_side

Deploy Multiple features in sequence

# deploy all global features
$ sls m deploy  --sm-features jedi,sith,dark_side --sm-parallel false

Author: aa2kb
Source Code: https://github.com/aa2kb/serverless-modular
License: MIT license

#serverless #aws #node #lambda

Kole Haag

1602403200

What is NoSQL and How is it Utilized?

Posted on September 25, 2020 by Dean Conally | Updated: October 8, 2020

Category: Tutorials | Tags: Cassandra, Columns, Database, Database Management, Database Structure, DB2, Document Stores, Dynamic Schema, Extensible Record Stores, Graph Stores, JSON, Key-Value, MSSQL, Multi-Row, MySQL, Node, Node Relationship Node, Non-Relational Databases, NoSQL, NoSQL Model, Query, Rows, Scalability, Schema Free, SQL, Stores, Tables, Wide-Column

Reading Time: 5 minutes

What is NoSQL?

A NoSQL or a NoSQL Database is a term used when referring to a “non SQL” or “not only SQL” database. NoSQL databases store data in a different format than a traditional relational database management systems. This is why NoSQL is often associated with the term “non-relational” database. Simply put, NoSQL databases are modern databases with high flexibility, blazing performance, and built for scalability. These databases are used when you require low latency and high extensibility while working with large data structures. The versatility of NoSQL is due to the nature of as being unrestricted in comparison to relational databases models such as MySQL or DB2.

SQL vs. NoSQL Comparison

There are multiple differences between SQL and NoSQL database types. In the table below, we will compare some of the most critical variations.

#tutorials #cassandra #columns #database #database management #database structure #db2 #document stores #dynamic schema #extensible record stores #graph stores #json #key-value #mssql #multi-row #mysql #node #node relationship node #non-relational databases #nosql #nosql model #query #rows #scalability #schema free #sql #stores #tables #wide-column

Matt Towne

1589791867

Serverless CI/CD on the AWS Cloud

CI/CD pipelines have long played a major role in speeding up the development and deployment of cloud-native apps. Cloud services like AWS lend themselves to more agile deployment through the services they offer as well as approaches such as Infrastructure as Code. There is no shortage of tools to help you manage your CI/CD pipeline as well.

While the majority of development teams have streamlined their pipelines to take full advantage of cloud-native features, there is still so much that can be done to refine CI/CD even further. The entire pipeline can now be built as code and managed either via Git as a single source of truth or by using visual tools to help guide the process.

The entire process can be fully automated. Even better, it can be made serverless, which allows the CI/CD pipeline to operate with immense efficiency. Git branches can even be utilized as a base for multiple pipelines. Thanks to the three tools from Amazon; AWS CodeCommit, AWS CodeBuild, and AWS CodeDeploy, serverless CI/CD on the AWS cloud is now easy to set up.

#aws #aws codebuild #aws codecommit #aws codedeploy #cd #cd pipeline #ci #ci/cd processes #ci/cd workflow #serverless