Working with AWS Lambda and SQS

Working with AWS Lambda and SQS

This blog is a short guide on implementing AWS Lambda and SQS together. First we’ll introduce these technologies and explain why they work well together, then walkthrough some configurations and invocation using a simple Nodejs script.

This blog is a short guide on implementing AWS Lambda and SQS together. First we’ll introduce these technologies and explain why they work well together, then walkthrough some configurations and invocation using a simple Nodejs script.

blog aws lambda sqs 1

What are they? AWS Lambda is a popular choice in serverless computing. As an event-driven platform, AWS Lambda requires users to configure event sources in order to run a function. Their built-in support for many AWS services like S3 and SQS as event sources allow for relatively simple configuration. Note that Lambda functions can also be triggered via HTTP(S) by using AWS API Gateway service.

AWS SQS (Simple Queue Service) is a queue service with 2 types: standard and FIFO. The major difference is that standard queues guarantee “at-least once” delivery of its messages, while FIFO queues guarantee “exactly-once” delivery. FIFO queues, however, provide lower maximum throughput than standard.

As previously mentioned, AWS SQS can be an event source for AWS Lambda. This means that when a message is added to the queue, the configured Lambda function will be invoked with an event containing this message.

blog aws lambda sqs 2

But why use them together?

Common reasons include better handling Lambda function invocations and errors. For function invocations, the default concurrency limit is 1000. If a function has reached this limit, any additional invocations will fail with a throttling error. However, with SQS as an event source, events resulting in throttling errors are automatically retried based on set configs. This prevents the loss of events due to unexpected jumps in usage. Additionally, dead-letter queues (DLQ) can be configured as a place where failed events go, to avoid retrying indefinitely. Failed events in DLQ can then be sent somewhere like a DB for analysis.

One limitation for many use cases was the “at-least once” delivery model of events with standard queues. Essentially, there was no guarantee that an event will only be delivered once to a consuming Lambda function. In cases where each event can strictly only be processed once, one would have to employ various strategies resulting in higher overhead and complexity. But, this is no longer an issue because recently the AWS team added support for FIFO queues as event sources to Lambda functions.

aws aws lambda sqs amazon web services

Bootstrap 5 Complete Course with Examples

Bootstrap 5 Tutorial - Bootstrap 5 Crash Course for Beginners

Nest.JS Tutorial for Beginners

Hello Vue 3: A First Look at Vue 3 and the Composition API

Building a simple Applications with Vue 3

Deno Crash Course: Explore Deno and Create a full REST API with Deno

How to Build a Real-time Chat App with Deno and WebSockets

Convert HTML to Markdown Online

HTML entity encoder decoder Online

Infrastructure as Code on Amazon Web Services (AWS)

How to Build and Manage Infrastructure as Code with Amazon Web Services - AWS CloudFormation, AWS CloudWatch and Amazon EC2

An Introduction to Infrastructures for WEB projects in AWS

An Introduction to Infrastructures for WEB projects in AWS - Let's review a particular project in detail and the cost of infrastructure maintenance. And so, we have an internal WEB application that consists of a static website on React, a backend on Go, and a database DynamoDB. All code is stored on Github and for CI/CD we use Github Actions. The infrastructure code is defined in Terraform.

AWS Launches a No-Code Mobile and Web App Builder in Beta: Amazon Honeycode

Recently, AWS announced the beta release of Amazon Honeycode, a fully managed service allowing customers to build mobile and web applications without writing any code quickly.

AWS Fargate for Amazon Elastic Kubernetes Service | Caylent

Easily run Kubernetes-based applications on AWS by leveraging AWS Fargate and Amazon Elastic Kubernetes Service together. Learn more here.

Amazon AWS Tutorial | Learning Amazon Web Services

In this video we will see What is AWS? and we will look into An Introduction to Amazon Web Services.