AWS Kinesis vs. SNS vs. SQS — A Comparison With Python Examples

AWS Kinesis vs. SNS vs. SQS — A Comparison With Python Examples

A comparisons between AWS services - Kinesis, SQS, SNS. AWS Kinesis vs. SNS vs. SQS — A Comparison With Python Examples. The following Python script demonstrates. This comparison article is looking at AWS services that allow you to decouple sending and receiving data - Kinesis, SQS, and SNS.

How to choose a decoupling service that suits your use case

Decoupling offers a myriad of advantages, but choosing the right tool for the job may be challenging. AWS alone provides several services that allow us to decouple sending and receiving data. While these services seem to provide similar functionality on the surface, they are designed for different use cases and each of them can be useful if applied properly to the problem at hand.

AWS SQS

As one of the oldest AWS services, SQS has a track record of providing an extremely simple and effective decoupling mechanism. The entire service is based on sending messages to the queueand allowing for applications (e.g. ECS containers, Lambda functions) to poll for messages and process them. The message stays in the queue until some application picks it up, processes it, and deletes the message when it’s done.

The most important distinction between SQS and other decoupling services is that it’s not a publish-subscribe service. SQS has no concept of producers, consumers, topics, or subscribers. All it does is provide a distributed queue that allows:

  • Sending messages to the queue.
  • Polling for messages being in the queue.
  • Picking up the messages from the queue.
  • Deleting a message from the queue once the message has been successfully processed.

SQS does not push messages to any applications. Instead, once a message is sent to SQS, an application must actively poll for messages to receive and process them. Also, it’s not enough to pick up the message from the queue to make it disappear. The message stays in the queue until:

  • It has been successfully processed and removed from the queue.
  • The visibility timeout has expired. Imagine that one containerized application picked up a message, but while processing it, it got stuck in a zombie process. To ensure that this message will be processed, it becomes visible to other workers once the visibility timeout expires. This timeout can be set from zero seconds up to 12 hours, with the default being 30 seconds.

The code snippet below demonstrates how you can:

  • Create a queue.
  • Send a message to the queue.
  • Receive, process, and delete a message.

serverless python data-engineering aws programming

What is Geek Coin

What is GeekCash, Geek Token

Best Visual Studio Code Themes of 2021

Bootstrap 5 Tutorial - Bootstrap 5 Crash Course for Beginners

Nest.JS Tutorial for Beginners

Hello Vue 3: A First Look at Vue 3 and the Composition API

top 30 Python Tips and Tricks for Beginners

In this post, we'll learn top 30 Python Tips and Tricks for Beginners

Python Programming & Data Handling

Python Programming & Data Handling

Basic Data Types in Python | Python Web Development For Beginners

In the programming world, Data types play an important role. Each Variable is stored in different data types and responsible for various functions. Python had two different objects, and They are mutable and immutable objects.

Your Data Architecture: Simple Best Practices for Your Data Strategy

Your Data Architecture: Simple Best Practices for Your Data Strategy. Don't miss this helpful article.

Serverless COVID-19 Data Scraper with Python and AWS Lambda

Step-by-Step Tutorial: Scheduling your Python Script with AWS Lambda