SNS and SQS are AWS services which are often used in an event-driven architecture. A combination of those services offers the possibility to fan-out messages. Often there is a need to filter these messages before replicating them to multiple queues. Now I want to explain a way to achieve this by taking full advantage of these AWS services and without the need to write any custom code.

Let’s start with a quick refresher on SNS-SQS fan-out. SNS is an AWS service that coordinates and manages the delivery or sending of messages to subscribing endpoints. It offers you the possibilitytocreatean “SNS topic” which is a logical access point that acts as a communication channel. By using AWS CLI, SDK, or the web console, you can start publishing messages to this topic. The SNS topic will forward the messages to its subscribers. One of the possible subscribers is SQS, which is the queuing service of AWS. You can subscribe multiple queues to the same topic to replicate a message over multiple queues. This is called fan-out.

The above scenario will replicate a message to all queues, and this allows for parallel asynchronous processing. Now what if you want to replicate messages to only some queues, depending on some message attributes? SNS offers a great solution for this case.

In this example, we will filter on two attributes:

• Color
• Number

The ColorQueue will receive all messages independent of the value of the color attribute. The BlueYellowQueue will receive all messages which have the attribute color set to blue or yellow. The RedQueue will only receive the messages with attribute color set to red. Last, there are two queues that can receive messages which have their attribute set to green. It depends on the second attribute (number) whether a message will end up on the GreenHighQueue (number > 100) or the GreenLowQueue (number ≤100).

#aws #amazon-web-services #programming #devops #software-development