Microservices Event Driven Architecture

Understanding Microservice Architecture and Different Microservice Patterns
► Pattern: Microservice Architecture
► Decomposition patterns
► The Database per Service pattern describes how each service has its own database in order to ensure loose coupling.
► The API Gateway pattern defines how clients access the services in a microservice architecture.
► The Client-side Discovery and Server-side Discovery patterns are used to route requests for a client to an available service instance in a microservice architecture.
The Messaging and Remote Procedure Invocation patterns are two different ways that services can communicate.
► The Single Service per Host and Multiple Services per Host patterns are two different deployment strategies.
► Cross-cutting concerns patterns: Microservice chassis pattern and Externalized configuration
► Testing patterns: Service Component Test and Service Integration Contract Test
► Circuit Breaker
► Observability patterns:

  • Log aggregation
  • Application metrics
  • Audit logging
  • Distributed tracing
  • Exception tracking
  • Health check API
  • Log deployments and changes
    UI patterns:
    ► Server-side page fragment composition
    ►Client-side UI composition

#microservices #developer

What is GEEK

Buddha Community

Microservices Event Driven Architecture

Serverless Vs Microservices Architecture - A Deep Dive

Companies need to be thinking long-term before even starting a software development project. These needs are solved at the level of architecture: business owners want to assure agility, scalability, and performance.

The top contenders for scalable solutions are serverless and microservices. Both architectures prioritize security but approach it in their own ways. Let’s take a look at how businesses can benefit from the adoption of serverless architecture vs microservices, examine their differences, advantages, and use cases.

#serverless #microservices #architecture #software-architecture #serverless-architecture #microservice-architecture #serverless-vs-microservices #hackernoon-top-story

Event-Driven Architecture as a Strategy

Event-driven architecture, or EDA, is an integration pattern where applications are oriented around publishing events and responding to events. It provides five key benefits to modern application architecture: scalability, resilience, agility, data sharing, and cloud-enabling.

This article explores how EDA fits into enterprise integration, its three styles, how it enables business strategy, its benefits and trade-offs, and the next steps to start an EDA implementation.

Although there are many brokers you can use to publish event messages, the open-source software Apache Kafka has emerged as the market leader in this space. This article is focused on a Kafka-based EDA, but much of the principles here apply to any EDA implementation.

Spectrum of Integration

If asked to describe integration a year ago, I would have said there are two modes: application integration and data integration. Today I’d say that integration is on a spectrum, with data on one end, application on the other end, and event integration in the middle.

A spectrum with event integration on the left, application integration on the right, and event integration in the middle.

The spectrum of integration.

Application integration is REST, SOAP, ESB, etc. These are patterns for making functionality in one application run in response to a request from another app. It’s especially strong for B2B partnership and exposing value in one application to another. It’s less strong for many data use cases, like BI reporting and ML pipelines, since most application integrations wait passively to be invoked by a client, rather than actively pushing data where it needs to go.Data integration is patterns for getting data from point A to point B, including ETL, managed file transfer, etc. They’re strong for BI reporting, ML pipelines, and other data movement tasks, but weaker than application integration for many B2B partnerships and applications sharing functionality.

Event integration has one foot in data and the other in application integration, and it largely gets the benefits of both. When one application subscribes to another app’s events, it can trigger application code in response to those events, which feels a bit like an API from application integration. The events triggering this functionality also carry with them a significant amount of data, which feels a bit like data integration.

EDA strikes a balance between the two classic integration modes. Refactoring traditional application integrations into an event integration pattern opens more doors for analytics, machine learning, BI, and data synchronization between applications. It gets the best of application and data integration patterns. This is especially relevant for companies moving towards an operating model of leveraging data to drive new channels and partnerships. If your integration strategy does not unlock your data, then that strategy will fail. But if your integration strategy unlocks data at the expense of application architecture that’s scalable and agile, then again it will fail. Event integration strikes a balance between both those needs.

Strategy vs. Tactic

EDA often begins with isolated teams as a tactic for delivering projects. Ideally, such a project would have a deliberative approach to EDA and a common event message broker, usually cloud-native brokers on AWS, Azure, etc. Different teams select different brokers to meet their immediate needs. They do not consider integration beyond their project scope. Eventually, they may face the need for enterprise integration at a later date.

A major transition in EDA maturity happens when the investment in EDA shifts from a project tactic to enterprise strategy via a common event bus, usually Apache Kafka. Events can take a role in the organization’s business and technical innovation across the enterprise. Data becomes more rapidly shareable across the enterprise and also between you and your external strategic partners.

EDA Styles

Before discussing the benefits of EDA, let’s cover the three common styles of EDA: event notification, event-carried state transfer, and event sourcing.

Event Notification

This pattern publishes events with minimal information: the event type, timestamps, and a key-value like an account number or some other key of the entity that raised the event. This informs subscribers that an event occurred, but if subscribers need any information about how that event changed things (like which fields changed, etc.), it must invoke a data retrieval service from the system of record. This is the simplest form of EDA, but it provides the least benefit.

Event-Carried State Transfer

In this pattern, the events carry all information about the state change, typically a before and after image. Subscribing systems can then store their cache of data without the need to retrieve it from the system of record.

This builds resilience since the subscribing systems can function if the source becomes unavailable. It helps performance, as there’s no remote call required to access source information. For example, if an inventory system publishes the full state of all inventory changes, a sales service subscribing to it can know the current inventory without retrieving from the inventory system — it can simply use the cache it built from the inventory events, even during an inventory service outage.

It also helps performance because the subscriber’s data storage can be custom-tuned just for that subscriber’s unique performance needs. Using the previous example, perhaps the inventory service is best suited using a relational database, but the sales service could get better performance from a no-SQL database like MongoDB. Since the sales services no longer need to retrieve from the inventory service, it’s at liberty to use a different DBMS than the inventory service. Additionally, if the inventory service is having an outage, the sales service would be unaffected since it pulls inventory data from its local cache.

The cons are that lots of data is copied around and there is more complexity on the receivers since they have to sort out maintaining all the state they are receiving.

#integration #microservices #data #kafka #enterprise architecture #event driven architecture #application architecture

The Service Mesh in the Microservices World - DZone Microservices

The software industry has come a long journey and throughout this journey, Software Architecture has evolved a lot. Starting with 1-tier (Single-node), 2-tier (Client/ Server), 3-tier, and Distributed are some of the Software Architectural patterns we saw in this journey.

The Problem

The majority of software companies are moving from Monolithic architecture to Microservices architecture, and Microservices architecture is taking over the software industry day-by-day. While monolithic architecture has many benefits, it also has so many shortcomings when catering to modern software development needs. With those shortcomings of monolithic architecture, it is very difficult to meet the demand of the modern-world software requirements and as a result, microservices architecture is taking control of the software development aggressively. The Microservices architecture enables us to deploy our applications more frequently, independently, and reliably meeting modern-day software application development requirements.

#microservice architecture #istio #microservice best practices #linkerd #microservice communication #microservice design #envoy proxy #kubernetes architecture #api gateways #service mesh architecture

Oral  Brekke

Oral Brekke

1621848840

How to Choreoghraph Event-Driven Microservices

Are you trying to claw your way out of the web of API calls that ties your microservices together? Does a seemingly innocent change or bug fix result in a ripple effect across several business serving capabilities? Well, you’re not alone.

Microservices have been gaining steam since their introduction as an architectural style in 2011. Initially pioneered by companies like Amazon and Netflix as an alternative to their exploding monolithic codebases, they are now increasingly popular even at companies operating at a much smaller scale than the behemoths. And with good reason. When designed well, microservices are a great alternative to the problems often seen with their monolithic counterparts. The key phrase there though is “when designed well”. It’s simple enough when you have ten microservices - so scalable, such fun! But when those quickly grow to 50, and then 100, and then 500, you have a real problem on hand if you haven’t paid attention to how they all talk to each other.

Imagine you have several microservices all communicating via API calls. In this web of tightly coupled services, changes to one service may necessitate corresponding changes across multiple other services, and scaling one service would necessitate scaling a number of others as well. This problem was first described as the Death Star Architecture.

In the world of microservices, Death Star architecture is an anti-pattern where poorly designed microservices become highly interdependent, forming a complex network of interservice communication. When this happens, the entire thing becomes slow, inflexible, and fragile – and easy to blow up. ( ref .)

At this point, you’ve lost many of the benefits of having microservices in the first place, and in reality, are left with a distributed monolith

So how do you avoid the Death Star Architecture trap and allow your microservices to scale? How do you keep your microservice relatively isolated but still remain an integral node in the set of business flows that it serves? Enter the Event-Driven Microservice Architecture. The golden rule of Event-Driven microservices is that all communication is asynchronous. No API calls for us! Microservices instead publish records of their doings, also known as events. An Event is a record of a business action and must contain all information relevant to that action. Events are published to messaging infrastructure (think Kafka, RabbitMQ) and it is left to consuming microservices to figure out how to operate on them. By removing this tight coupling between services, it’s possible to truly reap the benefits offered by the microservices architecture pattern.

Event-Driven Messaging comes in two flavors - choreography and orchestration.

What is the choreography pattern?

Choreography is pretty much what it sounds like! Each dancer in a ballet troupe knows their position and performs their routine based on musical cues. Choreographed microservices behave in the same way - each service (dancer) is aware of it’s place in the business flow and acts on certain cues (events).

Let’s look at a simplified example of an order processing flow. The customer completes checking out their cart and the following steps need to happen next

  1. An order needs to be created
  2. An email with the details of the order needs to be sent to the customer
  3. Inventory needs to be decreased
  4. A hold needs to be placed on the customer’s credit card

#event-driven #microservices #microservice-architecture #scaling

Carmen  Grimes

Carmen Grimes

1594083603

Event-Driven Architecture as a Strategy

Event-driven architecture, or EDA, is an integration pattern where applications are oriented around publishing events and responding to events. It provides five key benefits to modern application architecture: scalability, resilience, agility, data sharing, and cloud enabling.

This article explores how EDA fits into enterprise integration, its three styles, how it enables business strategy, its benefits and trade-offs, and the next steps to startan EDA implementation.

Although there are many brokers you can use to publish event messages, the open-source software Apache Kafka has emerged as the market leader in this space. This article is focused on a Kafka-based EDA, but much of the principles here apply to any EDA implementation.


Spectrum of Integration

If asked to describe integration a year ago, I would have said there are two modes: application integration and data integration. Today I’d say that integration is on a spectrum, with data on one end, application on the other end, and event integration in the middle.

A spectrum with event integration on the left, application integration on the right, and event integration in the middle.

The spectrum of integration.

Application integration is REST, SOAP, ESB, etc. These are patterns for making functionality in one application run in response to a request from another app. It’s especially strong for B2B partnership and exposing value in one application to another. It’s less strong for many data use cases, like BI reporting and ML pipelines, since most application integrations wait passively to be invoked by a client, rather than actively pushing data where it needs to go.

Data integration are patterns for getting data from point A to point B, including ETL, managed file transfer, etc. They’re strong for BI reporting, ML pipelines, and other data movement tasks, but weaker than application integration for many B2B partnerships and for applications sharing functionality with each other.

Event integration has one foot in data and the other in application integration, and it largely gets the benefits of both. When one application subscribes to another app’s events, it can trigger application code in response to those events, which feels a bit like an API from application integration. The events triggering this functionality also carry with them a significant amount of data, which feels a bit like data integration.

EDA strikes a balance between the two classic integration modes. Refactoring traditional application integrations into an event integration pattern opens more doors for analytics, machine learning, BI, and data synchronization between applications. It gets the best of application and data integration patterns. This is especially relevant for companies moving towards an operating model of leveraging data to drive new channels and partnerships. If your integration strategy does not unlock your data, then that strategy will fail. But if your integration strategy unlocks data at the expense of an application architecture that’s scalable and agile, then again it will fail. Event integration strikes a balance between both those needs.


Strategy vs. Tactic

EDA often begins with isolated teams as a tactic for delivering projects. Ideally such a project would have a deliberative approach to EDA and a common event message broker, usually cloud-native brokers on AWS, Azure, etc. Different teams select different brokers to meet their immediate needs. They do not consider integration beyond their project scope. Eventually they may face the need for enterprise integration at a later date.

A major transition in EDA maturity happens when the investment in EDA shifts from a project tactic to enterprise strategy via a common event bus, usually Apache Kafka. Events can take a role in the organization’s business and technical innovation across the enterprise. Data becomes more rapidly shareable across the enterprise and also between you and your external strategic partners.


EDA Styles

Before discussing the benefits of EDA, let’s cover the three common styles of EDA: event notification, event-carried state transfer, and event sourcing.

Event notification

This pattern publishes events with minimal information: the event type, time stamps, and a key value like an account number or some other key of the entity that raised the event. This informs subscribers that an event occurred, but if subscribers need any information about how that event changed things (like which fields changed, etc.), it must invoke a data retrieval service from the system of record. This is the simplest form of EDA, but it provides the least benefit.

Event-carried state transfer

In this pattern the events carry all information about the state change, typically a before and after image. Subscribing systems can then store their own cache of data without the need to retrieve it from the system of record.

This builds resilience since the subscribing systems can function if the source becomes unavailable. It helps performance, as there’s no remote call required to access source information. For example, if an inventory system publishes the full state of all inventory changes, a sales service subscribing to it can know the current inventory without retrieving from the inventory system — it can simply use the cache it built from the inventory events, even during an inventory service outage.

It also helps performance because the subscriber’s data storage can be custom-tuned just for that subscriber’s unique performance needs. Using the previous example, perhaps the inventory service is best suited using a relational database, but the sales service could get better performance from a no-SQL database like MongoDB. Since the sales services no longer needs to retrieve from the inventory service, it’s at liberty to use a different DBMS than the inventory service. Additionally, if the inventory service is having an outage, the sales service would be unaffected since it pulls inventory data from its local cache.

The cons are that lots of data is copied around and there is more complexity on the receivers since they have to sort out maintaining all the state they are receiving.

Event sourcing

With event-carried state transfer, you saw how the system of record can be reproduced from the event stream. Suppose your broker is a persistent event stream, meaning messages are never deleted. This is a feature that distinguishes Kafka from traditional message brokers. Now the event broker could be the system of record — that is event sourcing. All applications reconstruct an entity’s current state by replaying the events. This is the most mature version of EDA but it’s also the most difficult to implement and manage.

It creates a strong audit since we can see all events that led to the current state. We can recreate historic states for debugging purposes by replaying the event log up to a prior point in time. It also enables ephemeral working copies of application state, such as in-memory databases and caches that can be recreated on the fly by replaying the event stream. This can become the backbone for microservices, CQRS, or the saga data consistency pattern.

A challenge is that clients need to deal with changes in the event schema over time. Even if the schema changed several years ago, you can’t replay that event stream unless your code can still interpret the old, obsolete schema. There may also be challenges in physically deleting data for legal compliance purposes since event streams are usually immutable other than appending new events. Very few companies embrace event sourcing due to these challenges.

Reading event stream data can be challenging. With databases, you don’t see incomplete transactions due to commit/rollback functionality. Materializing data from an event stream could include an incomplete transaction. For example, take a purchase that’s been requested but it’s not yet been determined that there’s enough inventory for it to be successful. The purchase event will be in the event stream, even though we don’t yet know if the purchases will succeed or fail.

Also, the fact that you must materialize the stream into a database means a bug in the code for materializing the stream into a database could result in bad data (though to be fair, fixing the bug, purging the database, and re-materializing the stream would instantly correct the bad data.)

#microservices #integration #event-driven-architecture #enterprise-architecture #information-technology