Seamus  Quitzon

Seamus Quitzon

1599094027

Distributed Saga and Resiliency of Microservices

Sagas are typically used for modeling long-lived transactions like those involved in workflows. It is not advisable to use two-phase transaction protocols to control long-lived transactions since the locking of resources for prolonged durations across trust boundaries is not practical, rather is not at all advisable. Sagas are similar to nested transactions. In a nested transaction, atomic transactions are embedded in other transactions. In sagas, each of these transactions has a corresponding compensating transaction. While a Saga proceeds with its steps, if any of the transactions in a saga fails, the compensating actions for each transaction that was successfully run previously will be invoked so as to nullify the effect of the previously successful transactions.

Modeling a Saga and setting up a Saga Infrastructure is rather straight forward in Local and simple deployments, however when we want to scale out in public cloud environments, and that too with multiple instances of the same type of microservice, there comes a new list of challenges. We will look at few of them in this discussion.

Setting the Context for Saga

We will not attempt to explain what a Saga is in more details here since it’s covered well in the DZone article titled “Distributed Sagas for Microservices.” The sample described there is an apt business case for our discussion, and we will use that in our explanation here. The scenario is that of doing a Travel booking. Many Travel agent booking systems provide this feature where they aggregate multiple kinds of travel inventories from different enterprises who actually own the inventory. It’s very unlikely that a single enterprise owns inventory for all these resources, but it’s highly likely that the end-user wants to book one or more of these resources in a single transaction because a confirmed hotel booking with a non-confirmed flight booking is not very useful for him!

In our case, The Travel agent enterprise refers to www.makeyourtrip.com. Travelers can come to this web site and reserve a complete travel package, which involves booking inventories of multiple types.

Let’s assume that makeyourtrip has got a partnership with 3 other enterprises for retrieving inventories, as follows:

The microservices for business operations for above enterprises can be portrayed as illustrated in Figure 01.

Figure 01 Orchestration based Saga using HTTP

We will use a variant of the Hexagonal Architecture representation to depict each microservice. Figure 01 shows such a representation, and we have 4 such microservices each corresponding to the 4 enterprises mentioned earlier. A booking request from a traveler will come through the internet and hits the travel agent application first. We will assume that all that is required as input parameters to carry out a complete travel booking is captured in a single request from the traveler’s user agent device by the travel agent’s microservice, which is Trip microservice.

Referring to Figure 01, let us look at a typical sequence of actions:

  1. Traveler sends a “Book Trip” request from the browser, which will hit the Trip microservice.
  2. The Trip microservice is responsible for starting Saga. It calls on what is called as a Saga coordinator endpoint, starting a Saga. The coordinator announces a Saga identifier in response. The Trip microservice enlists itself with the created Saga by calling the Saga coordinator providing the Saga identifier and handing over addresses of REST endpoints for compensation (optionally confirmation) callbacks. Those are endpoint the coordinator can call back in response to the outcome of the execute action on any participating microservices.
  3. The Trip microservice takes the Saga ID and adds it as an HTTP header to the REST call to the 3 other fulfilling microservices, Cab Microservice, Hotel Microservice and Flight Microservice.
  4. The called microservices distinguishes the Saga and they can enlist themselves (by announcing REST endpoints for compensation/confirmation callbacks) to Saga coordinator.
  5. The participant microservices, viz. Cab Microservice, Hotel Microservice and Flight Microservice executes the business process
  6. Any of the participant microservices could fail the Saga by calling “Execution Failure” on the Saga coordinator.
  7. Saga coordinator sends commands either to confirm or to compensate, to all participants
  8. On way back, the initiator microservice is responsible for finishing the Saga by calling complete (with a success or failure flag) on the Saga coordinator with the saga identifier.

#travel booking #microservice architecture #orchestration #distributed transactions #saga pattern #kafka partitions #choreography #cqrs in php

What is GEEK

Buddha Community

Distributed Saga and Resiliency of Microservices
Seamus  Quitzon

Seamus Quitzon

1599094027

Distributed Saga and Resiliency of Microservices

Sagas are typically used for modeling long-lived transactions like those involved in workflows. It is not advisable to use two-phase transaction protocols to control long-lived transactions since the locking of resources for prolonged durations across trust boundaries is not practical, rather is not at all advisable. Sagas are similar to nested transactions. In a nested transaction, atomic transactions are embedded in other transactions. In sagas, each of these transactions has a corresponding compensating transaction. While a Saga proceeds with its steps, if any of the transactions in a saga fails, the compensating actions for each transaction that was successfully run previously will be invoked so as to nullify the effect of the previously successful transactions.

Modeling a Saga and setting up a Saga Infrastructure is rather straight forward in Local and simple deployments, however when we want to scale out in public cloud environments, and that too with multiple instances of the same type of microservice, there comes a new list of challenges. We will look at few of them in this discussion.

Setting the Context for Saga

We will not attempt to explain what a Saga is in more details here since it’s covered well in the DZone article titled “Distributed Sagas for Microservices.” The sample described there is an apt business case for our discussion, and we will use that in our explanation here. The scenario is that of doing a Travel booking. Many Travel agent booking systems provide this feature where they aggregate multiple kinds of travel inventories from different enterprises who actually own the inventory. It’s very unlikely that a single enterprise owns inventory for all these resources, but it’s highly likely that the end-user wants to book one or more of these resources in a single transaction because a confirmed hotel booking with a non-confirmed flight booking is not very useful for him!

In our case, The Travel agent enterprise refers to www.makeyourtrip.com. Travelers can come to this web site and reserve a complete travel package, which involves booking inventories of multiple types.

Let’s assume that makeyourtrip has got a partnership with 3 other enterprises for retrieving inventories, as follows:

The microservices for business operations for above enterprises can be portrayed as illustrated in Figure 01.

Figure 01 Orchestration based Saga using HTTP

We will use a variant of the Hexagonal Architecture representation to depict each microservice. Figure 01 shows such a representation, and we have 4 such microservices each corresponding to the 4 enterprises mentioned earlier. A booking request from a traveler will come through the internet and hits the travel agent application first. We will assume that all that is required as input parameters to carry out a complete travel booking is captured in a single request from the traveler’s user agent device by the travel agent’s microservice, which is Trip microservice.

Referring to Figure 01, let us look at a typical sequence of actions:

  1. Traveler sends a “Book Trip” request from the browser, which will hit the Trip microservice.
  2. The Trip microservice is responsible for starting Saga. It calls on what is called as a Saga coordinator endpoint, starting a Saga. The coordinator announces a Saga identifier in response. The Trip microservice enlists itself with the created Saga by calling the Saga coordinator providing the Saga identifier and handing over addresses of REST endpoints for compensation (optionally confirmation) callbacks. Those are endpoint the coordinator can call back in response to the outcome of the execute action on any participating microservices.
  3. The Trip microservice takes the Saga ID and adds it as an HTTP header to the REST call to the 3 other fulfilling microservices, Cab Microservice, Hotel Microservice and Flight Microservice.
  4. The called microservices distinguishes the Saga and they can enlist themselves (by announcing REST endpoints for compensation/confirmation callbacks) to Saga coordinator.
  5. The participant microservices, viz. Cab Microservice, Hotel Microservice and Flight Microservice executes the business process
  6. Any of the participant microservices could fail the Saga by calling “Execution Failure” on the Saga coordinator.
  7. Saga coordinator sends commands either to confirm or to compensate, to all participants
  8. On way back, the initiator microservice is responsible for finishing the Saga by calling complete (with a success or failure flag) on the Saga coordinator with the saga identifier.

#travel booking #microservice architecture #orchestration #distributed transactions #saga pattern #kafka partitions #choreography #cqrs in php

Einar  Hintz

Einar Hintz

1599055326

Testing Microservices Applications

The shift towards microservices and modular applications makes testing more important and more challenging at the same time. You have to make sure that the microservices running in containers perform well and as intended, but you can no longer rely on conventional testing strategies to get the job done.

This is where new testing approaches are needed. Testing your microservices applications require the right approach, a suitable set of tools, and immense attention to details. This article will guide you through the process of testing your microservices and talk about the challenges you will have to overcome along the way. Let’s get started, shall we?

A Brave New World

Traditionally, testing a monolith application meant configuring a test environment and setting up all of the application components in a way that matched the production environment. It took time to set up the testing environment, and there were a lot of complexities around the process.

Testing also requires the application to run in full. It is not possible to test monolith apps on a per-component basis, mainly because there is usually a base code that ties everything together, and the app is designed to run as a complete app to work properly.

Microservices running in containers offer one particular advantage: universal compatibility. You don’t have to match the testing environment with the deployment architecture exactly, and you can get away with testing individual components rather than the full app in some situations.

Of course, you will have to embrace the new cloud-native approach across the pipeline. Rather than creating critical dependencies between microservices, you need to treat each one as a semi-independent module.

The only monolith or centralized portion of the application is the database, but this too is an easy challenge to overcome. As long as you have a persistent database running on your test environment, you can perform tests at any time.

Keep in mind that there are additional things to focus on when testing microservices.

  • Microservices rely on network communications to talk to each other, so network reliability and requirements must be part of the testing.
  • Automation and infrastructure elements are now added as codes, and you have to make sure that they also run properly when microservices are pushed through the pipeline
  • While containerization is universal, you still have to pay attention to specific dependencies and create a testing strategy that allows for those dependencies to be included

Test containers are the method of choice for many developers. Unlike monolith apps, which lets you use stubs and mocks for testing, microservices need to be tested in test containers. Many CI/CD pipelines actually integrate production microservices as part of the testing process.

Contract Testing as an Approach

As mentioned before, there are many ways to test microservices effectively, but the one approach that developers now use reliably is contract testing. Loosely coupled microservices can be tested in an effective and efficient way using contract testing, mainly because this testing approach focuses on contracts; in other words, it focuses on how components or microservices communicate with each other.

Syntax and semantics construct how components communicate with each other. By defining syntax and semantics in a standardized way and testing microservices based on their ability to generate the right message formats and meet behavioral expectations, you can rest assured knowing that the microservices will behave as intended when deployed.

#testing #software testing #test automation #microservice architecture #microservice #test #software test automation #microservice best practices #microservice deployment #microservice components

Tia  Gottlieb

Tia Gottlieb

1597438200

What Is a Microservice Architecture? Why Is It Important Now?

We have been building software applications for many years using various tools, technologies, architectural patterns and best practices. It is evident that many software applications become large complex monolith over a period for various reasons. A monolith software application is like a large ball of spaghetti with criss-cross dependencies among its constituent modules. It becomes more complex to develop, deploy and maintain monoliths, constraining the agility and competitive advantages of development teams. Also, let us not undermine the challenge of clearing any sort of technical debt monoliths accumulate, as changing part of monolith code may have cascading impact of destabilizing a working software in production.

Over the years, architectural patterns such as Service Oriented Architecture (SOA) and Microservices have emerged as alternatives to Monoliths.

SOA was arguably the first architectural pattern aimed at solving the typical monolith issues by breaking down a large complex software application to sub-systems or “services”. All these services communicate over a common enterprise service bus (ESB). However, these sub-systems or services are actually mid-sized monoliths, as they share the same database. Also, more and more service-aware logic gets added to ESB and it becomes the single point of failure.

Microservice as an architectural pattern has gathered steam due to large scale adoption by companies like Amazon, Netflix, SoundCloud, Spotify etc. It breaks downs a large software application to a number of loosely coupled microservices. Each microservice is responsible for doing specific discrete tasks, can have its own database and can communicate with other microservices through Application Programming Interfaces (APIs) to solve a large complex business problem. Each microservice can be developed, deployed and maintained independently as long as it operates without breaching a well-defined set of APIs called contract to communicate with other microservices.

#microservice architecture #microservice #scaling #thought leadership #microservices build #microservice

Autumn  Blick

Autumn Blick

1595338835

Microservices and Data Management - DZone Microservices

Introduction

For pure frontend developers who doesn’t have much exposure to backend or middleware technology, microservices are a vague thing. They might have high-level introduction. So, let us have some deep understanding of what microservices are, and how it is different from monolithic application data management.

Monolithic and Microservice

In a monolithic application, all the stakeholders like all the business logic, routing features, middle-wares and Database access code get used to implement all the functionalities of the application. It is basically a single unit application. It has a lot of challenges in terms of scalability and agility. On the other side, in a microservice, all the business logic, routing features, middle-wares, and database access code get used to implement a single functionality of the application. We break down the functionalities to the core level and then connect to related services. So, the functionalities are actually dependent on related services only and does not get affected if there is an issue with other services. This helps to make the application agile, flexible, and highly scalable.

Monolithic architecture

Microservices Architecture

Why Microservices

Independent DB for the Services

The very first important thing associated with microservices is that each functionality requires its own database and never connects to the database of other services. In a monolithic service, since you have a single database. if something goes wrong with it then the whole application gets crashed. But in microservice, since we have an independent database for each service, in case of any problem with any particular database, it certainly does not affect other services and your application does not crash as a whole.

No Dependency on Schema

We have many services in our application and each service requires its own database. Hence, each database has its own schema or structure. But, if any service is connected to other service and shares the data and during development, the source database changes its schema and does not update the dependent services, then the service will not function correctly and may crash. So, there should be no dependency on databases.

Performance

Depending on the nature of service, we choose the appropriate type of DB. Some services are more efficient in specific database. So, creating a single database for all the services in the application might affect performance. In Microservice, since we have individual DB for each of the service, it is quite flexible, independent, and functions efficiently.

Data Management

Unlike the monolithic approach, in microservice, each functionality or service connects to its own database and never gets connected to other database. So, the big question arises of how we communicate between two services. It is quite generic in an application that we require to get some information based on the combination of many service outputs. But as a thumb rule, services dont communicate. Then what is the solution to this issue? Let us see, how data communicates between the services.

#data management #monolith vs microservice #microservices benefits #microservices communication #microservices archiecture

Autumn  Blick

Autumn Blick

1595335187

Microservices and Data Management - DZone Microservices

Introduction

For pure frontend developers who doesn’t have much exposure to backend or middleware technology, microservices are a vague thing. They might have high-level introduction. So, let us have some deep understanding of what microservices are, and how it is different from monolithic application data management.

Monolithic and Microservice

In a monolithic application, all the stakeholders like all the business logic, routing features, middle-wares and Database access code get used to implement all the functionalities of the application. It is basically a single unit application. It has a lot of challenges in terms of scalability and agility. On the other side, in a microservice, all the business logic, routing features, middle-wares, and database access code get used to implement a single functionality of the application. We break down the functionalities to the core level and then connect to related services. So, the functionalities are actually dependent on related services only and does not get affected if there is an issue with other services. This helps to make the application agile, flexible, and highly scalable.

Monolithic architecture

Microservices Architecture

Why Microservices

Independent DB for the Services

The very first important thing associated with microservices is that each functionality requires its own database and never connects to the database of other services. In a monolithic service, since you have a single database. if something goes wrong with it then the whole application gets crashed. But in microservice, since we have an independent database for each service, in case of any problem with any particular database, it certainly does not affect other services and your application does not crash as a whole.

No Dependency on Schema

We have many services in our application and each service requires its own database. Hence, each database has its own schema or structure. But, if any service is connected to other service and shares the data and during development, the source database changes its schema and does not update the dependent services, then the service will not function correctly and may crash. So, there should be no dependency on databases.

Performance

Depending on the nature of service, we choose the appropriate type of DB. Some services are more efficient in specific database. So, creating a single database for all the services in the application might affect performance. In Microservice, since we have individual DB for each of the service, it is quite flexible, independent, and functions efficiently.

Data Management

Unlike the monolithic approach, in microservice, each functionality or service connects to its own database and never gets connected to other database. So, the big question arises of how we communicate between two services. It is quite generic in an application that we require to get some information based on the combination of many service outputs. But as a thumb rule, services dont communicate. Then what is the solution to this issue? Let us see, how data communicates between the services.

#data management #monolith vs microservice #microservices benefits #microservices communication #microservices archiecture