Cyclic api calls in microservices

Assume there are two services,

Assume there are two services,

  1. Product Service
  2. Coupon service

Coupon is tagged against the product. And when the coupon is changed for any product there is an API call from product service to the coupon service which calculates the coupon discount and sends back to the product service. And product service maintains that cache. However, To calculate the discount, coupon service need the product information such as product category, SKU type etc. and discount varies based on that so there is again an API call to the product service making it cyclic call. There are a couple of solutions,

  1. maintain product cache at the coupon service. - This is an overhead as we need to maintain the cache on every product field update and this is not the solution we are looking for.
  2. When product service makes an API call to the coupon service, send all the fields it needs - There is tight coupling in this approach.

What is the correct way to solve this problem?

What is REST? What are RESTful Web Services?

What is REST? What are RESTful Web Services?

This tutorial provides an introduction to RESTful web services and goes over what REST is as well as HTTP.

REST stands for REpresentational State Transfer. It is a popular architectural approach to create your API's in today's world.

You Will Learn
  • What is REST?
  • What are the fundamentals of REST APIs?
  • How do you make use of HTTP when building REST API?
  • What is a Resource?
  • How do you identify REST API Resources?
  • What are some of the best practices in designing REST API?
What Is REST?

The acronym REST stands for REpresentational State Transfer. It was term originally coined by Roy Fielding, who was also the inventor of the HTTP protocol. The striking feature of REST services is that they want to make the best use of HTTP. Let's now have a quick overview of HTTP.

A Relook at HTTP

Let's open up the browser and visit a web page first:

And then click on one of the result pages:

Next, we can click on the link on the page we end up in:

And land upon another page:

This is how we typically browse the web.

When we browse the internet, there are a lot of things that happen behind the scenes. The following is a simplified view of what happens between the browser, and the servers running on the visited websites:

The HTTP Protocol

When you enter a URL such as https://www.google.com in the browser, a request is sent to the server on the website identified by the URL. That server then responds with a response. The important thing is the formats of these requests and responses. These formats are defined by a protocol called HTTPHyper Text Transfer Protocol.

When you type in a URL at the browser, it sends out a GET request to the identified server. The server then replies with an HTTP response that contains data in HTMLHyper Text Markup Language. The browser then takes this HTML and displays it on your screen.

Let's say you are filling in a form present on a web page with a list of details. In such a scenario when you click the Submit button, an HTTP POST request gets sent out to the server.

HTTP and RESTful Web Services

HTTP provides the base layer for building web services. Therefore, it is important to understand HTTP. Here are a few key abstractions.

Resource

A resource is a key abstraction that HTTP centers round. A resource is anything you want to expose to the outside world through your application. For instance, if we write a todo management application, instances of resources are:

  • A specific user
  • A specific todo
  • A list of todos

Resource URIs

When you develop RESTful services, you need to focus your thinking on the resources in the application. The way we identify a resource to expose, is to assign a URIUniform Resource Identifier — to it. For example:

  • The URI for the user Ranga is /user/ranga
  • The URI for all the todos belonging to Ranga is /user/Ranga/todos
  • The URI for the first todo that Ranga has is /user/Ranga/todos/1

Resource Representation

REST does not worry about how you represent your resource. It could be XML, HTML, JSON, or something entirely different! The only important thing is you clearly define your resource and perform whatever actions that are supported on it by making use of features already provided by HTTP. Examples are:

  • Create a user: POST /users
  • Delete a user: DELETE /users/1
  • Get all users: GET /users
  • Get a single user: GET /users/1
REST and Resources

A significant point to note is that with REST, you need to think about your application in terms of resources:

  • Identify what resources you want to expose to the outside world
  • Make use of the verbs already specified by HTTP to perform operations on these resources

Here is how a REST service is generally implemented:

  • Data Exchange Format: No restriction is imposed over here. JSON is a highly popular format, although other such as XML can be used as well
  • Transport: Always HTTP. REST is completely built on top of HTTP.
  • Service Definition: There is no standard to specify this, and REST is flexible. This could be a drawback in some scenarios, as it might be necessary for the consuming application to understand the request and response formats. There are widely used ones however, such as WADL (Web Application Definition Language) and Swagger.

REST focuses on resources and how effectively you perform operations on them using HTTP.

The Components of HTTP

HTTP defines the following for a request:

For the response, HTTP defines the:

HTTP Request Methods

The method used in a HTTP request indicates what action you want to perform with that request. Important examples are:

  • GET: Retrieve details of a resource
  • POST : Create a new resource
  • PUT: Update an existing resource
  • DELETE: Delete a resource

HTTP Response Status Code

A status code is always present in a HTTP response. Common examples are:

  • 200: Success
  • 404: Page not found
Summary

In this article, we had a high-level look at REST. We stressed the fact that HTTP is the building block of REST services. HTTP is a protocol that is used to define the structure of browser requests and responses. We saw that HTTP deals mainly with resources that are exposed on web servers. Resources are identified using URIs, and operations on these resources are performed using verbs defined by HTTP.

Finally, we looked at how REST services make the best use of features offered by HTTP to expose resources to the outside world. REST does not put any restrictions on the resource representation formats or on the service definition.

What is REST API? | Restful Web Service

What is REST API? | Restful Web Service

In this post "Restful Web Service", you'll learn: What is Web services, what is API, What is REST API, How REST works and Implementation of REST API

What is REST API? | Restful Web Service

A REST API defines a set of functions to process requests and responses via HTTP protocol.

REST is used in mobile application as well as in web applications.


Web Service Tutorial: Streaming Data with Spring Boot RESTful

Web Service Tutorial: Streaming Data with Spring Boot RESTful

In this article, we are going to look at an example to download files using StreamingResponseBody. In this approach, data is processed and written in chunks to the OutputStream.

In this article, we are going to look at an example to download files using StreamingResponseBody. In this approach, data is processed and written in chunks to the OutputStream.

Streaming data is a radical new approach to sending data to web browsers which provides for dramatically faster page load times. Quite often, we need to allow users to download files in web applications. When the data is too large, it becomes quite a challenge to provide a good user experience.

Spring offers support for asynchronous request processing via StreamingResponseBody. In this approach, an application can write data directly to the response OutputStream without holding up the Servlet container thread. There are a few other methods in Spring to handle asynchronous request processing.

Setting Up Spring Boot Project

Create a sample Spring Boot application. Here is my sample project structure. I have created the project manually, but you could also create using Spring Intializer.

Project structure

Let us add some basic dependencies to Maven POM.

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>com.techshard.streamingresponse</groupId>
    <artifactId>springboot-download</artifactId>
    <version>1.0-SNAPSHOT</version>
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.1.4.RELEASE</version>
        <relativePath />
    </parent>
    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-security</artifactId>
        </dependency>
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-api</artifactId>
        </dependency>
        <dependency>
            <groupId>org.apache.commons</groupId>
            <artifactId>commons-io</artifactId>
            <version>1.3.2</version>
        </dependency>
    </dependencies>
</project>

We will now create a controller and add an API endpoint for download. Here is my complete controller.

package com.techshard.download.controller;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.http.HttpStatus;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody;
import javax.servlet.http.HttpServletResponse;
import java.io.*;
import java.util.zip.ZipEntry;
import java.util.zip.ZipOutputStream;
@RestController
@RequestMapping ("/api")
public class DownloadController {
    private final Logger logger = LoggerFactory.getLogger(DownloadController.class);
    @GetMapping (value = "/download", produces = MediaType.APPLICATION_JSON_VALUE)
    public ResponseEntity<StreamingResponseBody> download(final HttpServletResponse response) {
        response.setContentType("application/zip");
        response.setHeader(
                "Content-Disposition",
                "attachment;filename=sample.zip");
        StreamingResponseBody stream = out -> {
            final String home = System.getProperty("user.home");
            final File directory = new File(home + File.separator + "Documents" + File.separator + "sample");
            final ZipOutputStream zipOut = new ZipOutputStream(response.getOutputStream());
            if(directory.exists() && directory.isDirectory()) {
                try {
                    for (final File file : directory.listFiles()) {
                        final InputStream inputStream=new FileInputStream(file);
                        final ZipEntry zipEntry=new ZipEntry(file.getName());
                        zipOut.putNextEntry(zipEntry);
                        byte[] bytes=new byte[1024];
                        int length;
                        while ((length=inputStream.read(bytes)) >= 0) {
                            zipOut.write(bytes, 0, length);
                        }
                        inputStream.close();
                    }
                    zipOut.close();
                } catch (final IOException e) {
                    logger.error("Exception while reading and streaming data {} ", e);
                }
            }
        };
        logger.info("steaming response {} ", stream);
        return new ResponseEntity(stream, HttpStatus.OK);
    }
}

In this API endpoint, we are reading multiple files from a directory and creating a zip file. We are executing this process within StreamingResponseBody*. It writes data directly to an OutputStream before passing that written information back to the client using aResponseEntity. *This means that the download process will start immediately on the client, while the server is processing and writing data in chunks.

Start the server and test this endpoint using http://localhost:8080/api/download.

When using StreamingResponseBody, it is highly recommended to configure TaskExecutor used in Spring MVC for executing asynchronous requests. TaskExecutor is an interface that abstracts the execution of a Runnable.

Let us configure the TaskExecutor. Here is the AsyncConfiguration class which configures timeout using WebMvcCofigurer and also registers an interceptor that is called when there's a timeout in case you need some special handling.

package com.techshard.download;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.aop.interceptor.AsyncUncaughtExceptionHandler;
import org.springframework.aop.interceptor.SimpleAsyncUncaughtExceptionHandler;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.task.AsyncTaskExecutor;
import org.springframework.scheduling.annotation.AsyncConfigurer;
import org.springframework.scheduling.annotation.EnableAsync;
import org.springframework.scheduling.annotation.EnableScheduling;
import org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor;
import org.springframework.web.context.request.NativeWebRequest;
import org.springframework.web.context.request.async.CallableProcessingInterceptor;
import org.springframework.web.context.request.async.TimeoutCallableProcessingInterceptor;
import org.springframework.web.servlet.config.annotation.AsyncSupportConfigurer;
import org.springframework.web.servlet.config.annotation.WebMvcConfigurer;
import java.util.concurrent.Callable;
@Configuration
@EnableAsync
@EnableScheduling
public class AsyncConfiguration implements AsyncConfigurer {
    private final Logger log = LoggerFactory.getLogger(AsyncConfiguration.class);
    @Override
    @Bean (name = "taskExecutor")
    public AsyncTaskExecutor getAsyncExecutor() {
        log.debug("Creating Async Task Executor");
        ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
        executor.setCorePoolSize(5);
        executor.setMaxPoolSize(10);
        executor.setQueueCapacity(25);
        return executor;
    }
    @Override
    public AsyncUncaughtExceptionHandler getAsyncUncaughtExceptionHandler() {
        return new SimpleAsyncUncaughtExceptionHandler();
    }
    /** Configure async support for Spring MVC. */
    @Bean
    public WebMvcConfigurer webMvcConfigurerConfigurer(AsyncTaskExecutor taskExecutor, CallableProcessingInterceptor callableProcessingInterceptor) {
        return new WebMvcConfigurer() {
            @Override
            public void configureAsyncSupport(AsyncSupportConfigurer configurer) {
                configurer.setDefaultTimeout(360000).setTaskExecutor(taskExecutor);
                configurer.registerCallableInterceptors(callableProcessingInterceptor);
                WebMvcConfigurer.super.configureAsyncSupport(configurer);
            }
        };
    }
    @Bean
    public CallableProcessingInterceptor callableProcessingInterceptor() {
        return new TimeoutCallableProcessingInterceptor() {
            @Override
            public <T> Object handleTimeout(NativeWebRequest request, Callable<T> task) throws Exception {
                log.error("timeout!");
                return super.handleTimeout(request, task);
            }
        };
    }
}

Conclusion

Using StreamingResponseBody, we can now stream data easily for highly-concurrent applications. I hope you enjoyed this article. Let me know if you have any comments or suggestion in the comments section below.

The example for this article can be found on GitHub repository.