Handle large traffic, improved application performance, and responsiveness

During an API call from a client, the server creates and assigns a thread of that request. This thread waits and is appointed until requests are served to that client.

Each server can open a limited number of threads, and if that number exceeds, then new clients request will store in the request queue of that server.

For the Tomcat server, its default number of threads maxThreads is 200, representing the maximum number of concurrent threads allowed to run at any given time.

There are other metrics, maxConnections, representing the total number of concurrent connections that the server will accept and process. Any additional incoming connections will be placed in a queue until a thread becomes available. The default value for NIO/NIO2 mode is 10,000 and APR/Native is 8,192.

Another metrics is acceptCount, which represents the maximum number of TCP requests that can wait in a queue at the OS level when there are no worker threads available. The default value is 100.

For a high traffic application, If a request takes much time or takes a long time to serve, it will create a huge problem. To overcome this thread management problem, we can use asynchronous programming.

According to visualstudiomagazine.com, the definition of asynchronous programming is

Asynchronous programming is a means of parallel programming in which a unit of work runs separately from the main application thread and notifies the calling thread of its completion, failure or progress.

How synchronous applications create problems?

…

#java #asynchronous-programming #spring #servers #api #writing asynchronous non-blocking rest api using java.