How to Securing REST APIs With Client Certificates

How to Securing REST APIs With Client Certificates

This post is about an example of securing a REST API with a client certificate (a.k.a. X.509 certificate authentication). In other words, a client verifies a server according to its certificate and the server identifies that client according to a client certificate

This post is about an example of securing a REST API with a client certificate (a.k.a. X.509 certificate authentication).

In other words, a client verifies a server according to its certificate and the server identifies that client according to a client certificate (so-called mutual authentication).

In connection with Spring Security, we will be able to perform some additional authentication and authorization.

Technologies used:

  1. Spring Boot 2.0.5.RELEASE
  2. Spring Web + Security 5.0.8.RELEASE
  3. Embedded Tomcat 8.5.34

Quick post overview:

  • Create a simple REST API service (without any security)
  • Create certificates for server and client
  • Configure the server to serve HTTPS content
  • Configure the server to require a client certificate
  • Spring Security for further client authentication and authorization
  • Test our secured REST API
Final Project Structure

Creating a New Base Spring Boot Project

We will start with a new project generated by Spring Initializr. We just need two Spring dependencies, i.e. Spring Web + Spring Security.

All required dependencies are shown here:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-security</artifactId>
</dependency>
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-web</artifactId>
</dependency>
Create a Simple REST API Service

Let's create a simple REST controller serving a detail about a customer using an HTTP GET method:

@RestController
@RequestMapping("/customer")
public class CustomerController {
    @GetMapping("/{id}")
    public Customer GetCustomer(@PathVariable Long id) {
        return new Customer(id, "Customer" + id);
    }
}

Displaying URL http://localhost:8080/customer/1 returns this JSON object:

{
    "id":1,
    "name":"Customer1"
}
Create Certificates for Server and Client

I want to stay focused on securing REST APIs so I will show you how to generate all required files in a very concise way. For more details about commands, visit my other blog post about creating a PKCS #12 key store.

#Create folders to generate all files (separated for client and server)
mkdir ssl && cd ssl && mkdir client && mkdir server
## Server
# Generate server private key and self-signed certificate in one step
openssl req -x509 -newkey rsa:4096 -keyout server/serverPrivateKey.pem -out server/server.crt -days 3650 -nodes
# Create PKCS12 keystore containing private key and related self-sign certificate
openssl pkcs12 -export -out server/keyStore.p12 -inkey server/serverPrivateKey.pem -in server/server.crt
# Generate server trust store from server certificate 
keytool -import -trustcacerts -alias root -file server/server.crt -keystore server/trustStore.jks
## Client
# Generate client's private key and a certificate signing request (CSR)
openssl req -new -newkey rsa:4096 -out client/request.csr -keyout client/myPrivateKey.pem -nodes
## Server
# Sign client's CSR with server private key and a related certificate
openssl x509 -req -days 360 -in request.csr -CA server/server.crt -CAkey server/serverPrivateKey.pem -CAcreateserial -out client/pavel.crt -sha256
## Client
# Verify client's certificate
openssl x509 -text -noout -in client/pavel.crt
# Create PKCS12 keystore containing client's private key and related self-sign certificate 
openssl pkcs12 -export -out client/client_pavel.p12 -inkey client/myPrivateKey.pem -in client/pavel.crt -certfile server/myCertificate.crt

We will use files in the server folder to configure our server.

The final client's file client/client_pavel.p12 can be either imported into your browser or used in another client application.

On Windows, just open this file and import it into your system to test the REST API with any browser.

Configure the Server to Serve HTTPS Content

Basically, there are two options.

You can use any standalone server (e.g. Tomcat, WildFly, etc.) so the configuration would be specific to your choice. I prefer this choice for production environments.

Instead of configuring an application server, I will show you the second, simpler way of using an embedded Tomcat server inside Spring Boot.

The configuration is quite easy, we will change the port to 8443 and configure the server key store generated in the previous steps:

# Define a custom port (instead of the default 8080)
server.port=8443
# The format used for the keystore
server.ssl.key-store-type=PKCS12
# The path to the keystore containing the certificate
server.ssl.key-store=classpath:keyStore.p12
# The password used to generate the certificate
server.ssl.key-store-password=changeit
Configure the Server to Require a Client Certificate

The configuration of any server to require a client certificate (i.e. the mutual authentication) is very similar to the server side configuration except using words like a trust store instead of a key store.

So the embedded Tomcat configuration seems like:

# Trust store that holds SSL certificates.
server.ssl.trust-store=classpath:trustStore.jks
# Password used to access the trust store.
server.ssl.trust-store-password=changeit
# Type of the trust store.
server.ssl.trust-store-type=JKS
# Whether client authentication is wanted ("want") or needed ("need").
server.ssl.client-auth=need

The embedded server now ensures (without any other configuration) that the clients with a valid certificate only are able to call our REST API.

Other clients will be declined by the server due to being unable to make correct SSL/TLS handshake (required by mutual authentication).

Please note that all configuration items starting server.* are related to an embedded Tomcat server only. You do not need it when using any standalone application server.

Spring Security for Further Client Authentication and Authorization

It would be fine to get an incoming client for our application as a logged user.

That gives us the possibility to perform some other authentications and authorizations using Spring Security (e.g. securing method calls to specific roles).

Until now, no Spring Security was needed, but all clients with any valid certificate may perform any call in our application without knowing who the caller is.

So we must configure Spring Security to create a logged user using a username from a client certificate (usually from the CN field, see the method call subjectPrincipalRegex):

@Configuration
@EnableWebSecurity
@EnableGlobalMethodSecurity(securedEnabled = true)
public class SecurityConfig extends WebSecurityConfigurerAdapter {
    @Override
    protected void configure(HttpSecurity http) throws Exception {
        http.authorizeRequests()
                .anyRequest().authenticated().and()
                .x509()
                    .subjectPrincipalRegex("CN=(.*?)(?:,|$)")
                    .userDetailsService(userDetailsService());
    }
    @Bean
    public UserDetailsService userDetailsService() {
        return (UserDetailsService) username -&amp;amp;amp;amp;gt; {
            if (username.equals("pavel")) {
                return new User(username, "",
                        AuthorityUtils
                                .commaSeparatedStringToAuthorityList("ROLE_USER"));
            } else {
                throw new UsernameNotFoundException(String.format("User %s not found", username));
            }
        };
    }
}

Using the bean UserDetailsService is a kind of fake, but it shows an example of an additional authentication to accept only the username "pavel".

In other words, it accepts a client with a certificate containing the value "pavel" only in the certificate's CN field (as mentioned before, configured with subjectPrincipalRegex).

As you might have noticed, only the user "pavel" is a member of the role "user", so now we are able to restrict method calls to specific roles:

 @GetMapping("/{id}")
    @Secured("ROLE_USER")
    public Customer GetCustomer(@PathVariable Long id) {
        return new Customer(id, "Customer" + id);
    }
Test Secured REST API

When you successfully import client/client_pavel.p12 into your system and the application runs, you can visit URL https://localhost:8443/customer/1.

The first access of this page displays a window to select the correct certificate to authenticate with the server: 

When you submit an incorrect certificate, you will see the "access denied" page (otherwise JSON object returned):

Thanks for reading. Originally published on https://dzone.com

Learn about Developing REST APIs

Learn about Developing REST APIs

Building RESTful web services, like other programming skills is part art, part science. As the Internet industry progresses, creating a REST API becomes more ...

This article introduces a set of tools essential to building REST APIs. The tools are platform independent, which means they are applicable to REST APIs built with any technology stack. The goal of this article is to familiarise novice API developers with different stages of API development and introduce tools that help with those stages. Detailed coverage of these tools can be found on the web. The different phases of API development are enumerated below.

  1. Design — The main goal here is to define the shape of APIs, document interfaces, and provide stub endpoints.
  2. Testing — Here, we do functional testing of APIs by sending a request and analyzing the response at different levels of visibility, namely, application, HTTP, and network.
  3. Web Hosting — When deployed on the web, there are HTTP tools that help with the hosting of APIs for performance, security, and reliability.
  4. Performance — Before moving on to production, we use tools for performance testing of APIs that tell us how much load APIs may support.
  5. Observability — Once the API is deployed in production, testing in production provides the overall health of live APIs and alert us if any problem occurs.
  6. Management — Lastly, we will take a look at some of the tools for API management activities like traffic shaping, blue-green deployment, canary, etc.

The following figure shows different stages highlighting the tools.

We will illustrate the usage of tools on APIs exposed by a web application as we elaborate on each phase of API development. Product Catalog is a Spring Boot web application that manages a catalog of products. It exposes REST APIs to perform CRUD operations on a product catalog.

Design

In the design phase, the API developer collaborates with clients of the API and the data provider to arrive at the shape of the API. REST API essentially consists of exchanging JSON messages over HTTP. JSON is a dominant format in REST API since it is a compact, easy to understand, and has a flexible format that does not require declaring schema up front. Different clients can use the same API and read the data that they need.

We will illustrate API design using Swagger. It is a tool that uses open format to describe the APIs coupled with Web UI for visualizing and sharing. There is no separation between design and implementation. It is an API documentation tool where the documentation is hosted alongside the API. The benefit of this is that the API and the documentation will also remain in sync. The drawback is that only API developers can change the structure of the API. The documentation is generated from the API. This means we need to build the skeleton of our API first. We have used Spring Boot to develop the API and Springfox package to generate the swagger documentation. Bring in swagger 2 and swagger-ui maven dependencies into your pom.xml.

<dependency>
    <groupId>io.springfox</groupId>
    <artifactId>springfox-swagger2</artifactId>
    <version>2.6.1</version>
</dependency>
<dependency>
    <groupId>io.springfox</groupId>
    <artifactId>springfox-swagger-ui</artifactId>
    <version>2.5.0</version>
</dependency>

Add SwaggerConfig.java to the project with following content.

package com.rks.catalog.configuration;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import springfox.documentation.builders.PathSelectors;
import springfox.documentation.builders.RequestHandlerSelectors;
import springfox.documentation.spi.DocumentationType;
import springfox.documentation.spring.web.plugins.Docket;
import springfox.documentation.swagger2.annotations.EnableSwagger2;
@Configuration
@EnableSwagger2
public class SwaggerConfig {
    @Bean
    public Docket api() {
        return new Docket(DocumentationType.SWAGGER_2)
        .select()
        .apis(RequestHandlerSelectors.any())
        .paths(PathSelectors.any()).build();
    }
}

This configuration tells Swagger to scan all the controllers and include all the URLs defined in those controllers for API documentation.

Once the application is started, Swagger documentation of the APIs can be accessed at the URL

http://localhost:8080/swagger-ui.html

Click on each API to examine the details — the URL, HTTP headers, and the HTTP body where applicable. A useful feature is the "Try it out!" button, which provides a sandbox environment that lets people play with the API to get a feel for it before they start plugging them in their apps.

Testing

Functional testing of REST APIs entails sending HTTP requests and checking responses so that we can verify that APIs behave as we expect. REST uses HTTP for transport that specifies the request and response formats of API. TCP/IP, in turn, takes the HTTP messages and decides how to transport them over the wire. We introduce three sets of tools to test APIs at these three layers of protocol stack, namely, REST Clients for REST layer, Web Debuggers for HTTP layer, and Packet Sniffers for TCP/IP layer.

  • Postman — Postman is a REST client that allows us to test REST APIs. It allows us to:
  • Create HTTP requests and generate equivalent cURL commands that can be used in scripts.
  • Create multiple environments for Dev, Test, Pre-Prod as each environment has different configurations.
  • Create a test collection having multiple tests for each product area. The different parts of a test can be parameterized that allows us to switch between environments.
  • Create code snippets in JavaScript to augment our tests, e.g., assert return codes or set an environment variables.
  • Automate running of tests with a command-line tool called Newman.
  • Import/export test collections and environments.

  • cURL — It is a command-line tool that uses it's own HTTP stack and is available cross platform.
curl -X POST \
  http://localhost:8080/books \
  -H 'Cache-Control: no-cache' \
  -H 'Content-Type: application/json' \
  -d '{
"id":"1",
"author":"shakespeare",
"title":"hamlet"
}'
  • Burp — Burp is a HTTP debugger that let us see the web traffic that goes between the client and the API. It runs as a proxy between the client and the server. This allows us to intercept the request and the reponse and modify them to create scenarios that are otherwise difficult to test without changing the client. It is a suite of tools that is mainly used for security testing but it can be very useful for API testing as well. Set up your postman to send request to Burp proxy and configure Burp to intercept client request and server response. Intercept request and response as shown below.

  • Wireshark — Verification of some features of API, e.g., encryption, compression, etc., will require us to look a level deeper to see what is being sent and received on the network. Wireshark is a tool that monitors network interface and keeps a copy of all TCP packets that pass through it. Traffic is split by layers — HTTP, TCP, IP, etc. It also helps us to troubleshoot issues that require us to go deeper, e.g., TLS handshake.

Web Hosting

In this section, we will look at some of the features of the HTTP protocol that, if properly used, help us deliver performant, highly available, robust, and secure APIs. In particular, we will cover three parts of HTTP protocol — Caching for performance, DNS for high availability and scalability, and TLS for transport security.

  • Caching — Caching is one of the best ways to improve client performance and reduce load on API. HTTP allows clients to save a copy of resource locally by sending a caching header in the response. Next time, the client sends HTTP request for the same resource, it will be served from the local cache. This saves both network traffic and compute load on the API.
  • HTTP 1.0 Expiration Caching. HTTP 1.0 provides Expires header in the HTTP response indicating the time when the resource will expire. This can be useful for shared resource with a fixed expiration time.
  • HTTP 1.1 Expiration Caching. HTTP 1.1 provides a more flexible expiration header cache-control that instructs a client to cache the resource for a period that is set in max-age value. There is another value s-maxage that can be set for the intermediaries, e.g., a caching proxy.
  • HTTP Validation Caching. With caching, there is a problem of a client having an out-dated resource or two clients to have different versions of the same resource. If this is not acceptable or if there are personalized resources that cannot be cached, e.g., auth tokens, HTTP provides validation caching. With validation caching, HTTP provides headers in the response Etag or last-modified timestamp. If API returns either of the two headers, clients cache it and include in subsequent GET calls to the API.
GET http://api.endpoint.com/books
If-none-match: "4v44ffgg1e"

If the resource is not changed, the API will return 304 Not Modified response with no body, and the client can safely use its cached copy.

  • DNS — Domain Name System finds IP addresses for a domain name so that clients can route their request to the correct server. When HTTP request is made, clients first query a DNS server to find the address for the host and then send the request directly to the IP address. DNS is a multi-tiered system that is heavily cached to ensure requests are not slowed down. Clients maintain a DNS cache, then there are intermediate DNS servers leading all the way to a nameserver. DNS provides CNAME (Canonical Names) to access different parts of the server, e.g., both API and the webserver may be hosted on the same server with two different CNAMEs — api.endpoint.com and www.endpoint.com or CNAMEs may point to different servers. CNAMEs also let us segregate parts of our API. For HTTP GET requests, we can have separate CNAME for static and transactional resources that let us set up a fronting proxy for resources that we know are likely to be cache hits. We can also have a CNAME for HTTP POST requests to separate reads and writes so that we can scale them independently. Or we can provide a fast lane for priority customers.

With advanced DNS like Route53, a single CNAME instead of just pointing to a single server may point to multiple servers. A routing policy may then be configured for weighted routing, latency routing or for fault tolerance.

  • TLS — We can secure our APIs with TLS which lets us serve our request over HTTPS. HTTPS works on the basic security principle of key-pair. To enable HTTPS on our API, we need a certificate on our server that contains public and private key-pair. The server sends a public key to the client, which uses it to encrypt data and the server uses its private key to decrypt it. When the client first connects to an HTTPS endpoint, there is a handshake where client and server agree upon how to encrypt the traffic. They exchange another key unique to the session which is used to encrypt and decrypt data for the life of that session. There is a performance hit during the initial handshake due to the asymmetric encryption, but once the connection is established, symmetric encryption is used which is quite fast.

For proxies to cache the TLS traffic, we have to upload the same certificate that is used to encrypt the traffic. Proxy should be able to decrypt the traffic, save it in its cache and encrypt it with the same certificate and send it to the client. Some proxy servers do not allow this. In such situations, one solution is to have two CNAMEs — one for static cacheable resources over HTTP and for non-cacheable personalized resources, requests over secured TLS channel will be served by the API directly.

Performance

In this section, we will look at tools to load test our API so that we can quantify how much traffic our infrastructure can cope with. The basic idea behind performance testing is to send lots of requests to the API at the same time and see at what point performance degrades and ultimately fails. The answers we look for are:

  • What response times can the API give under different load conditions?
  • How many concurrent requests can the API handle without errors?
  • What infrastructure is required to deliver the desired performance?

loader.io is a cloud-based free load testing service that allows us to stress test our APIs. To get a baseline performance of API, different kinds of load tests can be run with increasing loads, measured by the number of requests per second, to find out performance figures quantified by errors and response times, for

  • Soak test — average load for long periods, e.g., run for 48 hours @1 request per second. This will uncover any memory leaks or other similar latent bugs.
  • Load test — peak load, e.g., run 2K requests per second with 6 instances of API.
  • Stress test — way-over peak load, e.g., run10K requests per second for 10 minutes.

This also lets us decide the infrastructure that will let us deliver API with desired performance numbers and whether our solution scales linearly.

Observability

Once API is deployed in production, it does not mean we can forget about the API. Production deployment kicks off another phase of testing — testing in production that may uncover issues that remained uncaught in earlier phases. Testing in production includes a set of activities clubbed together as observability that includes logging, monitoring, and tracing. The tools for these activities will help us to diagnose and resolve issues found in production.

  • Logging — Logging needs to be done explicitly by the developers using their preferred logging framework and a logging standard. For example, one log statement for every 10 lines of code or more if the code is complex with log levels split as - 60 percent DEBUG, 25 percent INFO, 10 percent WARN and 5 percent ERROR.
  • Monitoring — Monitoring runs at a higher level than logging. While logging explicitly tells us what is going on with the API, monitoring provides the overall health of API using generic metrics exposed by the platform and the API itself. Metrics are typically exposed by an agent deployed on the server or it may be part of the solution and are collected periodically by the monitoring solution deployed remotely.

Diagnostic endpoints may be included in the solution that tells us the overall health of the API.

  • Tracing — Zipkin is a distributed tracing system. It helps gather timing data needed to troubleshoot latency problems in microservice architectures.

Enabling Centralized Logging covers logging and tracing. For monitoring, interesting metrics may be stored in a time-series store like Prometheus and visualized using Grafana.

Management

API Management tools serve as a gateway that provides services that let:

  • API Clients provision themselves by getting API key
  • API Providers configure DNS, caching, throttling policies, API versioning, canarying.

These features and more are available on AWS API Gateway.

Thanks for reading.

Set up Web App with Spring Boot and Spring Security

Set up Web App with Spring Boot and Spring Security

Download the Spring Boot Web App Example Project. Run the Initial Web App. Add Project Dependencies for Your Spring Boot + Spring Security Web App. Understand Your Spring Boot App. Set Up Okta for OAuth 2.0 Single Sign-On. Configure Your Spring Boot App for Single SignOn (SSO) Refine Our Permissions

Developers know that securing web apps can be a pain. Doing it right is tough. The worst part is that “right” is a moving target. Security protocols change. Vulnerabilities are found in dependencies and patches are released. Tons of often complex boilerplate code has to be generated. The software-as-service paradigm has proliferated over the last decade, and while I love reinventing the wheel as much as the next developer (because, clearly, I’m gonna write it better than the yahoo they hired), security is an area where I’m happy to offload this work to specialists. Enter Okta.

In this tutorial, you’re going to use Spring Boot to build a simple web application with a user registration system and a login system. It will have the following features:

  • Login and registration pages
  • Password reset workflows
  • Restricting access according to group membership
Download the Spring Boot Web App Example Project

The first thing you’re going to need is a free Okta account. If you don’t already have one

The next thing will be to download the example project for this tutorial from GitHub.

git clone https://github.com/oktadeveloper/okta-spring-simple-app-example.git spring-app

This project uses Gradle, as the build tool, and the Thymeleaf templating system.

Run the Initial Web App

Once you have downloaded the example code from the GitHub repository, checkout out the Start tag using the following git command: git checkout tags/Start.

The app at this point it not protected at all. There is no authorization or authentication enabled (even though the necessary dependencies are included in the build.gradle file). Go ahead and run the example by opening a terminal and, from the project root directory, running the command ./gradlew bootRun (The bootRun command is a task provided by the Gradle Spring Boot plugin, added to the build.gradle file in the buildscript section at the top of the file).

Navigate to http://localhost:8080 in your favorite browser, and you should see this:

And if you click on the “Restricted” button:

Add Project Dependencies for Your Spring Boot + Spring Security Web App

The project dependencies are defined in the build.gradle file (see below). There’s a lot going on in this file, and this tutorial isn’t going to try and explain the Gradle build system to you. Feel free to check out their documentation. I just want to point out a few things.

First off, notice that we’re including the okta-spring-boot-starter. This project greatly simplifies integrating Okta with your Spring Boot application. It’s entirely possible to use Okta and Spring Boot without this starter. In fact, up to the point where Groups and Roles are introduced, the differences are minor (mostly involve application.yml changes). However, once you start to trying to integrate Groups and Roles, the Okta Spring Boot Starter saves a lot of coding. If you’d like to look a little deeper, take a look at the Okta Spring Boot Starter GitHub project.

The rest of the dependencies deal with Spring and Spring Boot. You’ll notice none of the org.springframework.bootdependencies have version numbers. This is because of some behind-the-scenes magic being done by the Spring io.spring.dependency-management Gradle plugin. The Spring Boot Version is set by the build script property springBootVersion near the top of the build.gradle file. Based on this version number, the Spring dependency management plugin decides what versions of dependencies to include.

We’re also bringing in the org.springframework.boot Gradle plugin, which adds the bootRun task that we’ll use to run the app.

  • spring-boot-starter-security and spring-boot-starter-web are core Spring Boot dependencies.
  • spring-security-oauth2-autoconfigure is required to use the @EnableOAuth2Sso annotation that we use to hook OAuth and Single Sign-On into our app.
  • spring-boot-starter-thymeleaf and thymeleaf-extras-springsecurity4 bring in the Thymeleaf templating system and integrate it with Spring Security.
buildscript {  
   ext {  
      springBootVersion = '2.0.5.RELEASE'  
  }  
   repositories {  
      mavenCentral()  
   }  
   dependencies {  
      classpath("org.springframework.boot:spring-boot-gradle-plugin:${springBootVersion}")  
   }  
}  

apply plugin: 'java'
apply plugin: 'eclipse'
apply plugin: 'org.springframework.boot'
apply plugin: 'io.spring.dependency-management'

group = 'com.okta.springboot'
version = '0.0.1-SNAPSHOT'
sourceCompatibility = 1.8

repositories {
mavenCentral()
}

dependencies {
compile('com.okta.spring:okta-spring-boot-starter:0.6.0')
compile('org.springframework.boot:spring-boot-starter-security')
compile('org.springframework.boot:spring-boot-starter-web')
compile('org.springframework.boot:spring-boot-starter-thymeleaf')
compile('org.thymeleaf.extras:thymeleaf-extras-springsecurity4')
compile('org.springframework.security.oauth.boot:spring-security-oauth2-autoconfigure:2.0.5.RELEASE')
testCompile('org.springframework.boot:spring-boot-starter-test') "
testCompile('org.springframework.security:spring-security-test')
}

/*
This is required to resolve a logging dependency conflict between the
okta-spring-boot-starter and the various spring dependencies.
*/
configurations.all {
exclude group: 'org.springframework.boot', module: 'spring-boot-starter-logging'
exclude group: 'org.springframework.boot', module: 'logback-classic'
}

Understand Your Spring Boot App

The Java web application has only three class files and a few templates. Obviously Spring Boot is doing a lot of heavy hitting going on in the background, but what’s going on in our class files?

The application entry point is in the SpringSimpleApplication class:

@SpringBootApplication
public class SpringSimpleApplication {
public static void main(String[] args) {
SpringApplication.run(SpringSimpleApplication.class, args);
}
}

Two important things are happening here that get things rolling: 1) we use the @SpringBootApplication annotation, and 2) our main method calls the SpringApplication.run() method. This is the entry point to the entire Spring/Spring Boot system.

The SpringSecurityWebAppConfig class is a way to use Java code to configure how Spring Boot handles web app security. Here we use the HttpSecurity object to remove authorization from all endpoints. By default, the Spring Boot behavior is the opposite: all endpoints require authorization.

@Configuration
public class SpringSecurityWebAppConfig extends WebSecurityConfigurerAdapter {

@Override  
protected void configure(HttpSecurity http) throws Exception {
    http.authorizeRequests().anyRequest().permitAll();          
}

}

The @Configuration annotation tells Spring that we are using the class as a source of programmatic configuration, allowing us to override the configure() method.

The last Java class, SimpleAppController, is our only controller object. Controllers in a Spring Boot web application are where URL requests are mapped to Java code. The @Controller annotation tells Spring that this class is a controller.

@Controller
class SimpleAppController {

@RequestMapping("/")  
String home() {  
    return "home";  
}  

@RequestMapping("/restricted")  
String restricted() {  
    return "restricted";  
}  

}

Connections between class methods and URLs are made using the @RequestMapping annotation.

We have two mappings:

  1. “home” mapping
  2. “restricted” mapping

Remember that initially nothing is actually “restricted”, so don’t get confused by that. You’ll lock that mapping down in a bit.

Also notice that the classes return a simple text string, but this is getting auto-magically turned into a full html file. This is part of the Thymeleaf dependency that is included in the build.gradle file. These strings are assumed to be template file names, which are by default paths in the templates directory on the classpath.

Thus “home” is mapped to the src/main/resources/templates/home.html template file. When the web app is packaged in the the final jar, the entire resources folder is copied into the classpath, so that the templates directory is accessible at runtime.

Set Up Okta for OAuth 2.0 Single Sign-On

Now you’re going to set up authorization for our app. Okta makes this super easy. You should have already signed up for a free developer.okta.com account. Now you’re going to create an OpenID Connect (OIDC) application to use with OAuth 2.0 Single Sign-On (SSO).

That might be a lot of jargon and acronyms, if you’re not already familiar with them. Very simply, OAuth 2.0 is an industry standard for authorization - a standardized and tested method by which authorization servers and applications can communicate to facilitate user authorization. OpenID Connect is a layer on top of OAuth 2.0 that standardizes and simplifies the authorization procedure as well as providing user authentication. Together they provide a proven way for an application to interact with a remote server that provides authentication and authorization services (such as Okta).

To create an OIDC app, open your Okta developer dashboard. Click on the Applications top menu item, and then click on Add Application.

You should see the following screen. Click on the icon for the Web option. Click Next.

You need to update a few of the initial configuration options. First change the name to something more descriptive. I used “Okta Spring Boot Simple Web App.” Next update the Login redirect URIs to http://localhost:8080/login. Click Done.

This will take you to the new application’s general configuration tab. Scroll down and note the Client ID and Client secret. You’ll need these later.

That’s all you need to do to set up Okta for OAuth! Now let’s return to the Spring Boot app and hook our new OIDC application into the Spring Boot application.

Configure Your Spring Boot App for Single Sign-On (SSO)

Now you need to configure the Spring Boot app to interact with the Okta servers. This is super easy. We need to do two things:

  1. Add the @EnableOAuth2Sso annotation
  2. Update the application.yml configuration

First add the @EnableOAuth2Sso annotation to the SpringSecurityWebAppConfig class.

@EnableOAuth2Sso
@Configuration
public class WebSecurityConfigurerAdapter extends WebSecurityConfigurerAdapter {

@Override  
protected void configure(HttpSecurity http) throws Exception {  
    http.authorizeRequests().anyRequest().permitAll();          
}  

}

The @EnableOAuth2Sso annotation does a TON of stuff. It’s worth digging into to understand what’s going on. You can check out Spring’s docs on the annotation itself, and their Spring Boot and OAuth2 tutorial.

One thing I want to point out (bc this has been bugging me a while and I just figured it out) is that you can put this annotation on other classes in the project. However, if you do, be aware that Spring is going to create a WebSecurityConfigurerAdapter and add it to the security chain. Since we’re also creating a WebSecurityConfigurerAdapter, there will be two of them, and you’ll get an error about conflicting chain orders. This is because both WebSecurityConfigurerAdapters will by default use the same chain order. You can resolve this error by adding an @Order(101) annotation to our customized class. However, even better is to add the @EnableOAuth2Sso annotation to our WebSecurityConfigurerAdapter class, WebSecurityConfigurerAdapter, and Spring will use that class instead of creating a duplicate one.

The second change you need to make is update the src/main/resources/application.yml file, filling in some Okta-specific configuration options for the OAuth SSO values take from our Okta OIDC application.

You’ll need to fill in your Client ID and Client secret from the application you created above. You’ll also need to change the issuer URL so that it reflects your Okta preview URL, something like dev-123456.oktapreview.com.

server:
port: 8080

spring:
resources: static-locations: "classpath:/static/"

okta:
oauth2:
issuer: https://{yourOktaDomain}/oauth2/default
clientId: {yourClientId}
clientSecret: {yourClientSecret}
rolesClaim: groups

Refine Our Permissions

Now you’re going to want to update the SpringSecurityWebAppConfig class so that you have a public home page and a restricted “restricted” page. We do this by using Spring’s fluent API for the HttpSecurity object.

import org.springframework.boot.autoconfigure.security.oauth2.client.EnableOAuth2Sso;
import org.springframework.context.annotation.Configuration;
import org.springframework.security.config.annotation.web.builders.HttpSecurity;
import org.springframework.security.config.annotation.web.configuration.WebSecurityConfigurerAdapter;

@EnableOAuth2Sso
@Configuration
public class SpringSecurityWebAppConfig extends WebSecurityConfigurerAdapter {

@Override  
protected void configure(HttpSecurity http) throws Exception {  
    http.authorizeRequests()  
            .antMatchers("/").permitAll() // allow all at home page
            .antMatchers("/img/**").permitAll()  // allow all to access static images
            .anyRequest().authenticated();  // authenticate everything else!
}  

}

Restart your app and now you should be able to:

  1. See the home page without authenticating
  2. NOT see the /restricted page without authenticating
  3. Be able to authenticate using Okta Single Sign-On

This point in the tutorial corresponds to the OktaOAuthSSO tag in the GitHub repository.

Take a Look at the Thymeleaf Templates

The Thymeleaf templates are pretty self explanatory, on the whole, but I did want to point out a couple things. Thymeleaf templates are fully valid HTML5, which is nice. If you want to dig deeper, you can head over to their website and their documentation.

What I wanted to point out is how the template brings in authentication information. To do this, we’re using the thymeleaf-extras-springsecurity plugin. This is included in the build.gradle file with the following line:

compile ("org.thymeleaf.extras:thymeleaf-extras-springsecurity4")

And is included in the template file as an XML namespace attribute on the main <html> tag.

xmlns:sec="http://www.thymeleaf.org/thymeleaf-extras-springsecurity4"

This plugin is what allows us to check if a user is authenticated using the th:if attribute with a custom SPEL expression (Spring Expression Language). It also allows us to insert authentication properties. Below you see a span <span th:text="${#authentication.name}"></span> that is used to insert the name of the authenticated user.

<html xmlns:th="http://www.thymeleaf.org" xmlns:sec="http://www.thymeleaf.org/thymeleaf-extras-springsecurity4">
<head>
<!--// <th:block th:include="fragments/head :: head"/> //-->
</head>
<body>
<div class="container-fluid">
<div class="row">
<div class="box col-md-6 col-md-offset-3">
<div class="okta-header">
<img src="img/logo.png"/>
</div>

        &lt;!--/* displayed if account IS NOT null, indicating that the user IS logged in */--&gt;  
        &lt;div th:if="${#authorization.expression('isAuthenticated()')}"&gt;  
            &lt;h1 th:inline="text"&gt;Hello, &lt;span th:text="${#authentication.name}"&gt;&lt;/span&gt;!&lt;/h1&gt;  
            &lt;a href="/restricted" class="btn btn-primary"&gt;Restricted&lt;/a&gt;  
        &lt;/div&gt;  

        &lt;!--/* displayed if account IS null, indicating that the user IS NOT logged in */--&gt;  
        &lt;div th:unless="${#authorization.expression('isAuthenticated()')}"&gt;  
            &lt;h1&gt;Who are you?&lt;/h1&gt;  
            &lt;a href="/restricted" class="btn btn-primary"&gt;Restricted&lt;/a&gt;  
        &lt;/div&gt;  
    &lt;/div&gt;  
&lt;/div&gt;  

</div>
</body>
</html>

The thymeleaf-extras-springsecurity plugin has some other nice features as well. If you want to dig a little deeper, check out the project repository on GitHub.

Secure Access By Group Membership

The next step in our tutorial is to add Group-based authentication using user groups that we’ll create and define on Okta. A very common example of this is to have an “admin” section of a website and a “user” section of a website, along with perhaps a public home page open to everybody. In this example, “admin” and “user” would correspond to two different groups of which an authenticated user could be a member. What we want to do is be able to restrict access to URL endpoints based on user group membership, and to be able to assign users to these groups.

A side note: groups vs roles. What’s the difference?

  • A “group” is a collection of users, and permissions are assigned to the group. Generally speaking group membership is relatively static, at least throughout the duration of a session.
  • A “role” is a set of permissions that a user can inherit when he/she acts under that role. Roles are generally more dynamic in nature. Users can have many roles. Roles frequently are activated or deactivated depending on complex criteria and often may change throughout a user session.

In practice, for simple authorization systems, they’re pretty similar. The main difference is that groups classify based on individual identity, whereas roles classify based on permissible activities. You’ll probably see apps and tutorials on the wild and woolly internet that ignore this difference, as it’s functionally somewhat subtle. (But now you know. And you can get on the comment thread for the tutorial in question and write a comment correcting the author.)

Configure Authorization Groups in Okta

Go to your developer.okta.com dashboard. From the top menu, go to Users and click on Groups.

Click on the Add Group button.

Name the group “Admin” and give it a description (I put “Administrators,” doesn’t matter what you put here really, just something descriptive).

Click on the group Name to open the group and click on the Add Members button. Add your user to the Admin group.

Next add a new user that’s not an admin.

  • Go to Users from the top menu and click on People.
  • Click Add Person.
  • Fill out the popup form:
  • First name: Not
  • Last name: Admin
  • Username: [email protected]
  • No groups or secondary email
  • Password: Set by admin
  • Assign a password
  • Uncheck “User must change password on first login”
  • Click Save

The next thing you’ll need to do is add a “groups” claim to the default authorization server.

  • From the top menu, go to API and click on Authorization Servers”
  • Click on the default authorization server.
  • Click on the Claims tab.
  • Click the Add Claim button.
  • Update the popup form to match the image below
  • Name: groups
  • Token type: Access
  • Value type: Groups
  • Filter: Regex .*
  • Don’t disable
  • Include in any scope

What you’re doing here is telling Okta to include a “groups” claim in the access token that is sent to your application. This is the OAuth method of Okta telling your application about the groups your authenticated user is a member of. Somewhat confusingly, these will be called “authorities” on the Spring application side, which is an abstract term for groups/roles/privileges communicated by the OAuth server to the app.

Now we have two users. Your primary user, which has been added to the Admin group, and a new user that is not in the admin group. We’ve also configured Okta to add the groups claim to the access token. Now all we have to do is make a few changes to the app code!

Update Your Spring Boot + Spring Security App to Use Group-based Authorization

This is where the Okta Spring Boot Starter really starts to shine. Normally if you wanted to map the security groups and groups claims that we are sending in the token to groups in the app, you’d have to write an extractor class or two to handle the extraction, as well as perhaps a group class. The Okta Spring Boot Starter handles all of this for you!

The first thing you’re going to want to do is add the following annotation to your SpringSecurityWebAppConfig class.

@EnableGlobalMethodSecurity(prePostEnabled = true)

Like so:

import org.springframework.security.config.annotation.method.configuration.EnableGlobalMethodSecurity;

@EnableOAuth2Sso
@Configuration
@EnableGlobalMethodSecurity(prePostEnabled = true)
public class SpringSecurityWebAppConfig extends WebSecurityConfigurerAdapter {
/* class contents omitted for brevity */
}

This annotation enables the next annotation that we’re going to use, the @PreAuthorize annotation. This annotation allows us to use a Spring Expression Language (SpEL) predicate to determine if the controller method is authorized. The predicate expression is executed before the app even enters the controller method (hence the “pre”-authorize).

In the SimpleAppController class, add a new method called admin like so:

import org.springframework.security.access.prepost.PreAuthorize;

@Controller
class SimpleAppController {

/* other controllers omitted for clarity */ 

@RequestMapping("/admin")  
@PreAuthorize("hasAuthority('Admin')")  
String admin() {  
    return "admin";  
}  

}

Just to recap a little, this method does the following:

  • create a mapping for the /admin url endpoint;
  • assign the /admin endpoint an authorization scheme based on SpEL;
  • and simply return the name of a Thymeleaf template, assumed to be in the /templates directory (which we’ll create next).

Create the new admin template page. In the src/main/resources/templates directory, create a new file called admin.html with the following contents:

<html xmlns:th="http://www.thymeleaf.org" xmlns:sec="http://www.thymeleaf.org/thymeleaf-extras-springsecurity4">
<head>
<!--// <th:block th:include="fragments/head :: head"/> //-->
</head>
<body>
<div class="container-fluid">
<div class="row">
<div class="box col-md-6 col-md-offset-3">
<div class="okta-header">
<img src="img/logo.png"/>
</div>

        &lt;h1&gt;Welcome to the admin page!&lt;/h1&gt;  

        &lt;a href="/" class="btn btn-primary"&gt;Go Home&lt;/a&gt;  

    &lt;/div&gt;  
&lt;/div&gt;  

</div>
</body>
</html>

You may be asking yourself what the SpEL expression used in the @PreAuthorize annotation means. Why is the SpEL expression hasAuthority and not hasGroup? A correct answer is somewhat complicated, having to do with the fact that Spring calls permissions privileges and authorities in different contexts, which can be mapped to groups and roles in the app. When using Spring Boot and OAuth, an ‘authority’ is often equated with a ‘role’, which is fine. But you said we’re using groups, not roles? Right. Practically speaking, in this instance, it doesn’t matter because Okta knows we’re talking about groups and the app knows we’re talking about groups, and in the middle we just use the groups claim and the authorities fields to communicate the text strings that represent the groups the user is a member of.

A helpful hint:

If you want to inspect the authentication information that the Spring Boot App is receiving, you can add the following line in one of the controller methods before the return statement.

Authentication authentication = SecurityContextHolder.getContext().getAuthentication();

Set a breakpoint on this line, or right after it, really, and run the app with a debugger that allows you to inspect the authentication object. It’s a great way to learn and debug problems.

Try Out Your New Spring Boot + Spring Security Web App!

That’s pretty much it. You should be able to restart the app and log in with two different users. Only the user that was added to the Admin group should be able to access the admin page. You’ll have to directly navigate to http://localhost:8080/admin (as we didn’t add a link or a button). If you try to navigate to the admin page with the other user, you’ll see the beautiful whitelabel error page showing a 403 / Unauthorized error.

Keep in mind that when switching between users you’ll have to stop the app, log out of your developer.okta.com account, and restart the app. You can also use an incognito window in your browser.

This part of the tutorial corresponds to the GroupsAuth tag, which you can checkout using the following command git checkout tags/GroupsAuth.

Thanks for reading. If you liked this post, share it with all of your programming buddies!

Further reading

☞ Spring & Hibernate for Beginners (includes Spring Boot)

☞ Spring Framework Master Class - Learn Spring the Modern Way!

☞ Master Microservices with Spring Boot and Spring Cloud

☞ Spring Boot and OAuth2: Getting the Authorization Code

☞ An Introduction to Spring Boot

☞ How to build GraphQL APIs with Kotlin, Spring Boot, and MongoDB?

☞ Build a Rest API with Spring Boot using MySQL and JPA

☞ Angular 8 + Spring Boot 2.2: Build a CRUD App Today!

☞ Spring Boot vs. Spring MVC vs. Spring: How Do They Compare?

☞ Top 4 Spring Annotations for Java Developer in 2019


Originally published on developer.okta.com

Web Service Tutorial: Streaming Data with Spring Boot RESTful

Web Service Tutorial: Streaming Data with Spring Boot RESTful

In this article, we are going to look at an example to download files using StreamingResponseBody. In this approach, data is processed and written in chunks to the OutputStream.

In this article, we are going to look at an example to download files using StreamingResponseBody. In this approach, data is processed and written in chunks to the OutputStream.

Streaming data is a radical new approach to sending data to web browsers which provides for dramatically faster page load times. Quite often, we need to allow users to download files in web applications. When the data is too large, it becomes quite a challenge to provide a good user experience.

Spring offers support for asynchronous request processing via StreamingResponseBody. In this approach, an application can write data directly to the response OutputStream without holding up the Servlet container thread. There are a few other methods in Spring to handle asynchronous request processing.

Setting Up Spring Boot Project

Create a sample Spring Boot application. Here is my sample project structure. I have created the project manually, but you could also create using Spring Intializer.

Project structure

Let us add some basic dependencies to Maven POM.

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>com.techshard.streamingresponse</groupId>
    <artifactId>springboot-download</artifactId>
    <version>1.0-SNAPSHOT</version>
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.1.4.RELEASE</version>
        <relativePath />
    </parent>
    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-security</artifactId>
        </dependency>
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-api</artifactId>
        </dependency>
        <dependency>
            <groupId>org.apache.commons</groupId>
            <artifactId>commons-io</artifactId>
            <version>1.3.2</version>
        </dependency>
    </dependencies>
</project>

We will now create a controller and add an API endpoint for download. Here is my complete controller.

package com.techshard.download.controller;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.http.HttpStatus;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody;
import javax.servlet.http.HttpServletResponse;
import java.io.*;
import java.util.zip.ZipEntry;
import java.util.zip.ZipOutputStream;
@RestController
@RequestMapping ("/api")
public class DownloadController {
    private final Logger logger = LoggerFactory.getLogger(DownloadController.class);
    @GetMapping (value = "/download", produces = MediaType.APPLICATION_JSON_VALUE)
    public ResponseEntity<StreamingResponseBody> download(final HttpServletResponse response) {
        response.setContentType("application/zip");
        response.setHeader(
                "Content-Disposition",
                "attachment;filename=sample.zip");
        StreamingResponseBody stream = out -> {
            final String home = System.getProperty("user.home");
            final File directory = new File(home + File.separator + "Documents" + File.separator + "sample");
            final ZipOutputStream zipOut = new ZipOutputStream(response.getOutputStream());
            if(directory.exists() && directory.isDirectory()) {
                try {
                    for (final File file : directory.listFiles()) {
                        final InputStream inputStream=new FileInputStream(file);
                        final ZipEntry zipEntry=new ZipEntry(file.getName());
                        zipOut.putNextEntry(zipEntry);
                        byte[] bytes=new byte[1024];
                        int length;
                        while ((length=inputStream.read(bytes)) >= 0) {
                            zipOut.write(bytes, 0, length);
                        }
                        inputStream.close();
                    }
                    zipOut.close();
                } catch (final IOException e) {
                    logger.error("Exception while reading and streaming data {} ", e);
                }
            }
        };
        logger.info("steaming response {} ", stream);
        return new ResponseEntity(stream, HttpStatus.OK);
    }
}

In this API endpoint, we are reading multiple files from a directory and creating a zip file. We are executing this process within StreamingResponseBody*. It writes data directly to an OutputStream before passing that written information back to the client using aResponseEntity. *This means that the download process will start immediately on the client, while the server is processing and writing data in chunks.

Start the server and test this endpoint using http://localhost:8080/api/download.

When using StreamingResponseBody, it is highly recommended to configure TaskExecutor used in Spring MVC for executing asynchronous requests. TaskExecutor is an interface that abstracts the execution of a Runnable.

Let us configure the TaskExecutor. Here is the AsyncConfiguration class which configures timeout using WebMvcCofigurer and also registers an interceptor that is called when there's a timeout in case you need some special handling.

package com.techshard.download;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.aop.interceptor.AsyncUncaughtExceptionHandler;
import org.springframework.aop.interceptor.SimpleAsyncUncaughtExceptionHandler;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.task.AsyncTaskExecutor;
import org.springframework.scheduling.annotation.AsyncConfigurer;
import org.springframework.scheduling.annotation.EnableAsync;
import org.springframework.scheduling.annotation.EnableScheduling;
import org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor;
import org.springframework.web.context.request.NativeWebRequest;
import org.springframework.web.context.request.async.CallableProcessingInterceptor;
import org.springframework.web.context.request.async.TimeoutCallableProcessingInterceptor;
import org.springframework.web.servlet.config.annotation.AsyncSupportConfigurer;
import org.springframework.web.servlet.config.annotation.WebMvcConfigurer;
import java.util.concurrent.Callable;
@Configuration
@EnableAsync
@EnableScheduling
public class AsyncConfiguration implements AsyncConfigurer {
    private final Logger log = LoggerFactory.getLogger(AsyncConfiguration.class);
    @Override
    @Bean (name = "taskExecutor")
    public AsyncTaskExecutor getAsyncExecutor() {
        log.debug("Creating Async Task Executor");
        ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
        executor.setCorePoolSize(5);
        executor.setMaxPoolSize(10);
        executor.setQueueCapacity(25);
        return executor;
    }
    @Override
    public AsyncUncaughtExceptionHandler getAsyncUncaughtExceptionHandler() {
        return new SimpleAsyncUncaughtExceptionHandler();
    }
    /** Configure async support for Spring MVC. */
    @Bean
    public WebMvcConfigurer webMvcConfigurerConfigurer(AsyncTaskExecutor taskExecutor, CallableProcessingInterceptor callableProcessingInterceptor) {
        return new WebMvcConfigurer() {
            @Override
            public void configureAsyncSupport(AsyncSupportConfigurer configurer) {
                configurer.setDefaultTimeout(360000).setTaskExecutor(taskExecutor);
                configurer.registerCallableInterceptors(callableProcessingInterceptor);
                WebMvcConfigurer.super.configureAsyncSupport(configurer);
            }
        };
    }
    @Bean
    public CallableProcessingInterceptor callableProcessingInterceptor() {
        return new TimeoutCallableProcessingInterceptor() {
            @Override
            public <T> Object handleTimeout(NativeWebRequest request, Callable<T> task) throws Exception {
                log.error("timeout!");
                return super.handleTimeout(request, task);
            }
        };
    }
}

Conclusion

Using StreamingResponseBody, we can now stream data easily for highly-concurrent applications. I hope you enjoyed this article. Let me know if you have any comments or suggestion in the comments section below.

The example for this article can be found on GitHub repository.