How To Use Vuls as a Vulnerability Scanner on Ubuntu 18.04

How To Use Vuls as a Vulnerability Scanner on Ubuntu 18.04

In this tutorial, you’ll deploy Vuls to an Ubuntu 18.04 server. This includes building Vuls and its dependencies from source code, configuring scanning and reporting to Slack, and optionally connecting it to target machines to enable remote scanning. In the end, you’ll have an automated vulnerability reporting system in place that alerts you to vulnerabilities eliminating the need for manual checks.

In this tutorial, you’ll deploy Vuls to an Ubuntu 18.04 server. This includes building Vuls and its dependencies from source code, configuring scanning and reporting to Slack, and optionally connecting it to target machines to enable remote scanning. In the end, you’ll have an automated vulnerability reporting system in place that alerts you to vulnerabilities eliminating the need for manual checks.

Introduction

Vuls is an open-source, agentless vulnerability scanner written in Go. It automates security vulnerability analysis of the software installed on a system, which can be a burdensome task for system administrators to do manually in a production environment. Vuls uses multiple renowned vulnerability databases, such as the National Vulnerability Database (NVD). Light on resources, Vuls has the ability to scan multiple systems at once, and to send reports via email or Slack. It has three scan modes (fast, fast root, and deep), which you can select according to the situation.

Vuls is not a broad IT security scanner; for example, it does not monitor network traffic or protect against brute-force login attacks. However, Vuls provides a way of automating vulnerability reporting for Linux packages. When the databases Vuls uses are informed of a fix to certain vulnerabilities, Vuls will also pull this remediation information into its reports. When generating reports, Vuls prioritizes the most urgent vulnerabilities using the established ranking system from the database.

Prerequisites

Before you begin this tutorial, you’ll need:

  • A server with at least 2 GB RAM running Ubuntu 18.04 with root access, and a secondary, non-root account. You can set this up by following this initial server setup guide. For this tutorial the non-root user is sammy.
  • (Optional) Multiple servers running (preferably) Ubuntu 18.04 with root access and a secondary, non-root account, if you want to set up Vuls to scan them remotely. In this tutorial, the secondary account is sammy-shark.
Step 1 — Installing Dependencies

In this section, you’ll create a folder for storing Vuls data, install the latest version of the Go programming language, and install other packages Vuls and its dependencies require.

Start off by logging in as sammy:

ssh [email protected]_server_ip


For this tutorial, you’ll store all Vuls-related data in the /usr/share/vuls-data directory. Create it by running the following command:

sudo mkdir /usr/share/vuls-data


To make it accessible to sammy, run the following command:

sudo chown -R sammy /usr/share/vuls-data


You’ve now created the vuls-data folder, which will be your workspace. Before you continue installing the required packages, first update the package manager cache:

sudo apt update


To download and compile the dependencies, you’ll install git, gcc, make, sqlite, debian-goodies, golang-go, and wget.

sqlite is a database system, which you’ll use here for storing vulnerability information. [debian-goodies](https://packages.debian.org/stretch/debian-goodies "debian-goodies") contains the checkrestart utility, which provides information on what packages can and should be restarted at any given moment in time. golang-go is the Go programming language.

You can install them all in one command:

sudo apt install sqlite git debian-goodies gcc make wget golang-go -y


You have now installed the required packages, including Go.

In order to work, Go requires a few environment variables that you’ll set up: GOPATH and PATH. GOPATH specifies the working directory for Go and PATH (which contains directories in which programs are placed) that must be extended to tell the system where to find Go itself.

These environment variables need to be set each time the user logs on. To automate this, you will create a new executable file, called go-env.sh, under /etc/profile.d. This will result in the directory executing every time a user logs on.

Create go-env.sh using your text editor:

sudo nano /etc/profile.d/go-env.sh


Add the following commands to the file:

/etc/profile.d/go-env.sh

export GOPATH=$HOME/go
export PATH=$PATH:$GOPATH/bin


The export command sets the given environment variable to the desired value; here you use it to populate GOPATH and PATH with appropriate values.

Save and close the file.

Currently, go-env.sh is not executable. To fix this, mark it as executable by running the following command:

sudo chmod +x /etc/profile.d/go-env.sh


To avoid having to log in again, you can reload go-env.sh by running:

source /etc/profile.d/go-env.sh


The source command reloads the given file into the current shell while preserving its state.

In this section, you have installed the Go language, set up its environment variables, and installed packages that you’ll require later on. In the next steps, you’ll download and compile the Go programs that Vuls requires. Those programs are go-cve-dictionary and goval-dictionary, which Vuls uses for querying vulnerability databases.

Step 2 — Installing and Running go-cve-dictionary

In this section, you will download and compile go-cve-dictionary, a Go package that provides access to the NVD (National Vulnerability Database). Then, you will run it and fetch vulnerability data for Vuls to use. The NVD is the US government’s repository of publicly reported cybersecurity vulnerabilities, containing vulnerability IDs (CVE — Common Vulnerabilities and Exposures), summaries, and impact analysis, and is available in a machine-readable format.

Go stores packages under $GOPATH/src/. You can extend this further with the use of subdirectories to note origin. As an example, packages from GitHub, made by the user, example-user would be stored under $GOPATH/src/github.com/example-user.

You’ll first install go-cve-dictionary, made by kotakanbe, by cloning the Go package from GitHub and compiling it afterwards.

Start off by creating a directory to store it, according to the example path:

mkdir -p $GOPATH/src/github.com/kotakanbe


Navigate to it by running:

cd $GOPATH/src/github.com/kotakanbe


Now you’ll clone go-cve-dictionary from GitHub to your server by running:

git clone https://github.com/kotakanbe/go-cve-dictionary.git


Then, navigate to the package root:

cd go-cve-dictionary


Finally, compile and install it by running the following command:

make install


Keep in mind that this command may take some time to finish. To make it available system wide, copy it to the /usr/local/bin:

sudo cp $GOPATH/bin/go-cve-dictionary /usr/local/bin


go-cve-dictionary requires access to a log output directory, and by default it is /var/log/vuls. Create it by running:

sudo mkdir /var/log/vuls


Right now, the log directory is readable by everyone. Restrict access to the current user with the following command:

sudo chmod 700 /var/log/vuls


Setting the permission flags to 700 restricts access to only the owner.

To make it accessible to sammy, or another user, run the following command:

sudo chown -R sammy /var/log/vuls


Now, you’ll fetch vulnerability data from the NVD and store it in your Vuls workspace (/usr/share/vuls-data):

for i in `seq 2002 $(date +"%Y")`; do sudo go-cve-dictionary fetchnvd -dbpath /usr/share/vuls-data/cve.sqlite3 -years $i; done


This command loops from the year 2002 to the current year (seq 2002 $(date +"%Y")) and calls go-cve-dictionary fetchnvd to fetch the NVD data for the current (loop) year by passing -years $i. It then stores this information in a database under /usr/share/vuls-data.

Note: This command will take a long time to finish, and will fail if your server has less than 2 GB of RAM.

In this step, you have downloaded and installed go-cve-dictionary, and fetched NVD data for Vuls to later use. In the next section, you’ll download and install goval-dictionary and fetch OVAL data for Ubuntu.

Step 3 — Installing and Running goval-dictionary

In this section, you will download and compile goval-dictionary, a Go package that provides access to the OVAL database for Ubuntu. You’ll then run it and fetch vulnerability data for Vuls to use. OVAL stands for Open Vulnerability and Assessment Language, which is an open language used to express checks for determining whether software vulnerabilities exist on a given system.

The same author, kotakanbe, writes the goval-dictionary, and you’ll store it next to the previous package.

Navigate to the $GOPATH/src/github.com/kotakanbe folder:

cd $GOPATH/src/github.com/kotakanbe


Clone the package from GitHub by running the following command:

git clone https://github.com/kotakanbe/goval-dictionary.git


Enter the package folder:

cd goval-dictionary


Compile and install it with make:

make install


Copy it to /usr/local/bin to make it globally accessible:

sudo cp $GOPATH/bin/goval-dictionary /usr/local/bin


Then, fetch the OVAL data for Ubuntu 18.x by running the following command:

sudo goval-dictionary fetch-ubuntu -dbpath=/usr/share/vuls-data/oval.sqlite3 18


In this step, you have downloaded and installed goval-dictionary, and fetched the OVAL data for Ubuntu 18.x. In the next step, you’ll download and install Vuls.

Step 4 — Downloading and Configuring Vuls

With all of the dependencies installed, now you’ll download and compile Vuls from source code. Afterward, you’ll configure it to scan the local machine.

Create a new directory that contains the path to the Vuls repository, with the following command:

mkdir -p $GOPATH/src/github.com/future-architect


Navigate to it:

cd $GOPATH/src/github.com/future-architect


Clone Vuls from GitHub by running the following command:

git clone https://github.com/future-architect/vuls.git


Enter the package folder:

cd vuls


Compile and install it at the same time by running:

make install


Remember that it may take some time for this command to complete.

Copy it to /usr/local/bin to make it globally accessible:

sudo cp $GOPATH/bin/vuls /usr/local/bin


Now, you’ll create a configuration file for Vuls. Navigate back to /usr/share/vuls-data:

cd /usr/share/vuls-data


Vuls stores its configuration in a TOML file, which you’ll call config.toml. Create it using your text editor:

sudo nano config.toml


Enter the following configuration:

/usr/share/vuls-data/config.toml

[cveDict]
type = "sqlite3"
SQLite3Path = "/usr/share/vuls-data/cve.sqlite3"

[ovalDict]
type = "sqlite3"
SQLite3Path = "/usr/share/vuls-data/oval.sqlite3"

[servers]

[servers.localhost]
host = "localhost"
port = "local"
scanMode = [ "fast" ]
#scanMode = ["fast", "fast-root", "deep", "offline"]


The first two sections of this configuration (cveDict and ovalDict) point Vuls to the vulnerability databases you created in the last two steps. The next section (servers) marks the start of server-related information. Separate sections will group information about each server. The only server Vuls will scan with this outlined configuration is the local server (localhost).

Vuls provides four scan modes:

  • A server with at least 2 GB RAM running Ubuntu 18.04 with root access, and a secondary, non-root account. You can set this up by following this initial server setup guide. For this tutorial the non-root user is sammy.
  • (Optional) Multiple servers running (preferably) Ubuntu 18.04 with root access and a secondary, non-root account, if you want to set up Vuls to scan them remotely. In this tutorial, the secondary account is sammy-shark.

Save and close the file.

To test the validity of the configuration file, run the following command:

vuls configtest


You’ll see the following output:

[Feb 27 19:36:42]  INFO [localhost] Validating config...
[Feb 27 19:36:42]  INFO [localhost] Detecting Server/Container OS...
[Feb 27 19:36:42]  INFO [localhost] Detecting OS of servers...
[Feb 27 19:36:42]  INFO [localhost] (1/1) Detected: localhost: ubuntu 18.04
[Feb 27 19:36:42]  INFO [localhost] Detecting OS of containers...
[Feb 27 19:36:42]  INFO [localhost] Checking Scan Modes...
[Feb 27 19:36:42]  INFO [localhost] Checking dependencies...
[Feb 27 19:36:42]  INFO [localhost] Dependencies... Pass
[Feb 27 19:36:42]  INFO [localhost] Checking sudo settings...
[Feb 27 19:36:42]  INFO [localhost] sudo ... No need
[Feb 27 19:36:42]  INFO [localhost] It can be scanned with fast scan mode even if warn or err messages are displayed due to lack of dependent packages or sudo settings in fast-root or deep scan mode
[Feb 27 19:36:42]  INFO [localhost] Scannable servers are below...
localhost


You’ve entered the configuration correctly, and Vuls has detected that it can scan the local server.

You’ve installed and configured Vuls to scan the local server. In the next step, you will run a local scan and view the generated report.

Step 5 — Running a Local Scan

In this section, you will run a local scan and then view the generated vulnerability report. By now, you have configured only the local server, which Vuls correctly detected in the last step. The default scan mode, if not explicitly specified, is fast.

To run a scan, execute the following command:

vuls scan


You’ll see output similar to this:

[Feb 27 19:44:12]  INFO [localhost] Start scanning
[Feb 27 19:44:12]  INFO [localhost] config: /usr/share/vuls-data/config.toml
[Feb 27 19:44:12]  INFO [localhost] Validating config...
[Feb 27 19:44:12]  INFO [localhost] Detecting Server/Container OS...
[Feb 27 19:44:12]  INFO [localhost] Detecting OS of servers...
[Feb 27 19:44:12]  INFO [localhost] (1/1) Detected: localhost: ubuntu 18.04
[Feb 27 19:44:12]  INFO [localhost] Detecting OS of containers...
[Feb 27 19:44:12]  INFO [localhost] Checking Scan Modes...
[Feb 27 19:44:12]  INFO [localhost] Detecting Platforms...
[Feb 27 19:44:12]  INFO [localhost] (1/1) localhost is running on other
[Feb 27 19:44:12]  INFO [localhost] Scanning vulnerabilities...
[Feb 27 19:44:12]  INFO [localhost] Scanning vulnerable OS packages...
[Feb 27 19:44:12]  INFO [localhost] Scanning in fast mode

One Line Summary
================
localhost       ubuntu18.04     539 installed

To view the detail, vuls tui is useful.
To send a report, run vuls report -h.


Vuls has logged what it did in the process. To view a report of vulnerabilities it has identified, run:

vuls tui


Vuls divides the report view into four panels:

  • A server with at least 2 GB RAM running Ubuntu 18.04 with root access, and a secondary, non-root account. You can set this up by following this initial server setup guide. For this tutorial the non-root user is sammy.
  • (Optional) Multiple servers running (preferably) Ubuntu 18.04 with root access and a secondary, non-root account, if you want to set up Vuls to scan them remotely. In this tutorial, the secondary account is sammy-shark.

You can cycle the cursor through the panels by pressing ENTER, and navigate with the keyboard arrows.

In this step, you have run a local scan and inspected the results. In the next optional section, you’ll configure Vuls to scan multiple target machines.

Step 6 — (Optional) Configuring Multiple Target Machines

In this section, you’ll configure Vuls to scan multiple target machines. This entails configuring /etc/sudoers on the target and configuring Vuls to scan the target.

In the previous step, you configured Vuls to scan the local machine (localhost). You can add as many servers as you wish, provided you have the following:

  • A server with at least 2 GB RAM running Ubuntu 18.04 with root access, and a secondary, non-root account. You can set this up by following this initial server setup guide. For this tutorial the non-root user is sammy.
  • (Optional) Multiple servers running (preferably) Ubuntu 18.04 with root access and a secondary, non-root account, if you want to set up Vuls to scan them remotely. In this tutorial, the secondary account is sammy-shark.

You can only use a non-root user account on the target server for scanning in fast mode. To enable scanning in fast root and deep modes, you’ll need to edit the /etc/sudoers file on the target machine(s). The sudoers file controls which users can run what commands, and also whether you need a password for specified commands.

Since visudo is the utility for defining rules for access and privileged access, you can only run it as root. Because of the importance of sudoers, the file will not exit with errors without giving a warning.

On the target server, log in as root and open sudoers for editing by running visudo:

visudo


Add this line to the end of the file:

/etc/sudoers

sammy-shark ALL=(ALL) NOPASSWD: /usr/bin/apt-get update, /usr/bin/stat *, /usr/sbin/checkrestart


This line instructs sudo to allow user sammy-shark to run apt-get update, checkrestart, and every command available from stat, without providing a password.

Save and close the file. If you made a syntax error in the process, visudo will inform you and offer to edit it again or exit.

Note: By allowing the sammy-shark user in sudoers, you are allowing Vuls to scan using fast root and deep modes. If you want to allow those modes for the local machine (localhost) too, edit sudoers on localhost as shown earlier.

Vuls uses the checkrestart utility to check for packages that are updated, but require restart. To ensure the target server has it, log in as your non-root user, and install it by running the following command:

sudo apt install debian-goodies -y


That is all you need to do on the target server; you can now log out from the target and log back in to your first server.

To add a new server for scanning, open config.toml and add the following lines under the [servers] mark:

/usr/share/vuls-data/config.toml

[servers.target_name]
host = "target_ip"
port = "22"
user = "account_username"
keyPath = "account_rsa_key"
scanMode = [ "deep" ] # "fast", "fast-root" or "deep"


The lines above serve as a template for adding new servers. Remember to replace target_name with the desired name, target_ip with the IP of the target server, account_username with the username, and account_rsa_key with the path to the RSA key. Vuls does not support SSH password authentication, so specifying a keyPath is necessary.

Save and close the file.

Next, for each target server you’ve added, you’ll confirm the RSA keys on the local machine. To achieve this, you’ll log in to the target server from your first server with the appropriate key, like so:

ssh [email protected]_ip -i account_rsa_key


When asked whether you want to continue connecting, enter yes, then log out by pressing CTRL + D.

If you get an error about key file permissions being too open, set them to 600 by running the following command:

chmod 600 account_rsa_key


Setting permissions to 600 ensures that only the owner can read and write the key file.

To check the validity of the new configuration, run the following command:

vuls configtest


There will be no output from this command. If there are any errors, check your config.toml against the configuration in the tutorial.

In this step, you’ve added more target servers to your Vuls configuration, thus marking them for scanning. In the next section, you will configure Vuls to periodically scan and send reports to a configured Slack workspace.

Step 7 — Configuring Periodic Scanning and Reporting to Slack

In this section, you will configure Vuls to send reports to Slack and make a cron job to run Vuls scans periodically.

To use Slack integration, you’ll need to have an incoming webhook on Slack for your workspace. Incoming webhooks are a simple way of an application providing other applications real-time information. In this case, you’ll be configuring the Vuls to report to your Slack channel.

If you haven’t ever created a webhook, you’ll first need to create an app for your workspace. To do so, first log in to Slack and navigate to the app creation page. Pick a name that you’ll recognize, select the desired workspace, and click Create App.

You’ll be redirected to the settings page for the new app. Click on Incoming Webhooks on the left navigation bar.

Enable webhooks by flipping the switch button next to the title Activate Incoming Webhooks.

A new section further down the page will be uncovered. Scroll down and click the Add New Webhook to Workspace button. On the next page, select the channel you want the reports to be sent to and click Authorize.

You’ll be redirected back to the settings page for webhooks, and you’ll see a new webhook listed in the table. Click on Copy to copy it to clipboard and make note of it for later use.

Then, open config.toml for editing:

sudo nano config.toml


Add the following lines:

/usr/share/vuls-data/config.toml

[slack]
hookURL      = "your_hook_url"
channel      = "#your_channel_name"
authUser     = "your_username"
#notifyUsers  = ["@username"]


Replace the your_hook_URL with the webhook URL you noted earlier, your_username with the username of the user that created the web hook, and your_channel_name with the name of the desired channel. Save and close the file.

To test the integration, you can generate a report by running vuls report, like this:

sudo vuls report -to-slack


Vuls will take a few moments to run and exit successfully. If it shows an error, check what you’ve entered against the preceding lines.

You can check the Slack app and confirm that Vuls has successfully sent the report.

Now that you’ve configured reporting, you’ll set up scheduled scans. cron is a time-based job scheduler found on every Ubuntu machine. It is configured via the crontab file that defines in precise syntax when a command should run. To help ease the editing, you’ll use the crontab utility, which opens the current crontab file in an editor.

Open the current crontab file by running the following command:

crontab -e


When prompted, select your preferred text editor from the list.

Add the following line to the end of the file:

0 0 * * * vuls scan -config=/usr/share/vuls-data/config.toml; vuls report -config=/usr/share/vuls-data/config.toml > /dev/null 2>&1


The line above instructs cron to run vuls scan and vuls report with the given configuration every day at noon (denoted by 0 0 * * * in cron syntax).

Save and close the file.

In this step, you have connected Vuls to your Slack workspace and configured cron to run a Vuls scan and report every day at noon.

Conclusion

You have now successfully set up Vuls with automated scanning and reporting on an Ubuntu 18.04 server. For more reporting options, as well as troubleshooting, visit the Vuls documentation.

With Vuls, vulnerability assessment becomes more seamless for production environments. As an alternative to setting up cron, it is also possible to use Vuls in a continuous deployment workflow, as its scans are lightweight and you can run them as needed. You could also consider implementing a firewall with Vuls to restrict access and reduce the need for root access.

How to build a secure Grails 4 Application using Spring Security Core

How to build a secure Grails 4 Application using Spring Security Core

In this Grails 4 tutorial, we will show you how to build a secure Grails 4 application using Spring Security Core Plugin. We will add the login and register function to the Grails 4 application.

In this Grails 4 tutorial, we will show you how to build a secure Grails 4 application using Spring Security Core Plugin. We will add the login and register function to the Grails 4 application. The purpose of using the Spring Security plugin has simplified the integration of Spring Security Java (we have written this tutorial). The usage of this Grails 4 Spring Security plugin similar to Grails 2 or 3, but there's a lot of updates on the Spring Security code and its dependencies to match the compatibilities.

Table of Contents:

The flow of this tutorial is very simple. We have a secure Product list that only accessible to the authorized user with ROLE_USER and Product CRUD for the user with ROLE_ADMIN. Any access to this Product resource will be redirected to the Login Page as default (if no authorized user). In the login page, there will be a link to the registration page that will register a new user.

The following tools, frameworks, libraries, and dependencies are required for this tutorial:

  1. JDK 8
  2. Grails 4
  3. Grails Spring Security Core Plugin
  4. Terminal or Command Line
  5. Text Editor or IDE

Before starting the main steps, make sure you have downloaded and installed the latest Grails 4. In Mac, we are using the SDKMan. For that, type this command in the Terminal to install SDKMan.

curl -s https://get.sdkman.io | bash

Follow all instructions that showed up during installation. Next, open the new Terminal window or tab then type this command.

source "$HOME/.sdkman/bin/sdkman-init.sh"

Now, you can install Grails 4 using SDKMan.

sdk install grails 4.0.1

Set that new Grails 4 as default. To check the Grails version, type this command.

grails -version
| Grails Version: 4.0.1
| JVM Version: 1.8.0_92
Create Grails 4 Application

Same as previous Grails version, to create a new Grails application, simply type this command.

grails create-app com.djamware.gadgethouse

That command will create a new Grails 4 application with the name "gadgethouse" with the package name "com.djamware". Next, go to the newly created project folder then enter the Grails 4 interactive console.

cd ./gadgethouse
grails

In the Grails interactive console, type this command to run this Grails application for the first time.

grails> run-app

Here's the new Grails 4 look like.

Install Grails Spring Security Core Plugin

Now, we will install and configure Grails Spring Security Core Plugin. For the database, we keep H2 in-memory database (You can change to other relational database configuration). To install the Grails Spring Security Core Plugin, open and edit build.gradle then add this dependency in dependencies array.

compile "org.grails.plugins:spring-security-core:4.0.0.RC2"

Next, stop the running Grails application using this command in the Grails interactive console.

stop-app

Compile the Grails application, to install the Spring Security Core plugin.

compile
Create User, Role, and Product Domain Class

We will use the Grails s2-quickstart command to create User and Role domain class for authentication. Type this command in Grails interactive console.

s2-quickstart com.djamware User Role

That command will create User and Role domain class with the package name "com.djamware". Next, open and edit grails-app/domain/com/djamware/Role.groovy to add default field after the bracket closing when this domain calls.

String toString() {
  authority
}

Next, create a domain class using regular Grails command for Product.

create-domain-class com.djamware.Product

Next, we need to create an additional field in the user domain class. For that, open and edit grails-app/domain/com/djamware/User.groovy then add a fullName field after the password field.

String fullname

Also, add a constraint for that field.

static constraints = {
    password nullable: false, blank: false, password: true
    username nullable: false, blank: false, unique: true
    fullname nullable: false, blank: false
}

Next, open and edit grails-app/domain/com/djamware/Product.groovy then replace that domain class with these Groovy codes.

package com.djamware

class Product {

    String prodCode
    String prodName
    String prodModel
    String prodDesc
    String prodImageUrl
    String prodPrice

    static constraints = {
        prodCode nullable: false, blank: false
        prodName nullable: false, blank: false
        prodModel nullable: false, blank: false
        prodDesc nullable: false, blank: false
        prodImageUrl nullable: true
        prodPrice nullable: false, blank: false
    }

    String toString() {
        prodName
    }
}
Create CustomUserDetailsService

Because we have added a field in the previous User domain class, we need to create a custom UserDetails. Create a new Groovy file src/main/groovy/com/djamware/CustomUserDetails.groovy then add these lines of Groovy codes that add full name field to the Grails UserDetails.

package com.djamware

import grails.plugin.springsecurity.userdetails.GrailsUser
import org.springframework.security.core.GrantedAuthority

class CustomUserDetails extends GrailsUser {

   final String fullname

   CustomUserDetails(String username, String password, boolean enabled,
                 boolean accountNonExpired, boolean credentialsNonExpired,
                 boolean accountNonLocked,
                 Collection<GrantedAuthority> authorities,
                 long id, String fullname) {
      super(username, password, enabled, accountNonExpired,
            credentialsNonExpired, accountNonLocked, authorities, id)

      this.fullname = fullname
   }
}

Next, type this command in the Grails interactive console to create a new Grails service.

grails> create-service com.djamware.CustomUserDetails

Open that file then replace all Groovy codes with these codes.

package com.djamware

import grails.plugin.springsecurity.SpringSecurityUtils
import grails.plugin.springsecurity.userdetails.GrailsUserDetailsService
import grails.plugin.springsecurity.userdetails.NoStackUsernameNotFoundException
import grails.gorm.transactions.Transactional
import org.springframework.security.core.authority.SimpleGrantedAuthority
import org.springframework.security.core.userdetails.UserDetails
import org.springframework.security.core.userdetails.UsernameNotFoundException

class CustomUserDetailsService implements GrailsUserDetailsService {

   /**
    * Some Spring Security classes (e.g. RoleHierarchyVoter) expect at least
    * one role, so we give a user with no granted roles this one which gets
    * past that restriction but doesn't grant anything.
    */
   static final List NO_ROLES = [new SimpleGrantedAuthority(SpringSecurityUtils.NO_ROLE)]

   UserDetails loadUserByUsername(String username, boolean loadRoles)
         throws UsernameNotFoundException {
      return loadUserByUsername(username)
   }

   @Transactional(readOnly=true, noRollbackFor=[IllegalArgumentException, UsernameNotFoundException])
   UserDetails loadUserByUsername(String username) throws UsernameNotFoundException {

      User user = User.findByUsername(username)
      if (!user) throw new NoStackUsernameNotFoundException()

      def roles = user.authorities

      // or if you are using role groups:
      // def roles = user.authorities.collect { it.authorities }.flatten().unique()

      def authorities = roles.collect {
         new SimpleGrantedAuthority(it.authority)
      }

      return new CustomUserDetails(user.username, user.password, user.enabled,
            !user.accountExpired, !user.passwordExpired,
            !user.accountLocked, authorities ?: NO_ROLES, user.id,
            user.fullname)
   }
}

Next, register that new CustomUserDetailsService in the grails-app/conf/spring/resources.groovy.

import com.djamware.UserPasswordEncoderListener
import com.djamware.CustomUserDetailsService
// Place your Spring DSL code here
beans = {
    userPasswordEncoderListener(UserPasswordEncoderListener)
    userDetailsService(CustomUserDetailsService)
}
Override Login Auth View

We will customize the login page to make UI better and add a link to the Register page. For that, create a login folder under views then create an auth.gsp file in that folder. Open and edit grails-app/views/login/auth.gsp then add these lines of GSP HTML tags.

<html>
<head>
    <meta name="layout" content="${gspLayout ?: 'main'}"/>
    <title><g:message code='springSecurity.login.title'/></title>
</head>

<body>
    <div class="row">
      <div class="col-sm-9 col-md-7 col-lg-5 mx-auto">
        <div class="card card-signin my-5">
          <div class="card-body">
            <h5 class="card-title text-center">Please Login</h5>
            <g:if test='${flash.message}'>
                <div class="alert alert-danger" role="alert">${flash.message}</div>
            </g:if>
            <form class="form-signin" action="${postUrl ?: '/login/authenticate'}" method="POST" id="loginForm" autocomplete="off">
              <div class="form-group">
                  <label for="username">Username</label>
                <input type="text" class="form-control" name="${usernameParameter ?: 'username'}" id="username" autocapitalize="none"/>
              </div>

              <div class="form-group">
                  <label for="password">Password</label>
                <input type="password" class="form-control" name="${passwordParameter ?: 'password'}" id="password"/>
                <i id="passwordToggler" title="toggle password display" onclick="passwordDisplayToggle()">&#128065;</i>
              </div>

              <div class="form-group form-check">
                  <label class="form-check-label">
                      <input type="checkbox" class="form-check-input" name="${rememberMeParameter ?: 'remember-me'}" id="remember_me" <g:if test='${hasCookie}'>checked="checked"</g:if>/> Remember me
                </label>
              </div>
              <button id="submit" class="btn btn-lg btn-primary btn-block text-uppercase" type="submit">Sign in</button>
              <hr class="my-4">
              <p>Don't have an account? <g:link controller="register">Register</g:link></p>
            </form>
          </div>
        </div>
      </div>
    </div>
    <script type="text/javascript">
        document.addEventListener("DOMContentLoaded", function(event) {
            document.forms['loginForm'].elements['username'].focus();
        });
        function passwordDisplayToggle() {
            var toggleEl = document.getElementById("passwordToggler");
            var eyeIcon = '\u{1F441}';
            var xIcon = '\u{2715}';
            var passEl = document.getElementById("password");
            if (passEl.type === "password") {
                toggleEl.innerHTML = xIcon;
                passEl.type = "text";
            } else {
                toggleEl.innerHTML = eyeIcon;
                passEl.type = "password";
            }
        }
    </script>
</body>
</html>

Next, we will make this login page as default or homepage when the application opens in the browser. For that, open and edit grails-app/controllers/UrlMappings.groovy then replace this line.

"/"(view: "index")

With this line.

"/"(controller:'login', action:'auth')
Add User Info and Logout to the Navbar

Now, we have to implement POST logout to the Navbar. This logout button active when the user logged in successfully along with user info. For that, modify grails-app/views/layout/main.gsp then replace all GSP HTML tags with these.

<!doctype html>
<html lang="en" class="no-js">
<head>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
    <meta http-equiv="X-UA-Compatible" content="IE=edge"/>
    <title>
        <g:layoutTitle default="Grails"/>
    </title>
    <meta name="viewport" content="width=device-width, initial-scale=1"/>
    <asset:link rel="icon" href="favicon.ico" type="image/x-ico"/>

    <asset:stylesheet src="application.css"/>

    <g:layoutHead/>
</head>

<body>

<nav class="navbar navbar-expand-lg navbar-dark navbar-static-top" role="navigation">
    <a class="navbar-brand" href="/#"><asset:image src="grails.svg" alt="Grails Logo"/></a>
    <button class="navbar-toggler" type="button" data-toggle="collapse" data-target="#navbarContent" aria-controls="navbarContent" aria-expanded="false" aria-label="Toggle navigation">
        <span class="navbar-toggler-icon"></span>
    </button>

    <div class="collapse navbar-collapse" aria-expanded="false" style="height: 0.8px;" id="navbarContent">
        <ul class="nav navbar-nav ml-auto">
            <g:pageProperty name="page.nav"/>
            <sec:ifLoggedIn>
              <li class="nav-item dropdown">
                  <a class="nav-link dropdown-toggle" href="#" id="navbardrop" data-toggle="dropdown">
                    <sec:loggedInUserInfo field='fullname'/>
                  </a>
                  <div class="dropdown-menu navbar-dark">
                    <g:form controller="logout">
                      <g:submitButton class="dropdown-item navbar-dark color-light" name="Submit" value="Logout" style="color:gray" />
                    </g:form>
                  </div>
              </li>
            </sec:ifLoggedIn>
        </ul>
    </div>

</nav>

<div class="container">
    <g:layoutBody/>
</div>

<div class="footer row" role="contentinfo">
    <div class="col">
        <a href="http://guides.grails.org" target="_blank">
            <asset:image src="advancedgrails.svg" alt="Grails Guides" class="float-left"/>
        </a>
        <strong class="centered"><a href="http://guides.grails.org" target="_blank">Grails Guides</a></strong>
        <p>Building your first Grails app? Looking to add security, or create a Single-Page-App? Check out the <a href="http://guides.grails.org" target="_blank">Grails Guides</a> for step-by-step tutorials.</p>

    </div>
    <div class="col">
        <a href="http://docs.grails.org" target="_blank">
            <asset:image src="documentation.svg" alt="Grails Documentation" class="float-left"/>
        </a>
        <strong class="centered"><a href="http://docs.grails.org" target="_blank">Documentation</a></strong>
        <p>Ready to dig in? You can find in-depth documentation for all the features of Grails in the <a href="http://docs.grails.org" target="_blank">User Guide</a>.</p>

    </div>

    <div class="col">
        <a href="https://grails-slack.cfapps.io" target="_blank">
            <asset:image src="slack.svg" alt="Grails Slack" class="float-left"/>
        </a>
        <strong class="centered"><a href="https://grails-slack.cfapps.io" target="_blank">Join the Community</a></strong>
        <p>Get feedback and share your experience with other Grails developers in the community <a href="https://grails-slack.cfapps.io" target="_blank">Slack channel</a>.</p>
    </div>
</div>

<div id="spinner" class="spinner" style="display:none;">
    <g:message code="spinner.alt" default="Loading&hellip;"/>
</div>

<asset:javascript src="application.js"/>

</body>
</html>

As you see, there are built in Grails Spring Security TagLib sec:ifLoggedIn and <sec:loggedInUserInfo field='fullname'/>. The <sec:loggedInUserInfo field='fullname'/> only working when you implementing CustomUserDetailsService.

Create Register Controller and View

Back to the Grails interactive console to create a controller for the Register page.

grails> create-controller com.djamware.Register

Open and edit that Groovy file then replace all Groovy codes with these codes that have 2 methods of Register landing page and register action.

package com.djamware

import grails.validation.ValidationException
import grails.gorm.transactions.Transactional
import grails.plugin.springsecurity.annotation.Secured
import com.djamware.User
import com.djamware.Role
import com.djamware.UserRole

@Transactional
@Secured('permitAll')
class RegisterController {

    static allowedMethods = [register: "POST"]

    def index() { }

    def register() {
        if(!params.password.equals(params.repassword)) {
            flash.message = "Password and Re-Password not match"
            redirect action: "index"
            return
        } else {
            try {
                def user = User.findByUsername(params.username)?: new User(username: params.username, password: params.password, fullname: params.fullname).save()
                def role = Role.get(params.role.id)
                if(user && role) {
                    UserRole.create user, role

                    UserRole.withSession {
                      it.flush()
                      it.clear()
                    }

                    flash.message = "You have registered successfully. Please login."
                    redirect controller: "login", action: "auth"
                } else {
                    flash.message = "Register failed"
                    render view: "index"
                    return
                }
            } catch (ValidationException e) {
                flash.message = "Register Failed"
                redirect action: "index"
                return
            }
        }
    }
}

Next, add the index.gsp inside grails-app/views/register/. Open and edit that file then add these lines of GSP HTML tags.

<html>
<head>
    <meta name="layout" content="${gspLayout ?: 'main'}"/>
    <title>Register</title>
</head>

<body>
    <div class="row">
    <div class="col-sm-9 col-md-7 col-lg-5 mx-auto">
      <div class="card card-signin my-5">
        <div class="card-body">
          <h5 class="card-title text-center">Register Here</h5>
                    <g:if test='${flash.message}'>
                        <div class="alert alert-danger" role="alert">${flash.message}</div>
                    </g:if>
              <form class="form-signin" action="register" method="POST" id="loginForm" autocomplete="off">
                      <div class="form-group">
                          <label for="role">Role</label>
              <g:select class="form-control" name="role.id"
                    from="${com.djamware.Role.list()}"
                    optionKey="id" />
                </div>

            <div class="form-group">
                    <label for="username">Username</label>
              <input type="text" placeholder="Your username" class="form-control" name="username" id="username" autocapitalize="none"/>
            </div>

            <div class="form-group">
                          <label for="password">Password</label>
              <input type="password" placeholder="Your password" class="form-control" name="password" id="password"/>
            </div>

            <div class="form-group">
                          <label for="password">Re-Enter Password</label>
              <input type="password" placeholder="Re-enter password" class="form-control" name="repassword" id="repassword"/>
            </div>

                      <div class="form-group">
                          <label for="username">Full Name</label>
              <input type="text" placeholder="Your full name" class="form-control" name="fullname" id="fullname" autocapitalize="none"/>
            </div>

            <button id="submit" class="btn btn-lg btn-primary btn-block text-uppercase" type="submit">Register</button>
            <hr class="my-4">
            <p>Already have an account? <g:link controller="login" action="auth">Login</g:link></p>
          </form>
        </div>
      </div>
    </div>
  </div>
    <script type="text/javascript">
        document.addEventListener("DOMContentLoaded", function(event) {
            document.forms['loginForm'].elements['username'].focus();
        });
    </script>
</body>
</html>
Create the Secure Product CRUD Scaffolding

Now, we will make Product CRUD scaffolding and make them secured and accessible to ROLE_USER and ROLE_ADMIN. To create CRUD scaffolding, simply run this command inside Grails interactive console.

grails>generate-all com.djamware.Product

That command will generate controller, service, and view for a Product domain class. Next, open and edit grails-app/controllers/ProductController.groovy then add the Secure annotation like these.

package com.djamware

import grails.validation.ValidationException
import static org.springframework.http.HttpStatus.*
import grails.plugin.springsecurity.annotation.Secured

class ProductController {

    ProductService productService

    static allowedMethods = [save: "POST", update: "PUT", delete: "DELETE"]

    @Secured(['ROLE_ADMIN', 'ROLE_USER'])
    def index(Integer max) {
        params.max = Math.min(max ?: 10, 100)
        respond productService.list(params), model:[productCount: productService.count()]
    }

    @Secured(['ROLE_ADMIN', 'ROLE_USER'])
    def show(Long id) {
        respond productService.get(id)
    }

    @Secured('ROLE_ADMIN')
    def create() {
        respond new Product(params)
    }

    @Secured('ROLE_ADMIN')
    def save(Product product) {
        if (product == null) {
            notFound()
            return
        }

        try {
            productService.save(product)
        } catch (ValidationException e) {
            respond product.errors, view:'create'
            return
        }

        request.withFormat {
            form multipartForm {
                flash.message = message(code: 'default.created.message', args: [message(code: 'product.label', default: 'Product'), product.id])
                redirect product
            }
            '*' { respond product, [status: CREATED] }
        }
    }

    @Secured('ROLE_ADMIN')
    def edit(Long id) {
        respond productService.get(id)
    }

    @Secured('ROLE_ADMIN')
    def update(Product product) {
        if (product == null) {
            notFound()
            return
        }

        try {
            productService.save(product)
        } catch (ValidationException e) {
            respond product.errors, view:'edit'
            return
        }

        request.withFormat {
            form multipartForm {
                flash.message = message(code: 'default.updated.message', args: [message(code: 'product.label', default: 'Product'), product.id])
                redirect product
            }
            '*'{ respond product, [status: OK] }
        }
    }

    @Secured('ROLE_ADMIN')
    def delete(Long id) {
        if (id == null) {
            notFound()
            return
        }

        productService.delete(id)

        request.withFormat {
            form multipartForm {
                flash.message = message(code: 'default.deleted.message', args: [message(code: 'product.label', default: 'Product'), id])
                redirect action:"index", method:"GET"
            }
            '*'{ render status: NO_CONTENT }
        }
    }

    protected void notFound() {
        request.withFormat {
            form multipartForm {
                flash.message = message(code: 'default.not.found.message', args: [message(code: 'product.label', default: 'Product'), params.id])
                redirect action: "index", method: "GET"
            }
            '*'{ render status: NOT_FOUND }
        }
    }
}

Next, we will make Product controller as a default landing page after succeful login. For that, open and edit grails-app/conf/application.groovy then add this configuration.

grails.plugin.springsecurity.successHandler.defaultTargetUrl = '/product'
Run and Test Grails 4 Spring Security Core

Now, we will run and test the Grails 4 Spring Security Core application. In the Grails interactive console run this command.

grails>run-app

Open the browser then go to http://localhost:8080 and you will see this page.




That it's, the Grails 4 Tutorial: Spring Security Core Login Example. You can find the full working source code in our GitHub.

Thanks!

Two-Factor Authentication for SSH Ubuntu

Two-Factor Authentication for SSH Ubuntu

In this post, we first understand the Two Factor Authentication (2FA) and why do we need it and types of the authentication methods. We learn about the implementation of 2FA using Duo in ubuntu.

Originally published by Ankit Jain at https://blog.bitsrc.io

Two Factor Authentication (2FA) is a two-step verification or more appropriately two-step authentication method. A user provides two different authentication factors to verify themselves to protect the resources that he can access. Two Factor Authentication adds an additional layer of Security to our Single Factor Authentication (SFA) in which the user only needs to provide a password or passcode.

Two Factor Authentication method is almost used by everyone to increase the security and make it difficult for the attackers to gain access to the user’s account because knowing the user’s password alone is not enough for completing the authentication process.

Why Two Factor Authentication?

As I said earlier, it is required to make the authentication more secure as the password is not enough to gain access to the user’s account. We also need to complete the additional authentication step which is only known to the verified user. There is a term called Out-of-Band Authentication (OOBA)

Source: Out-of-Band Authentication

Out-of-Band Authentication refers to 2FA in which we conduct the second step authentication over the different network from the primary network/channel. This is because in case a hacker is able to intercept our requests over the primary network, he will have our username and password but as we have added an additional layer of security which is on some other network that he doesn’t have any access, he won’t be able to access user’s resources.

Types of Authentication Methods

Authentication is generally of Three types

  1. Something we know:
  2. It is something that we know or memorise like passwords, passcodes, security questions etc.
  3. Something we have
  4. It refers to hardware-based authentication like codes sent to a mobile device via SMS, authentication via a voice channel, codes sent to a mobile app via push notifications, U2F etc.
  5. Something we are
  6. It refers to biometric authentication like fingerprints, iris scan etc.
Two Factor Authentication for SSH

2FA for SSH is generally used in Enterprises where we need to follow high-security measures to keep our data safe and secure or in case we are heavily dealing with some critical user information which needs to be kept safe. In such scenarios, we increase the level of security by adding another layer in Secure Shell (SSH).

For implementing 2FA in Linux distributions, we need to make some changes in the Pluggable Authentication Modules (PAM). We are using Duo as a third party service for 2FA. We can also use any other third party service like Google Authenticator etc.

Prerequisites
  1. Ubuntu machine with an SSH key and firewall enabled. (Recommend VMs for testing)
  2. Duo Keys (integration key, secret key, and API hostname).
  3. Log in to the Duo Admin Account and click Protect an Application under Applications and locate UNIX Application in the applications list. Click Protect this Application to get your integration key, secret key, and API hostname. See Getting Started for help.
Setup

Let’s set up 2FA for ssh in ubuntu using Duo. During SSH, the client tries to connect with the server at port 22 for which the client has to pass either key-based or password-based authentication. During this whole process, we pass the flow to the PAM modules through which we extend this authentication process and add another level of authentication in which the client verifies themselves.

For using the PAM module, we need the pam_duo binary to authenticate using Duo.

Install pam_duo

pam_duo binary requires some Openssl headers and libraries and other libraries like libpam during compilation.

sudo apt-get install autoconf libtool libpam-dev libssl-dev

After installing the required dependencies, let’s build and install pam_duo

Follow these steps

  1. Download and extract the latest version of duo_unix (checksum)
$ wget https://dl.duosecurity.com/duo_unix-latest.tar.gz 

$ tar zxf duo_unix-latest.tar.gz

$ cd duo_unix-1.11.2

2. Build and install the pam_duo

./configure --with-pam --prefix=/tmp && sudo make install

3. Let’s create a pam_duo.conf file for setting up duo configuration.

# /etc/duo/pam_duo.conf[duo]

; Duo integration key

ikey =

; Duo secret key

skey =

; Duo API host

host =

We can also add additional options to configure our pam_duo, Read them in Duo Configuration Options for all available settings.

4. For using pam_duo with public key authentication, we need to make changes in the ssh daemon sshd_config file (usually in /etc/ssh).

PubkeyAuthentication yes

PasswordAuthentication no

AuthenticationMethods publickey,keyboard-interactive

UsePAM yes

ChallengeResponseAuthentication yes

UseDNS no

5. Now, we need to add our binary i.e pam_duo.so which we have generated in the step-1 in/etc/pam.d/sshd file.

# @include common-auth

auth  [success=1 default=ignore] /lib64/security/pam_duo.so

auth  requisite pam_deny.so

auth  required pam_permit.so

We have commented the common-auth file that contains some other PAM modules like pam_unix.so, pam_deny.so and pam_permit.so that checks for password-based authentication if configured.

6. Restart the SSH service to apply the updated configuration.

systemctl restart sshd
Test our 2FA

Let’s try to SSH localhost to check whether everything is working properly or not before logging out the system else we will get locked and won’t able to SSH again in case anything gets wrong.

# keygen

$ ssh-keygen -t rsa$ cat ~/.ssh/id_rsa.pub >> authorized_keys

$ ssh localhost

For the very first time, we will get an output as shown in the image below, asking us to enroll at the given URL so as we can set up our device for 2FA.

ssh localhost

On opening the link, we will get this page where we have to follow the instructions and proceed with our setup.

I have added my phone for the 2FA for which I need to install the Duo app where I will get push notifications. We can add other devices like a tablet, landline, touch Id etc. Once we complete the setup, we are good to SSH again to test our 2FA. Let’s try again

ssh localhost

Choose your authentication method and approve the request using your enrolled device to log in the system.

Demo

I have created a GitHub repo with 2FA configured using Ubuntu Vagrant machine.

Follow the steps

1. Install dependencies

VirtualBox 4.3.10 or greater.

Vagrant 1.6.3 or greater.

2. Clone this project and get it running!

$ git clone https://github.com/ankitjain28may/pam_duo_linux

$ cd pam_duo_linux/ubuntu

3. Visit the Duo Admin Panel and create a “Unix” integration if you don’t have one already.

4. Copy your ikey, skey, and api_host into the duo.env file

mv duo.env.example duo.env

5. Open Vagrantfile and change ./install.sh ankit to whatever username you want -> ./install.sh username.

6. Run this commands-

# To craete a ubuntu VM

$ vagrant up # First time

$ vagrant ssh# create keys and try local ssh so as you can enroll your device

$ sudo su - username

$ ssh-keygen -t rsa

$ cat ~/.ssh/id_rsa.pub >> authorized_keys

$ ssh localhost# SSH in VM

$ ssh [email protected] -p 2222

Note: In case you are facing issues while SSH using username, open Vagrantfile and add this-

config.ssh.username = ‘username’ # Now SSH again

$ vagrant ssh

Read more about pam_duo from its docs: Duo Unix — Two-Factor Authentication for SSH with PAM Support (pam_duo)

Conclusion

In this post, we first understand the Two Factor Authentication (2FA) and why do we need it and types of the authentication methods. We learn about the implementation of 2FA using Duo in ubuntu. Duo has its own advantage like we can manage users, devices from Duo Admin panel and many more. The code is available in this Github Repo.

Thanks for reading

If you liked this post, share it with all of your programming buddies!

Follow us on Facebook | Twitter