How To Use Vuls as a Vulnerability Scanner on Ubuntu 18.04
In this tutorial, you’ll deploy Vuls to an Ubuntu 18.04 server. This includes building Vuls and its dependencies from source code, configuring scanning and reporting to Slack, and optionally connecting it to target machines to enable remote scanning. In the end, you’ll have an automated vulnerability reporting system in place that alerts you to vulnerabilities eliminating the need for manual checks.
Introduction
Vuls is an open-source, agentless vulnerability scanner written in Go. It automates security vulnerability analysis of the software installed on a system, which can be a burdensome task for system administrators to do manually in a production environment. Vuls uses multiple renowned vulnerability databases, such as the National Vulnerability Database (NVD). Light on resources, Vuls has the ability to scan multiple systems at once, and to send reports via email or Slack. It has three scan modes (fast, fast root, and deep), which you can select according to the situation.
Vuls is not a broad IT security scanner; for example, it does not monitor network traffic or protect against brute-force login attacks. However, Vuls provides a way of automating vulnerability reporting for Linux packages. When the databases Vuls uses are informed of a fix to certain vulnerabilities, Vuls will also pull this remediation information into its reports. When generating reports, Vuls prioritizes the most urgent vulnerabilities using the established ranking system from the database.
Prerequisites
Before you begin this tutorial, you’ll need:
- A server with at least 2 GB RAM running Ubuntu 18.04 with root access, and a secondary, non-root account. You can set this up by following this initial server setup guide. For this tutorial the non-root user is
sammy. - (Optional) Multiple servers running (preferably) Ubuntu 18.04 with root access and a secondary, non-root account, if you want to set up Vuls to scan them remotely. In this tutorial, the secondary account is
sammy-shark.
Step 1 — Installing Dependencies
In this section, you’ll create a folder for storing Vuls data, install the latest version of the Go programming language, and install other packages Vuls and its dependencies require.
Start off by logging in as sammy:
ssh sammy@your_server_ip
For this tutorial, you’ll store all Vuls-related data in the /usr/share/vuls-data directory. Create it by running the following command:
sudo mkdir /usr/share/vuls-data
To make it accessible to sammy, run the following command:
sudo chown -R sammy /usr/share/vuls-data
You’ve now created the vuls-data folder, which will be your workspace. Before you continue installing the required packages, first update the package manager cache:
sudo apt update
To download and compile the dependencies, you’ll install git, gcc, make, sqlite, debian-goodies, golang-go, and wget.
sqlite is a database system, which you’ll use here for storing vulnerability information. [debian-goodies](https://packages.debian.org/stretch/debian-goodies "debian-goodies") contains the checkrestart utility, which provides information on what packages can and should be restarted at any given moment in time. golang-go is the Go programming language.
You can install them all in one command:
sudo apt install sqlite git debian-goodies gcc make wget golang-go -y
You have now installed the required packages, including Go.
In order to work, Go requires a few environment variables that you’ll set up: GOPATH and PATH. GOPATH specifies the working directory for Go and PATH (which contains directories in which programs are placed) that must be extended to tell the system where to find Go itself.
These environment variables need to be set each time the user logs on. To automate this, you will create a new executable file, called go-env.sh, under /etc/profile.d. This will result in the directory executing every time a user logs on.
Create go-env.sh using your text editor:
sudo nano /etc/profile.d/go-env.sh
Add the following commands to the file:
/etc/profile.d/go-env.sh
export GOPATH=$HOME/go
export PATH=$PATH:$GOPATH/bin
The export command sets the given environment variable to the desired value; here you use it to populate GOPATH and PATH with appropriate values.
Save and close the file.
Currently, go-env.sh is not executable. To fix this, mark it as executable by running the following command:
sudo chmod +x /etc/profile.d/go-env.sh
To avoid having to log in again, you can reload go-env.sh by running:
source /etc/profile.d/go-env.sh
The source command reloads the given file into the current shell while preserving its state.
In this section, you have installed the Go language, set up its environment variables, and installed packages that you’ll require later on. In the next steps, you’ll download and compile the Go programs that Vuls requires. Those programs are go-cve-dictionary and goval-dictionary, which Vuls uses for querying vulnerability databases.
Step 2 — Installing and Running go-cve-dictionary
In this section, you will download and compile go-cve-dictionary, a Go package that provides access to the NVD (National Vulnerability Database). Then, you will run it and fetch vulnerability data for Vuls to use. The NVD is the US government’s repository of publicly reported cybersecurity vulnerabilities, containing vulnerability IDs (CVE — Common Vulnerabilities and Exposures), summaries, and impact analysis, and is available in a machine-readable format.
Go stores packages under $GOPATH/src/. You can extend this further with the use of subdirectories to note origin. As an example, packages from GitHub, made by the user, example-user would be stored under $GOPATH/src/github.com/example-user.
You’ll first install go-cve-dictionary, made by kotakanbe, by cloning the Go package from GitHub and compiling it afterwards.
Start off by creating a directory to store it, according to the example path:
mkdir -p $GOPATH/src/github.com/kotakanbe
Navigate to it by running:
cd $GOPATH/src/github.com/kotakanbe
Now you’ll clone go-cve-dictionary from GitHub to your server by running:
git clone https://github.com/kotakanbe/go-cve-dictionary.git
Then, navigate to the package root:
cd go-cve-dictionary
Finally, compile and install it by running the following command:
make install
Keep in mind that this command may take some time to finish. To make it available system wide, copy it to the /usr/local/bin:
sudo cp $GOPATH/bin/go-cve-dictionary /usr/local/bin
go-cve-dictionary requires access to a log output directory, and by default it is /var/log/vuls. Create it by running:
sudo mkdir /var/log/vuls
Right now, the log directory is readable by everyone. Restrict access to the current user with the following command:
sudo chmod 700 /var/log/vuls
Setting the permission flags to 700 restricts access to only the owner.
To make it accessible to sammy, or another user, run the following command:
sudo chown -R sammy /var/log/vuls
Now, you’ll fetch vulnerability data from the NVD and store it in your Vuls workspace (/usr/share/vuls-data):
for i in `seq 2002 $(date +"%Y")`; do sudo go-cve-dictionary fetchnvd -dbpath /usr/share/vuls-data/cve.sqlite3 -years $i; done
This command loops from the year 2002 to the current year (seq 2002 $(date +"%Y")) and calls go-cve-dictionary fetchnvd to fetch the NVD data for the current (loop) year by passing -years $i. It then stores this information in a database under /usr/share/vuls-data.
Note: This command will take a long time to finish, and will fail if your server has less than 2 GB of RAM.
In this step, you have downloaded and installed go-cve-dictionary, and fetched NVD data for Vuls to later use. In the next section, you’ll download and install goval-dictionary and fetch OVAL data for Ubuntu.
Step 3 — Installing and Running goval-dictionary
In this section, you will download and compile goval-dictionary, a Go package that provides access to the OVAL database for Ubuntu. You’ll then run it and fetch vulnerability data for Vuls to use. OVAL stands for Open Vulnerability and Assessment Language, which is an open language used to express checks for determining whether software vulnerabilities exist on a given system.
The same author, kotakanbe, writes the goval-dictionary, and you’ll store it next to the previous package.
Navigate to the $GOPATH/src/github.com/kotakanbe folder:
cd $GOPATH/src/github.com/kotakanbe
Clone the package from GitHub by running the following command:
git clone https://github.com/kotakanbe/goval-dictionary.git
Enter the package folder:
cd goval-dictionary
Compile and install it with make:
make install
Copy it to /usr/local/bin to make it globally accessible:
sudo cp $GOPATH/bin/goval-dictionary /usr/local/bin
Then, fetch the OVAL data for Ubuntu 18.x by running the following command:
sudo goval-dictionary fetch-ubuntu -dbpath=/usr/share/vuls-data/oval.sqlite3 18
In this step, you have downloaded and installed goval-dictionary, and fetched the OVAL data for Ubuntu 18.x. In the next step, you’ll download and install Vuls.
Step 4 — Downloading and Configuring Vuls
With all of the dependencies installed, now you’ll download and compile Vuls from source code. Afterward, you’ll configure it to scan the local machine.
Create a new directory that contains the path to the Vuls repository, with the following command:
mkdir -p $GOPATH/src/github.com/future-architect
Navigate to it:
cd $GOPATH/src/github.com/future-architect
Clone Vuls from GitHub by running the following command:
git clone https://github.com/future-architect/vuls.git
Enter the package folder:
cd vuls
Compile and install it at the same time by running:
make install
Remember that it may take some time for this command to complete.
Copy it to /usr/local/bin to make it globally accessible:
sudo cp $GOPATH/bin/vuls /usr/local/bin
Now, you’ll create a configuration file for Vuls. Navigate back to /usr/share/vuls-data:
cd /usr/share/vuls-data
Vuls stores its configuration in a TOML file, which you’ll call config.toml. Create it using your text editor:
sudo nano config.toml
Enter the following configuration:
/usr/share/vuls-data/config.toml
[cveDict]
type = "sqlite3"
SQLite3Path = "/usr/share/vuls-data/cve.sqlite3"
[ovalDict]
type = "sqlite3"
SQLite3Path = "/usr/share/vuls-data/oval.sqlite3"
[servers]
[servers.localhost]
host = "localhost"
port = "local"
scanMode = [ "fast" ]
#scanMode = ["fast", "fast-root", "deep", "offline"]
The first two sections of this configuration (cveDict and ovalDict) point Vuls to the vulnerability databases you created in the last two steps. The next section (servers) marks the start of server-related information. Separate sections will group information about each server. The only server Vuls will scan with this outlined configuration is the local server (localhost).
Vuls provides four scan modes:
- A server with at least 2 GB RAM running Ubuntu 18.04 with root access, and a secondary, non-root account. You can set this up by following this initial server setup guide. For this tutorial the non-root user is
sammy. - (Optional) Multiple servers running (preferably) Ubuntu 18.04 with root access and a secondary, non-root account, if you want to set up Vuls to scan them remotely. In this tutorial, the secondary account is
sammy-shark.
Save and close the file.
To test the validity of the configuration file, run the following command:
vuls configtest
You’ll see the following output:
[Feb 27 19:36:42] INFO [localhost] Validating config...
[Feb 27 19:36:42] INFO [localhost] Detecting Server/Container OS...
[Feb 27 19:36:42] INFO [localhost] Detecting OS of servers...
[Feb 27 19:36:42] INFO [localhost] (1/1) Detected: localhost: ubuntu 18.04
[Feb 27 19:36:42] INFO [localhost] Detecting OS of containers...
[Feb 27 19:36:42] INFO [localhost] Checking Scan Modes...
[Feb 27 19:36:42] INFO [localhost] Checking dependencies...
[Feb 27 19:36:42] INFO [localhost] Dependencies... Pass
[Feb 27 19:36:42] INFO [localhost] Checking sudo settings...
[Feb 27 19:36:42] INFO [localhost] sudo ... No need
[Feb 27 19:36:42] INFO [localhost] It can be scanned with fast scan mode even if warn or err messages are displayed due to lack of dependent packages or sudo settings in fast-root or deep scan mode
[Feb 27 19:36:42] INFO [localhost] Scannable servers are below...
localhost
You’ve entered the configuration correctly, and Vuls has detected that it can scan the local server.
You’ve installed and configured Vuls to scan the local server. In the next step, you will run a local scan and view the generated report.
Step 5 — Running a Local Scan
In this section, you will run a local scan and then view the generated vulnerability report. By now, you have configured only the local server, which Vuls correctly detected in the last step. The default scan mode, if not explicitly specified, is fast.
To run a scan, execute the following command:
vuls scan
You’ll see output similar to this:
[Feb 27 19:44:12] INFO [localhost] Start scanning
[Feb 27 19:44:12] INFO [localhost] config: /usr/share/vuls-data/config.toml
[Feb 27 19:44:12] INFO [localhost] Validating config...
[Feb 27 19:44:12] INFO [localhost] Detecting Server/Container OS...
[Feb 27 19:44:12] INFO [localhost] Detecting OS of servers...
[Feb 27 19:44:12] INFO [localhost] (1/1) Detected: localhost: ubuntu 18.04
[Feb 27 19:44:12] INFO [localhost] Detecting OS of containers...
[Feb 27 19:44:12] INFO [localhost] Checking Scan Modes...
[Feb 27 19:44:12] INFO [localhost] Detecting Platforms...
[Feb 27 19:44:12] INFO [localhost] (1/1) localhost is running on other
[Feb 27 19:44:12] INFO [localhost] Scanning vulnerabilities...
[Feb 27 19:44:12] INFO [localhost] Scanning vulnerable OS packages...
[Feb 27 19:44:12] INFO [localhost] Scanning in fast mode
One Line Summary
================
localhost ubuntu18.04 539 installed
To view the detail, vuls tui is useful.
To send a report, run vuls report -h.
Vuls has logged what it did in the process. To view a report of vulnerabilities it has identified, run:
vuls tui
Vuls divides the report view into four panels:
- A server with at least 2 GB RAM running Ubuntu 18.04 with root access, and a secondary, non-root account. You can set this up by following this initial server setup guide. For this tutorial the non-root user is
sammy. - (Optional) Multiple servers running (preferably) Ubuntu 18.04 with root access and a secondary, non-root account, if you want to set up Vuls to scan them remotely. In this tutorial, the secondary account is
sammy-shark.
You can cycle the cursor through the panels by pressing ENTER, and navigate with the keyboard arrows.
In this step, you have run a local scan and inspected the results. In the next optional section, you’ll configure Vuls to scan multiple target machines.
Step 6 — (Optional) Configuring Multiple Target Machines
In this section, you’ll configure Vuls to scan multiple target machines. This entails configuring /etc/sudoers on the target and configuring Vuls to scan the target.
In the previous step, you configured Vuls to scan the local machine (localhost). You can add as many servers as you wish, provided you have the following:
- A server with at least 2 GB RAM running Ubuntu 18.04 with root access, and a secondary, non-root account. You can set this up by following this initial server setup guide. For this tutorial the non-root user is
sammy. - (Optional) Multiple servers running (preferably) Ubuntu 18.04 with root access and a secondary, non-root account, if you want to set up Vuls to scan them remotely. In this tutorial, the secondary account is
sammy-shark.
You can only use a non-root user account on the target server for scanning in fast mode. To enable scanning in fast root and deep modes, you’ll need to edit the /etc/sudoers file on the target machine(s). The sudoers file controls which users can run what commands, and also whether you need a password for specified commands.
Since visudo is the utility for defining rules for access and privileged access, you can only run it as root. Because of the importance of sudoers, the file will not exit with errors without giving a warning.
On the target server, log in as root and open sudoers for editing by running visudo:
visudo
Add this line to the end of the file:
/etc/sudoers
sammy-shark ALL=(ALL) NOPASSWD: /usr/bin/apt-get update, /usr/bin/stat *, /usr/sbin/checkrestart
This line instructs sudo to allow user sammy-shark to run apt-get update, checkrestart, and every command available from stat, without providing a password.
Save and close the file. If you made a syntax error in the process, visudo will inform you and offer to edit it again or exit.
Note: By allowing the sammy-shark user in sudoers, you are allowing Vuls to scan using fast root and deep modes. If you want to allow those modes for the local machine (localhost) too, edit sudoers on localhost as shown earlier.
Vuls uses the checkrestart utility to check for packages that are updated, but require restart. To ensure the target server has it, log in as your non-root user, and install it by running the following command:
sudo apt install debian-goodies -y
That is all you need to do on the target server; you can now log out from the target and log back in to your first server.
To add a new server for scanning, open config.toml and add the following lines under the [servers] mark:
/usr/share/vuls-data/config.toml
[servers.target_name]
host = "target_ip"
port = "22"
user = "account_username"
keyPath = "account_rsa_key"
scanMode = [ "deep" ] # "fast", "fast-root" or "deep"
The lines above serve as a template for adding new servers. Remember to replace target_name with the desired name, target_ip with the IP of the target server, account_username with the username, and account_rsa_key with the path to the RSA key. Vuls does not support SSH password authentication, so specifying a keyPath is necessary.
Save and close the file.
Next, for each target server you’ve added, you’ll confirm the RSA keys on the local machine. To achieve this, you’ll log in to the target server from your first server with the appropriate key, like so:
ssh sammy-shark@target_ip -i account_rsa_key
When asked whether you want to continue connecting, enter yes, then log out by pressing CTRL + D.
If you get an error about key file permissions being too open, set them to 600 by running the following command:
chmod 600 account_rsa_key
Setting permissions to 600 ensures that only the owner can read and write the key file.
To check the validity of the new configuration, run the following command:
vuls configtest
There will be no output from this command. If there are any errors, check your config.toml against the configuration in the tutorial.
In this step, you’ve added more target servers to your Vuls configuration, thus marking them for scanning. In the next section, you will configure Vuls to periodically scan and send reports to a configured Slack workspace.
Step 7 — Configuring Periodic Scanning and Reporting to Slack
In this section, you will configure Vuls to send reports to Slack and make a cron job to run Vuls scans periodically.
To use Slack integration, you’ll need to have an incoming webhook on Slack for your workspace. Incoming webhooks are a simple way of an application providing other applications real-time information. In this case, you’ll be configuring the Vuls to report to your Slack channel.
If you haven’t ever created a webhook, you’ll first need to create an app for your workspace. To do so, first log in to Slack and navigate to the app creation page. Pick a name that you’ll recognize, select the desired workspace, and click Create App.
You’ll be redirected to the settings page for the new app. Click on Incoming Webhooks on the left navigation bar.
Enable webhooks by flipping the switch button next to the title Activate Incoming Webhooks.
A new section further down the page will be uncovered. Scroll down and click the Add New Webhook to Workspace button. On the next page, select the channel you want the reports to be sent to and click Authorize.
You’ll be redirected back to the settings page for webhooks, and you’ll see a new webhook listed in the table. Click on Copy to copy it to clipboard and make note of it for later use.
Then, open config.toml for editing:
sudo nano config.toml
Add the following lines:
/usr/share/vuls-data/config.toml
[slack]
hookURL = "your_hook_url"
channel = "#your_channel_name"
authUser = "your_username"
#notifyUsers = ["@username"]
Replace the your_hook_URL with the webhook URL you noted earlier, your_username with the username of the user that created the web hook, and your_channel_name with the name of the desired channel. Save and close the file.
To test the integration, you can generate a report by running vuls report, like this:
sudo vuls report -to-slack
Vuls will take a few moments to run and exit successfully. If it shows an error, check what you’ve entered against the preceding lines.
You can check the Slack app and confirm that Vuls has successfully sent the report.
Now that you’ve configured reporting, you’ll set up scheduled scans. cron is a time-based job scheduler found on every Ubuntu machine. It is configured via the crontab file that defines in precise syntax when a command should run. To help ease the editing, you’ll use the crontab utility, which opens the current crontab file in an editor.
Open the current crontab file by running the following command:
crontab -e
When prompted, select your preferred text editor from the list.
Add the following line to the end of the file:
0 0 * * * vuls scan -config=/usr/share/vuls-data/config.toml; vuls report -config=/usr/share/vuls-data/config.toml > /dev/null 2>&1
The line above instructs cron to run vuls scan and vuls report with the given configuration every day at noon (denoted by 0 0 * * * in cron syntax).
Save and close the file.
In this step, you have connected Vuls to your Slack workspace and configured cron to run a Vuls scan and report every day at noon.
Conclusion
You have now successfully set up Vuls with automated scanning and reporting on an Ubuntu 18.04 server. For more reporting options, as well as troubleshooting, visit the Vuls documentation.
With Vuls, vulnerability assessment becomes more seamless for production environments. As an alternative to setting up cron, it is also possible to use Vuls in a continuous deployment workflow, as its scans are lightweight and you can run them as needed. You could also consider implementing a firewall with Vuls to restrict access and reduce the need for root access.
#ubuntu #security #go
