How To Install MySQL on Ubuntu 18.04

How To Install MySQL on Ubuntu 18.04

This tutorial will explain how to install MySQL version 5.7 on an Ubuntu 18.04 server.

This tutorial will explain how to install MySQL version 5.7 on an Ubuntu 18.04 server.

Introduction

MySQL is an open-source database management system, commonly installed as part of the popular LAMP (Linux, Apache, MySQL, PHP/Python/Perl) stack. It uses a relational database and SQL (Structured Query Language) to manage its data.

The short version of the installation is simple: update your package index, install the mysql-server package, and then run the included security script.

    sudo apt update
    sudo apt install mysql-server
    sudo mysql_secure_installation

Prerequisites

To follow this tutorial, you will need:

  • One Ubuntu 18.04 server set up by following this initial server setup guide, including a non-root user with sudo privileges and a firewall.
Step 1 — Installing MySQL

On Ubuntu 18.04, only the latest version of MySQL is included in the APT package repository by default. At the time of writing, that's MySQL 5.7

To install it, update the package index on your server with apt:

sudo apt update

Then install the default package:

sudo apt install mysql-server

This will install MySQL, but will not prompt you to set a password or make any other configuration changes. Because this leaves your installation of MySQL insecure, we will address this next.

Step 2 — Configuring MySQL

For fresh installations, you'll want to run the included security script. This changes some of the less secure default options for things like remote root logins and sample users. On older versions of MySQL, you needed to initialize the data directory manually as well, but this is done automatically now.

Run the security script:

sudo mysql_secure_installation

This will take you through a series of prompts where you can make some changes to your MySQL installation’s security options. The first prompt will ask whether you’d like to set up the Validate Password Plugin, which can be used to test the strength of your MySQL password. Regardless of your choice, the next prompt will be to set a password for the MySQL root user. Enter and then confirm a secure password of your choice.

From there, you can press Y and then ENTER to accept the defaults for all the subsequent questions. This will remove some anonymous users and the test database, disable remote root logins, and load these new rules so that MySQL immediately respects the changes you have made.

To initialize the MySQL data directory, you would use mysql_install_db for versions before 5.7.6, and mysqld --initialize for 5.7.6 and later. However, if you installed MySQL from the Debian distribution, as described in Step 1, the data directory was initialized automatically; you don't have to do anything. If you try running the command anyway, you'll see the following error:

Output

mysqld: Can't create directory '/var/lib/mysql/' (Errcode: 17 - File exists)
. . .
2018-04-23T13:48:00.572066Z 0 [ERROR] Aborting

Note that even though you’ve set a password for the root MySQL user, this user is not configured to authenticate with a password when connecting to the MySQL shell. If you’d like, you can adjust this setting by following Step 3.

Step 3 — (Optional) Adjusting User Authentication and Privileges

In Ubuntu systems running MySQL 5.7 (and later versions), the root MySQL user is set to authenticate using the auth_socket plugin by default rather than with a password. This allows for some greater security and usability in many cases, but it can also complicate things when you need to allow an external program (e.g., phpMyAdmin) to access the user.

In order to use a password to connect to MySQL as root, you will need to switch its authentication method from auth_socket to mysql_native_password. To do this, open up the MySQL prompt from your terminal:

sudo mysql

Next, check which authentication method each of your MySQL user accounts use with the following command:

Output

+------------------+-------------------------------------------+-----------------------+-----------+
| user             | authentication_string                     | plugin                | host      |
+------------------+-------------------------------------------+-----------------------+-----------+
| root             |                                           | auth_socket           | localhost |
| mysql.session    | *THISISNOTAVALIDPASSWORDTHATCANBEUSEDHERE | mysql_native_password | localhost |
| mysql.sys        | *THISISNOTAVALIDPASSWORDTHATCANBEUSEDHERE | mysql_native_password | localhost |
| debian-sys-maint | *CC744277A401A7D25BE1CA89AFF17BF607F876FF | mysql_native_password | localhost |
+------------------+-------------------------------------------+-----------------------+-----------+
4 rows in set (0.00 sec)

In this example, you can see that the root user does in fact authenticate using the auth_socket plugin. To configure the root account to authenticate with a password, run the following ALTER USER command. Be sure to change password to a strong password of your choosing, and note that this command will change the root password you set in Step 2:

ALTER USER 'root'@'localhost' IDENTIFIED WITH mysql_native_password BY 'password';

Then, run FLUSH PRIVILEGES which tells the server to reload the grant tables and put your new changes into effect:

FLUSH PRIVILEGES;

Check the authentication methods employed by each of your users again to confirm that root no longer authenticates using the auth_socket plugin:

SELECT user,authentication_string,plugin,host FROM mysql.user;

Output

+------------------+-------------------------------------------+-----------------------+-----------+
| user             | authentication_string                     | plugin                | host      |
+------------------+-------------------------------------------+-----------------------+-----------+
| root             | *3636DACC8616D997782ADD0839F92C1571D6D78F | mysql_native_password | localhost |
| mysql.session    | *THISISNOTAVALIDPASSWORDTHATCANBEUSEDHERE | mysql_native_password | localhost |
| mysql.sys        | *THISISNOTAVALIDPASSWORDTHATCANBEUSEDHERE | mysql_native_password | localhost |
| debian-sys-maint | *CC744277A401A7D25BE1CA89AFF17BF607F876FF | mysql_native_password | localhost |
+------------------+-------------------------------------------+-----------------------+-----------+
4 rows in set (0.00 sec)

You can see in this example output that the root MySQL user now authenticates using a password. Once you confirm this on your own server, you can exit the MySQL shell:

exit

Alternatively, some may find that it better suits their workflow to connect to MySQL with a dedicated user. To create such a user, open up the MySQL shell once again:

sudo mysql

Note: If you have password authentication enabled for root, as described in the preceding paragraphs, you will need to use a different command to access the MySQL shell. The following will run your MySQL client with regular user privileges, and you will only gain administrator privileges within the database by authenticating:

mysql -u root -p

From there, create a new user and give it a strong password:

CREATE USER 'sammy'@'localhost' IDENTIFIED BY 'password';

Then, grant your new user the appropriate privileges. For example, you could grant the user privileges to all tables within the database, as well as the power to add, change, and remove user privileges, with this command:

GRANT ALL PRIVILEGES ON *.* TO 'sammy'@'localhost' WITH GRANT OPTION;

Note that, at this point, you do not need to run the FLUSH PRIVILEGES command again. This command is only needed when you modify the grant tables using statements like INSERT, UPDATE, or DELETE. Because you created a new user, instead of modifying an existing one, FLUSH PRIVILEGES is unnecessary here.

Following this, exit the MySQL shell:

exit

Finally, let's test the MySQL installation.

Step 4 — Testing MySQL

Regardless of how you installed it, MySQL should have started running automatically. To test this, check its status.

systemctl status mysql.service

You'll see output similar to the following:

Output

● mysql.service - MySQL Community Server
   Loaded: loaded (/lib/systemd/system/mysql.service; enabled; vendor preset: en
   Active: active (running) since Wed 2018-04-23 21:21:25 UTC; 30min ago
 Main PID: 3754 (mysqld)
    Tasks: 28
   Memory: 142.3M
      CPU: 1.994s
   CGroup: /system.slice/mysql.service
           └─3754 /usr/sbin/mysqld

If MySQL isn't running, you can start it with sudo systemctl start mysql.

For an additional check, you can try connecting to the database using the mysqladmin tool, which is a client that lets you run administrative commands. For example, this command says to connect to MySQL as root (-u root), prompt for a password (-p), and return the version.

sudo mysqladmin -p -u root version

You should see output similar to this:

Output

mysqladmin  Ver 8.42 Distrib 5.7.21, for Linux on x86_64
Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Server version      5.7.21-1ubuntu1
Protocol version    10
Connection      Localhost via UNIX socket
UNIX socket     /var/run/mysqld/mysqld.sock
Uptime:         30 min 54 sec

Threads: 1  Questions: 12  Slow queries: 0  Opens: 115  Flush tables: 1  Open tables: 34  Queries per second avg: 0.006

This means MySQL is up and running.

Conclusion

You now have a basic MySQL setup installed on your server.

How To Configure SSL/TLS for MySQL on Ubuntu 18.04

How To Configure SSL/TLS for MySQL on Ubuntu 18.04

In this guide, we will demonstrate how to configure MySQL on Ubuntu 18.04 to accept remote connections with SSL/TLS encryption.

Introduction

MySQL is the most popular open-source relational database management system in the world. While modern package managers have reduced some of the friction to getting MySQL up and running, there is still some further configuration that should be performed after you install it. One of the most important aspects to spend some extra time on is security.

By default, MySQL is configured to only accept local connections, or connections that originate from the same machine where MySQL is installed. If you need to access your MySQL database from a remote location, it's important that you do so securely. In this guide, we will demonstrate how to configure MySQL on Ubuntu 18.04 to accept remote connections with SSL/TLS encryption.

Prerequisites

To complete this guide, you will need:

  • Two Ubuntu 18.04 servers. We will use one of these servers as the MySQL server while we'll use the other as the client machine. Create a non-root user with sudo privileges and enable a firewall with ufw on each of these servers. Follow our Ubuntu 18.04 initial server setup guide to get both servers into the appropriate initial state.
  • On one of the machines, install and configure the MySQL server. Follow Steps 1 through 3 of our MySQL installation guide for Ubuntu 18.04 to do this. As you follow this guide, be sure to configure your root MySQL user to authenticate with a password, as described in Step 3 of the guide, as this is necessary to connect to MySQL using TCP rather than the local Unix socket.

Please note that throughout this guide, the server on which you installed MySQL will be referred to as the MySQL server and any commands that should be run on this machine will be shown with a blue background, like this:

Similarly, this guide will refer to the other server as the MySQL client and any commands that must be run on that machine will be shown with a red background:

Please keep these in mind as you follow along with this tutorial so as to avoid any confusion.

Step 1 — Checking MySQL's Current SSL/TLS Status

Before you make any configuration changes, you can check the current SSL/TLS status on the MySQL server instance.

Use the following command to begin a MySQL session as the root MySQL user. This command includes the -p option, which instructs mysql to prompt you for a password in order to log in. It also includes the -h option which is used to specify the host to connect to. In this case it points it to 127.0.0.1, the IPv4 loopback interface also known as localhost. This will force the client to connect with TCP instead of using the local socket file. MySQL attempts to make connections through a Unix socket file by default. This is generally faster and more secure, since these connections can only be made locally and don't have to go through all the checks and routing operations that TCP connections must perform. Connecting with TCP, however, allows us to check the SSL status of the connection:

mysql -u root -p -h 127.0.0.1

You will be prompted for the MySQL root password that you chose when you installed and configured MySQL. After entering it you'll be dropped into an interactive MySQL session.

Show the state of the SSL/TLS variables issuing the following command:

SHOW VARIABLES LIKE '%ssl%';
Output
+---------------+----------+
| Variable_name | Value    |
+---------------+----------+
| have_openssl  | DISABLED |
| have_ssl      | DISABLED |
| ssl_ca        |          |
| ssl_capath    |          |
| ssl_cert      |          |
| ssl_cipher    |          |
| ssl_crl       |          |
| ssl_crlpath   |          |
| ssl_key       |          |
+---------------+----------+
9 rows in set (0.01 sec)

The have_openssl and have_ssl variables are both marked as DISABLED. This means that SSL functionality has been compiled into the server, but that it is not yet enabled.

Check the status of your current connection to confirm this:

\s
Output

mysql Ver 14.14 Distrib 5.7.26, for Linux (x86_64) using EditLine wrapper

Connection id: 9
Current database:
Current user: [email protected]
SSL: Not in use
Current pager: stdout
Using outfile: ''
Using delimiter: ;
Server version: 5.7.26-0ubuntu0.18.04.1 (Ubuntu)
Protocol version: 10
Connection: 127.0.0.1 via TCP/IP
Server characterset: latin1
Db characterset: latin1
Client characterset: utf8
Conn. characterset: utf8
TCP port: 3306
Uptime: 40 min 11 sec

Threads: 1 Questions: 33 Slow queries: 0 Opens: 113 Flush tables: 1 Open tables: 106 Queries per second avg: 0.013

As the above output indicates, SSL is not currently in use for this connection, even though you're connected over TCP.

Close the current MySQL session when you are finished:

exit

Now that you've confirmed your MySQL server isn't using SSL, you can move on to the next step where you will begin the process of enabling SSL by generating some certificates and keys. These will allow your server and client to communicate with one another securely.

Step 2 — Generating SSL/TLS Certificates and Keys

To enable SSL connections to MySQL, you first need to generate the appropriate certificate and key files. MySQL versions 5.7 and above provide a utility called mysql_ssl_rsa_setup that helps simplify this process. The version of MySQL you installed by following the prerequisite MySQL tutorial includes this utility, so we will use it here to generate the necessary files.

The MySQL process must be able to read the generated files, so use the --uid option to declare mysql as the system user that should own the generated files:

sudo mysql_ssl_rsa_setup --uid=mysql

This will produce output that looks similar to the following:

Output
Generating a 2048 bit RSA private key
.+++
..........+++
writing new private key to 'ca-key.pem' Generating a 2048 bit RSA private key
........................................+++
............+++
writing new private key to 'server-key.pem' Generating a 2048 bit RSA private key
.................................+++
............................................................+++
writing new private key to 'client-key.pem'

These new files will be stored in MySQL's data directory, located by default at /var/lib/mysql. Check the generated files by typing:

sudo find /var/lib/mysql -name '*.pem' -ls
Output
258930 4 -rw-r--r-- 1 mysql mysql 1107 May 3 16:43 /var/lib/mysql/client-cert.pem
258919 4 -rw-r--r-- 1 mysql mysql 451 May 3 16:43 /var/lib/mysql/public_key.pem
258925 4 -rw------- 1 mysql mysql 1675 May 3 16:43 /var/lib/mysql/server-key.pem
258927 4 -rw-r--r-- 1 mysql mysql 1107 May 3 16:43 /var/lib/mysql/server-cert.pem
258922 4 -rw------- 1 mysql mysql 1675 May 3 16:43 /var/lib/mysql/ca-key.pem
258928 4 -rw------- 1 mysql mysql 1675 May 3 16:43 /var/lib/mysql/client-key.pem
258924 4 -rw-r--r-- 1 mysql mysql 1107 May 3 16:43 /var/lib/mysql/ca.pem
258918 4 -rw------- 1 mysql mysql 1679 May 3 16:43 /var/lib/mysql/private_key.pem

These files are the key and certificate pairs for the certificate authority (starting with "ca"), the MySQL server process (starting with "server"), and for MySQL clients (starting with "client"). Additionally, the private_key.pem and public_key.pem files are used by MySQL to securely transfer passwords when not using SSL.

Now that you have the necessary certificate and key files, continue on to enable the use of SSL on your MySQL instance.

Step 3 — Enabling SSL Connections on the MySQL Server

Modern versions of MySQL look for the appropriate certificate files within the MySQL data directory whenever the server starts. Because of this, you won't need to modify MySQL’s configuration to enable SSL.

Instead, enable SSL by restarting the MySQL service:

sudo systemctl restart mysql

After restarting, open up a new MySQL session using the same command as before. The MySQL client will automatically attempt to connect using SSL if it is supported by the server:

mysql -u root -p -h 127.0.0.1

Let's take another look at the same information we requested last time. Check the values of the SSL-related variables:

SHOW VARIABLES LIKE '%ssl%';
Output
+---------------+-----------------+
| Variable_name | Value |
+---------------+-----------------+
| have_openssl | YES |
| have_ssl | YES |
| ssl_ca | ca.pem |
| ssl_capath | |
| ssl_cert | server-cert.pem |
| ssl_cipher | |
| ssl_crl | |
| ssl_crlpath | |
| ssl_key | server-key.pem |
+---------------+-----------------+
9 rows in set (0.00 sec)

The have_openssl and have_ssl variables now read YES instead of DISABLED. Furthermore, the ssl_ca, ssl_cert, and ssl_key variables have been populated with the names of the respective files that we just generated.

Next, check the connection details again:

\s
Output


. . .
SSL: Cipher in use is DHE-RSA-AES256-SHA
. . .
Connection: 127.0.0.1 via TCP/IP
. . .

This time, the specific SSL cipher is displayed, indicating that SSL is being used to secure the connection.

Exit back out to the shell:

exit

Your server is now capable of using encryption, but some additional configuration is required to allow remote access and mandate the use of secure connections.

Step 4 — Configuring Secure Connections for Remote Clients

Now that you've enabled SSL on the MySQL server, you can begin configuring secure remote access. To do this, you'll configure your MySQL server to require that any remote connections be made over SSL, bind MySQL to listen on a public interface, and adjust your system's firewall rules to allow external connections

Currently, the MySQL server is configured to accept SSL connections from clients. However, it will still allow unencrypted connections if requested by the client. We can change this by turning on the require_secure_transport option. This requires all connections to be made either with SSL or with a local Unix socket. Since Unix sockets are only accessible from within the server itself, the only connection option available to remote users will be with SSL.

To enable this setting, open the MySQL configuration file in your preferred text editor. Here, we'll use nano:

sudo nano /etc/mysql/my.cnf

Inside there will be two !includedir directives which are used to source additional configuration files. You must add your own configuration beneath these lines so that it overrides any conflicting settings found in these additional configuration files.

Start by creating a [mysqld] section to target the MySQL server process. Under that section header, set require_secure_transport to ON, which will force MySQL to only allow secure connections:

/etc/mysql/my.cnf

. . .

!includedir /etc/mysql/conf.d/
!includedir /etc/mysql/mysql.conf.d/

[mysqld]

Require clients to connect either using SSL or through a local socket file

require_secure_transport = ON

By default, MySQL is configured to only listen for connections that originate from 127.0.0.1, the loopback IP address that represents localhost. This means that MySQL is configured to only listen for connections that originate from the machine on which the MySQL server is installed.

In order to allow MySQL to listen for external connections, you must configure it to listen for connections on an external IP address. To do this, you can add the bind-address setting and point it to 0.0.0.0, a wildcard IP address that represents all IP addresses. Essentially, this will force MySQL to listen for connections on every interface:

/etc/mysql/my.cnf

. . .

!includedir /etc/mysql/conf.d/
!includedir /etc/mysql/mysql.conf.d/

[mysqld]

Require clients to connect either using SSL or through a local socket file

require_secure_transport = ON
bind-address = 0.0.0.0

Note: You could alternatively set bind-address to your MySQL server's public IP address. However, you would need to remember to update your my.cnf file if you ever migrate your database to another machine.

After adding these lines, save and close the file. If you used nano to edit the file, you can do so by pressing CTRL+X, Y, then ENTER.

Next, restart MySQL to apply the new settings:

sudo systemctl restart mysql

Verify that MySQL is listening on 0.0.0.0 instead of 127.0.0.1 by typing:

sudo netstat -plunt

The output of this command will look like this:

Output

Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:3306 0.0.0.0:* LISTEN 13317/mysqld
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 1293/sshd
tcp6 0 0 :::22 :::* LISTEN 1293/sshd

The 0.0.0.0 highlighted in the above output indicates that MySQL is listening for connections on all available interfaces.

Next, allow MySQL connections through your server's firewall. Add an exception to your ufw rules by typing:

sudo ufw allow mysql
Output
Rule added
Rule added (v6)

With that, remote connection attempts are now able to reach your MySQL server. However, you don't currently have any users configured that can connect from a remote machine. We'll create and configure a MySQL user that can connect from your client machine in the next step.

Step 5 — Creating a Dedicated MySQL User

At this point, your MySQL server will reject any attempt to connect from a remote client machine. This is because the existing MySQL users are all only configured to connect locally from the MySQL server. To resolve this, you will create a dedicated user that will only be able to connect from your client machine.

To create such a user, log back into MySQL as the root user:

mysql -u root -p

From the prompt, create a new remote user with the CREATE USER command. You can name this user whatever you'd like, but in this guide we name it mysql_user. Be sure to specify your client machine's IP address in the host portion of the user specification to restrict connections to that machine and to replace password with a secure password of your choosing. Also, for some redundancy in case the require_secure_transport option is turned off in the future, specify that this user requires SSL by including the REQUIRE SSL clause, as shown here:

CREATE USER 'mysql_user'@'your_mysql_client_IP' IDENTIFIED BY 'password' REQUIRE SSL;

Next, grant the new user permissions on whichever databases or tables that they should have access to. To demonstrate, create an example database:

CREATE DATABASE example;

Then give your new user access to this database and all of its tables:

GRANT ALL ON example.* TO 'mysql_user'@'your_mysql_client_IP';

Next, flush the privileges to apply those settings immediately:

FLUSH PRIVILEGES;

Then exit back out to the shell when you are done:

exit

Your MySQL server is now set up to allow connections from your remote user. To test that you can connect to MySQL successfully, you will need to install the mysql-client package on the MySQL client.

Log in to your client machine with ssh

ssh [email protected]_mysql_client_ip

Then update the client machine's package index:

sudo apt update

And install mysql-client with the following command:

sudo apt install mysql-client

When prompted, confirm the installation by pressing ENTER.

Once APT finishes installing the package, run the following command to test whether you can connect to the server successfully. This command includes the -u user option to specify mysql_user and the -h option to specify the MySQL server's IP address:

mysql -u mysql_user -p -h your_mysql_server_IP

After submitting the password, you will be logged in to the remote server. Use \s to check the server's status and confirm that your connection is secure:

\s
Output


. . .
SSL: Cipher in use is DHE-RSA-AES256-SHA
. . .
Connection: your_mysql_server_IP via TCP/IP
. . .

Exit back out to the shell:

exit

You've confirmed that you're able to connect to MySQL over SSL. However, you've not yet confirmed that the MySQL server is rejecting insecure connections. To test this, try connecting once more, but this time append --ssl-mode=disabled to the login command. This will instruct mysql-client to attempt an unencrypted connection:

mysql -u mysql_user -p -h mysql_server_IP --ssl-mode=disabled

After entering your password when prompted, your connection will be refused:

Output
ERROR 1045 (28000): Access denied for user 'mysql_user'@'mysql_server_IP' (using password: YES)

This shows that SSL connections are permitted while unencrypted connections are refused.

At this point, your MySQL server has been configured to accept secure remote connections. You can stop here if this satisfies your security requirements, but there are some additional pieces that you can put into place to enhance security and trust between your two servers.

Step 6 — (Optional) Configuring Validation for MySQL Connections

Currently, your MySQL server is configured with an SSL certificate signed by a locally generated certificate authority (CA). The server's certificate and key pair are enough to provide encryption for incoming connections.

However, you aren't yet fully leveraging the trust relationship that a certificate authority can provide. By distributing the CA certificate to clients — as well as the client certificate and key — both parties can provide proof that their certificates were signed by a mutually trusted certificate authority. This can help prevent spoofed connections from malicious servers.

In order to implement this extra, optional safeguard, we will transfer the appropriate SSL files to the client machine, create a client configuration file, and alter the remote MySQL user to require a trusted certificate.

Note: The process for transferring the CA certificate, client certificate, and client key to the MySQL client outlined in the following paragraphs involves displaying each file's contents with cat, copying those contents to your clipboard, and pasting them in to a new file on the client machine. While it is possible to copy these files directly with a program like scp or sftp, this also requires you to set up SSH keys for both servers so as to allow them to communicate over SSH.

Our goal here is to keep the number of potential avenues for connecting to your MySQL server down to a minimum. While this process is slightly more laborious than directly transferring the files, it is equally secure and doesn't require you to open an SSH connection between the two machines.


Begin by making a directory on the MySQL client in the home directory of your non-root user. Call this directory client-ssl:

mkdir ~/client-ssl

Because the certificate key is sensitive, lock down access to this directory so that only the current user can access it:

chmod 700 ~/client-ssl

On the MySQL server, display the contents of the CA certificate by typing:

sudo cat /var/lib/mysql/ca.pem
Output
-----BEGIN CERTIFICATE-----

. . .

-----END CERTIFICATE-----

Copy the entire output, including the BEGIN CERTIFICATE and END CERTIFICATE lines, to your clipboard.

On the MySQL client, create a file with the same name inside the new directory:

nano ~/client-ssl/ca.pem

Inside, paste the copied certificate contents from your clipboard. Save and close the file when you are finished.

Next, display the client certificate on the MySQL server:

sudo cat /var/lib/mysql/client-cert.pem
Output
-----BEGIN CERTIFICATE-----

. . .

-----END CERTIFICATE-----

Copy the file contents to your clipboard. Again, remember to include the first and last line.

Open a file with the same name on the MySQL client within the client-ssl directory:

nano ~/client-ssl/client-cert.pem

Paste the contents from your clipboard. Save and close the file.

Finally, display the contents of the client key file on the MySQL server:

sudo cat /var/lib/mysql/client-key.pem
Output
-----BEGIN RSA PRIVATE KEY-----

. . .

-----END RSA PRIVATE KEY-----

Copy the displayed contents, including the first and last line, to your clipboard.

On the MySQL client, open a file with the same name in the client-ssl directory:

nano ~/client-ssl/client-key.pem

Paste the contents from your clipboard. Save and close the file.

The client machine now has all of the credentials required to access the MySQL server. However, the MySQL server is still not set up to require trusted certificates for client connections.

To change this, log in to the MySQL root account again on the MySQL server:

mysql -u root -p

From here, change the security requirements for your remote user. Instead of the REQUIRE SSL clause, apply the REQUIRE X509 clause. This implies all of the security provided by the REQUIRE SSL clause, but additionally requires the connecting client to present a certificate signed by a certificate authority that the MySQL server trusts.

To adjust the user requirements, use the ALTER USER command:

ALTER USER 'mysql_user'@'mysql_client_IP' REQUIRE X509;

Then flush the changes to ensure that they are applied immediately:

FLUSH PRIVILEGES;

Exit back out to the shell when you are finished:

exit

Following that, check whether you can validate both parties when you connect.

On the MySQL client, first try to connect without providing the client certificates:

mysql -u mysql_user -p -h mysql_server_IP
Output
ERROR 1045 (28000): Access denied for user 'mysql_user'@'mysql_client_IP' (using password: YES)

As expected, the server rejects the connection when no client certificate is presented.

Now, connect while using the --ssl-ca, --ssl-cert, and --ssl-key options to point to the relevant files within the ~/client-ssl directory:

mysql -u mysql_user -p -h mysql_server_IP --ssl-ca=~/client-ssl/ca.pem --ssl-cert=~/client-ssl/client-cert.pem --ssl-key=~/client-ssl/client-key.pem

You've provided the client with the appropriate certificates and keys, so this attempt will be successful:


Log back out to regain access to your shell session:

exit

Now that you've confirmed access to the server, let's implement a small usability improvement in order to avoid having to specify the certificate files each time you connect.

Inside your home directory on the MySQL client machine, create a hidden configuration file called ~/.my.cnf:

nano ~/.my.cnf

At the top of the file, create a section called [client]. Underneath, add the ssl-ca, ssl-cert, and ssl-key options and point them to the respective files you copied over from the server. It will look like this:

~/.my.cnf

[client]
ssl-ca = ~/client-ssl/ca.pem
ssl-cert = ~/client-ssl/client-cert.pem
ssl-key = ~/client-ssl/client-key.pem

The ssl-ca option tells the client to verify that the certificate presented by the MySQL server is signed by the certificate authority you pointed to. This allows the client to trust that it is connecting to a trusted MySQL server. Likewise, the ssl-cert and ssl-key options point to the files needed to prove to the MySQL server that it too has a certificate that has been signed by the same certificate authority. You'll need this if you want the MySQL server to verify that the client was trusted by the CA as well.

Save and close the file when you are finished.

Now, you can connect to the MySQL server without adding the --ssl-ca, --ssl-cert, and --ssl-key options on the command line:

mysql -u remote_user -p -h mysql_server_ip

Your client and server will now each be presenting certificates when negotiating the connection. Each party is configured to verify the remote certificate against the CA certificate it has locally.

Conclusion

Your MySQL server is now configured to require secure connections from remote clients. Additionally, if you followed the steps to validate connections using the certificate authority, some level of trust is established by both sides that the remote party is legitimate. 

Thanks for reading

If you liked this post, share it with all of your programming buddies!

Follow us on Facebook | Twitter

Learn More

The Complete SQL Bootcamp

The Ultimate MySQL Bootcamp: Go from SQL Beginner to Expert

The Complete Oracle SQL Certification Course

MySQL Databases With Python Tutorial

SQL vs NoSQL or MySQL vs MongoDB

MySQL Database Tutorial for Beginners to Advanced Part 1/2

An Introduction to Queries in MySQL

How To Troubleshoot MySQL Queries?

SQL with MySQL - Complete Tutorial for Beginners

Build a Basic CRUD App with PHP and MySQL

Originally published on https://www.digitalocean.com

How To Optimize MySQL with Query Cache on Ubuntu 18.04

How To Optimize MySQL with Query Cache on Ubuntu 18.04

In this tutorial, you will first configure MySQL without query cache and run queries to see how quickly they are executed. Then you’ll set up query cache and test your MySQL server with it enabled to show the difference in performance.

In this tutorial, you will first configure MySQL without query cache and run queries to see how quickly they are executed. Then you’ll set up query cache and test your MySQL server with it enabled to show the difference in performance.

Introduction

Query cache is a prominent MySQL feature that speeds up data retrieval from a database. It achieves this by storing MySQL SELECT statements together with the retrieved record set in memory, then if a client requests identical queries it can serve the data faster without executing commands again from the database.

Compared to data read from disk, cached data from RAM (Random Access Memory) has a shorter access time, which reduces latency and improves input/output (I/O) operations. As an example, for a WordPress site or an e-commerce portal with high read calls and infrequent data changes, query cache can drastically boost the performance of the database server and make it more scalable.

Note: Although query cache is deprecated as of MySQL 5.7.20, and removed in MySQL 8.0, it is still a powerful tool if you’re using supported versions of MySQL. However, if you are using newer versions of MySQL, you may adopt alternative third-party tools like ProxySQL to optimize performance on your MySQL database.

Prerequisites

Before you begin, you will need the following:

  • One Ubuntu 18.04 server configured with a firewall and a non-root user.
  • A MySQL server. Ensure you set a root password for the MySQL server.
Step 1 — Checking the Availability of Query Cache

Before you set up query cache, you’ll check whether your version of MySQL supports this feature. First, ssh into your Ubuntu 18.04 server:

ssh [email protected]_server_ip 

Then, run the following command to log in to the MySQL server as the root user:

sudo mysql -u root -p 

Enter your MySQL server root password when prompted and then press ENTER to continue.

Use the following command to check if query cache is supported:

show variables like 'have_query_cache'; 

You should get an output similar to the following:

+------------------+-------+
| Variable_name    | Value |
+------------------+-------+
| have_query_cache | YES   |
+------------------+-------+
1 row in set (0.01 sec)

You can see the value of have_query_cache is set to YES and this means query cache is supported. If you receive an output showing that your version does not support query cache, please see the note in the Introduction section for more information.

Now that you have checked and confirmed that your version of MySQL supports query cache, you will move on to examining the variables that control this feature on your database server.

Step 2 — Checking the Default Query Cache Variables

In MySQL, a number of variables control query cache. In this step, you’ll check the default values that ship with MySQL and understand what each variable controls.

You can examine these variables using the following command:

show variables like 'query_cache_%' ; 

You will see the variables listed in your output:

+------------------------------+----------+
| Variable_name                | Value    |
+------------------------------+----------+
| query_cache_limit            | 1048576  |
| query_cache_min_res_unit     | 4096     |
| query_cache_size             | 16777216 |
| query_cache_type             | OFF      |
| query_cache_wlock_invalidate | OFF      |
+------------------------------+----------+
5 rows in set (0.00 sec)

The query_cache_limit value determines the maximum size of individual query results that can be cached. The default value is 1,048,576 bytes and this is equivalent to 1MB.

MySQL does not handle cached data in one big chunk; instead it is handled in blocks. The minimum amount of memory allocated to each block is determined by the query_cache_min_res_unit variable. The default value is 4096 bytes or 4KB.

query_cache_size controls the total amount of memory allocated to the query cache. If the value is set to zero, it means query cache is disabled. In most cases, the default value may be set to 16,777,216 (around 16MB). Also, keep in mind that query_cache_size needs at least 40KB to allocate its structures. The value allocated here is aligned to the nearest 1024 byte block. This means the reported value may be slightly different from what you set.

MySQL determines the queries to cache by examining the query_cache_type variable. Setting this value to 0 or OFF prevents caching or retrieval of cached queries. You can also set it to 1 to enable caching for all queries except for ones beginning with the [SELECT SQL_NO_CACHE](https://dev.mysql.com/doc/refman/5.7/en/query-cache-in-select.html) statement. A value of 2 tells MySQL to only cache queries that begin with SELECT SQL_CACHE command.

The variable query_cache_wlock_invalidate controls whether MySQL should retrieve results from the cache if the table used on the query is locked. The default value is OFF.

Note: The query_cache_wlock_invalidate variable is deprecated as of MySQL version 5.7.20. As a result, you may not see this in your output depending on the MySQL version you’re using.

Having reviewed the system variables that control the MySQL query cache, you’ll now test how MySQL performs without first enabling the feature.

Step 3 — Testing Your MySQL Server Without Query Cache

The goal of this tutorial is to optimize your MySQL server by using the query cache feature. To see the difference in speed, you’re going to run queries and see their performance before and after implementing the feature.

In this step you’re going to create a sample database and insert some data to see how MySQL performs without query cache.

While still logged in to your MySQL server, create a database and name it sample_db by running the following command:

mysql> Create database sample_db;
Output
Query OK, 1 row affected (0.00 sec)

Then switch to the database:

mysql> Use sample_db;
Output
Database changed

Create a table with two fields (customer_id and customer_name) and name it customers`:

mysql> Create table customers (customer_id INT PRIMARY KEY, customer_name VARCHAR(50) NOT NULL) Engine = InnoDB;
Output
Query OK, 0 rows affected (0.01 sec)

Then, run the following commands to insert some sample data:

    Insert into customers(customer_id, customer_name) values ('1', 'JANE DOE');
    Insert into customers(customer_id, customer_name) values ('2', 'JANIE DOE');
    Insert into customers(customer_id, customer_name) values ('3', 'JOHN ROE');
    Insert into customers(customer_id, customer_name) values ('4', 'MARY ROE');
    Insert into customers(customer_id, customer_name) values ('5', 'RICHARD ROE');
    Insert into customers(customer_id, customer_name) values ('6', 'JOHNNY DOE');
    Insert into customers(customer_id, customer_name) values ('7', 'JOHN SMITH');
    Insert into customers(customer_id, customer_name) values ('8', 'JOE BLOGGS');
    Insert into customers(customer_id, customer_name) values ('9', 'JANE POE');
    Insert into customers(customer_id, customer_name) values ('10', 'MARK MOE');
Output
Query OK, 1 row affected (0.01 sec)
Query OK, 1 row affected (0.00 sec)
...

The next step is starting the MySQL profiler, which is an analysis service for monitoring the performance of MySQL queries. To turn the profile on for the current session, run the following command, setting it to 1, which is on:

mysql> SET profiling = 1;
Output
Query OK, 0 rows affected, 1 warning (0.00 sec)

Then, run the following query to retrieve all customers:

mysql> Select * from customers;

You’ll receive the following output:

Output
+-------------+---------------+
| customer_id | customer_name |
+-------------+---------------+
|           1 | JANE DOE      |
|           2 | JANIE DOE     |
|           3 | JOHN ROE      |
|           4 | MARY ROE      |
|           5 | RICHARD ROE   |
|           6 | JOHNNY DOE    |
|           7 | JOHN SMITH    |
|           8 | JOE BLOGGS    |
|           9 | JANE POE      |
|          10 | MARK MOE      |
+-------------+---------------+
10 rows in set (0.00 sec)

Then, run the SHOW PROFILES command to retrieve performance information about the SELECT query you just ran:

SHOW PROFILES; 

You will get output similar to the following:

Output
+----------+------------+-------------------------+
| Query_ID | Duration   | Query                   |
+----------+------------+-------------------------+
|        1 | 0.00044075 | Select * from customers |
+----------+------------+-------------------------+
1 row in set, 1 warning (0.00 sec)

The output shows the total time spent by MySQL when retrieving records from the database. You are going to compare this data in the next steps when query cache is enabled, so keep note of your Duration. You can ignore the warning within the output since this simply indicates that SHOW PROFILES command will be removed in a future MySQL release and replaced with Performance Schema.

Next, exit from the MySQL Command Line Interface.

quit; 

You have ran a query with MySQL before enabling query cache and noted down the Duration or time spent to retrieve records. Next, you will enable query cache and see if there is a performance boost when running the same query.

Step 4 — Setting Up Query Cache

In the previous step, you created sample data and ran a SELECT statement before you enabled query cache. In this step, you’ll enable query cache by editing the MySQL configuration file.

Use nano to edit the file:

sudo nano /etc/mysql/my.cnf 

Add the following information to the end of your file:

/etc/mysql/my.cnf

...
[mysqld]
query_cache_type=1
query_cache_size = 10M
query_cache_limit=256K

Here you’ve enabled query cache by setting the query_cache_type to 1. You’ve also set up the individual query limit size to 256K and instructed MySQL to allocate 10 megabytes to query cache by setting the value of query_cache_size to 10M.

Save and close the file by pressing CTRL + X, Y, then ENTER. Then, restart your MySQL server to implement the changes:

sudo systemctl restart mysql 

You have now enabled query cache.

Once you have configured query cache and restarted MySQL to apply the changes, you will go ahead and test the performance of MySQL with the feature enabled.

Step 5 — Testing Your MySQL Server with Query Cache Enabled

In this step, you’ll run the same query you ran in Step 3 one more time to check how query cache has optimized the performance of your MySQL server.

First, connect to your MySQL server as the root user:

sudo mysql -u root -p 

Enter your root password for the database server and hit ENTER to continue.

Now confirm your configuration set in the previous step to ensure you enabled query cache:

show variables like 'query_cache_%' ; 

You’ll see the following output:

Output
+------------------------------+----------+
| Variable_name                | Value    |
+------------------------------+----------+
| query_cache_limit            | 262144   |
| query_cache_min_res_unit     | 4096     |
| query_cache_size             | 10485760 |
| query_cache_type             | ON       |
| query_cache_wlock_invalidate | OFF      |
+------------------------------+----------+
5 rows in set (0.01 sec)

The variable query_cache_type is set to ON; this confirms that you enabled query cache with the parameters defined in the previous step.

Switch to the sample_db database that you created earlier.

Use sample_db; 

Start the MySQL profiler:

SET profiling = 1; 

Then, run the query to retrieve all customers at least two times in order to generate enough profiling information.

Remember, once you’ve run the first query, MySQL will create a cache of the results and therefore, you must run the query twice to trigger the cache:

    Select * from customers;
    Select * from customers;

Then, list the profiles information:

SHOW PROFILES; 

You’ll receive an output similar to the following:

Output
+----------+------------+-------------------------+
| Query_ID | Duration   | Query                   |
+----------+------------+-------------------------+
|        1 | 0.00049250 | Select * from customers |
|        2 | 0.00026000 | Select * from customers |
+----------+------------+-------------------------+
2 rows in set, 1 warning (0.00 sec)

As you can see the time taken to run the query has drastically reduced from 0.00044075 (without query cache in Step 3) to 0.00026000 (the second query) in this step.

You can see the optimization from enabling the query cache feature by profiling the first query in detail:

mysql> SHOW PROFILE FOR QUERY 1;
Output
+--------------------------------+----------+
| Status                         | Duration |
+--------------------------------+----------+
| starting                       | 0.000025 |
| Waiting for query cache lock   | 0.000004 |
| starting                       | 0.000003 |
| checking query cache for query | 0.000045 |
| checking permissions           | 0.000008 |
| Opening tables                 | 0.000014 |
| init                           | 0.000018 |
| System lock                    | 0.000008 |
| Waiting for query cache lock   | 0.000002 |
| System lock                    | 0.000018 |
| optimizing                     | 0.000003 |
| statistics                     | 0.000013 |
| preparing                      | 0.000010 |
| executing                      | 0.000003 |
| Sending data                   | 0.000048 |
| end                            | 0.000004 |
| query end                      | 0.000006 |
| closing tables                 | 0.000006 |
| freeing items                  | 0.000006 |
| Waiting for query cache lock   | 0.000003 |
| freeing items                  | 0.000213 |
| Waiting for query cache lock   | 0.000019 |
| freeing items                  | 0.000002 |
| storing result in query cache  | 0.000003 |
| cleaning up                    | 0.000012 |
+--------------------------------+----------+
25 rows in set, 1 warning (0.00 sec)

Run the following command to show profile information for the second query, which is cached:

mysql> SHOW PROFILE FOR QUERY 2;
Output
+--------------------------------+----------+
| Status                         | Duration |
+--------------------------------+----------+
| starting                       | 0.000024 |
| Waiting for query cache lock   | 0.000003 |
| starting                       | 0.000002 |
| checking query cache for query | 0.000006 |
| checking privileges on cached  | 0.000003 |
| checking permissions           | 0.000027 |
| sending cached result to clien | 0.000187 |
| cleaning up                    | 0.000008 |
+--------------------------------+----------+
8 rows in set, 1 warning (0.00 sec)

The outputs from the profiler show that MySQL took less time on the second query because it was able to retrieve data from the query cache instead of reading it from the disk. You can compare the two sets of output for each of the queries. If you look at the profile information on QUERY 2, the status of sending cached result to client shows that data was read from the cache and no tables were opened since the Opening tables status is missing.

With the MySQL query cache feature enabled on your server, you’ll now experience improved read speeds.

Conclusion

You have set up query cache to speed up your MySQL server on Ubuntu 18.04. Using features like MySQL’s query cache can enhance the speed of your website or web application. Caching reduces unnecessary execution for SQL statements and is a highly recommended and popular method for optimizing your database.

How To Implement Pagination in MySQL with PHP on Ubuntu 18.04

How To Implement Pagination in MySQL with PHP on Ubuntu 18.04

In this tutorial, you’ll build a PHP script to connect to your database and implement pagination to your script using the MySQL LIMIT clause.

Introduction

Pagination is the concept of constraining the number of returned rows in a recordset into separate, orderly pages to allow easy navigation between them, so when there is a large dataset you can configure your pagination to only return a specific number of rows on each page. For example, pagination can help to avoid overwhelming users when a web store contains thousands of products by reducing the number of items listed on a page, as it’s often unlikely a user will need to view every product. Another example is an application that shows records on a mobile device; enabling pagination in such a case would split records into multiple pages that can fit better on a screen.

Besides the visual benefits for end-users, pagination makes applications faster because it reduces the number of records that are returned at a time. This limits the data that needs to be transmitted between the client and the server, which helps preserve server resources such as RAM.

In this tutorial, you’ll build a PHP script to connect to your database and implement pagination to your script using the MySQL LIMIT clause.

Step 1 — Creating a Database User and a Test Database

In this tutorial you’ll create a PHP script that will connect to a MySQL database, fetch records, and display them in an HTML page within a table. You’ll test the PHP script in two different ways from your web browser. First, creating a script without any pagination code to see how the records are displayed. Second, adding page navigation code in the PHP file to understand how pagination works practically.

The PHP code requires a MySQL user for authentication purposes and a sample database to connect to. In this step you’ll create a non-root user for your MySQL database, a sample database, and a table to test the PHP script.

To begin log in to your server. Then log in to your MySQL server with the following command:

sudo mysql -u root -p

Enter the root password of your MySQL server and hit ENTER to continue. Then, you’ll see the MySQL prompt. To create a sample database, which we will call test_db in this tutorial, run the following command:

Create database test_db;

You will see the following output:

OutputQuery OK, 1 row affected (0.00 sec)

Then, create a test_user and grant the user all privileges to the test_db. Replace PASSWORD with a strong value:

GRANT ALL PRIVILEGES ON test_db.* TO 'test_user'@'localhost' IDENTIFIED BY 'PASSWORD';

OutputQuery OK, 1 row affected (0.00 sec)

Reload the MySQL privileges with:

FLUSH PRIVILEGES;

OutputQuery OK, 1 row affected (0.00 sec)

Next, switch to the test_db database to start working directly on the test_db database:

Use test_db;

OutputDatabase changed

Now create a products table. The table will hold your sample products—for this tutorial you’ll require only two columns for the data. The product_id column will serve as the primary key to uniquely identify each record. You’ll use the product_name field to differentiate each item by name:

Create table products (product_id BIGINT PRIMARY KEY, product_name VARCHAR(50) NOT NULL ) Engine = InnoDB;

OutputQuery OK, 0 rows affected (0.02 sec)

To add ten test products to the products table run the following SQL statements:

Insert into products(product_id, product_name) values ('1', 'WIRELESS MOUSE');
Insert into products(product_id, product_name) values ('2', 'BLUETOOTH SPEAKER');
Insert into products(product_id, product_name) values ('3', 'GAMING KEYBOARD');
Insert into products(product_id, product_name) values ('4', '320GB FAST SSD');
Insert into products(product_id, product_name) values ('5', '17 INCHES TFT');
Insert into products(product_id, product_name) values ('6', 'SPECIAL HEADPHONES');
Insert into products(product_id, product_name) values ('7', 'HD GRAPHIC CARD');
Insert into products(product_id, product_name) values ('8', '80MM THERMAL PRINTER');
Insert into products(product_id, product_name) values ('9', 'HDMI TO VGA CONVERTER');
Insert into products(product_id, product_name) values ('10', 'FINGERPRINT SCANNER');

You’ll see this output:

OutputQuery OK, 1 row affected (0.02 sec)

Verify that the products were inserted to the table by running:

select * from products;

You’ll see the products in your output within the two columns:

Output+------------+-----------------------+
| product_id | product_name          |
+------------+-----------------------+
|          1 | WIRELESS MOUSE        |
|          2 | BLUETOOTH SPEAKER     |
|          3 | GAMING KEYBOARD       |
|          4 | 320GB FAST SSD        |
|          5 | 17 INCHES TFT         |
|          6 | SPECIAL HEADPHONES    |
|          7 | HD GRAPHIC CARD       |
|          8 | 80MM THERMAL PRINTER  |
|          9 | HDMI TO VGA CONVERTER |
|         10 | FINGERPRINT SCANNER   |
+------------+-----------------------+
10 rows in set (0.00 sec)

Exit MySQL:

quit;

With the sample database, table, and test data in place, you can now create a PHP script to display data on a web page.

Step 2 — Displaying MySQL Records Without Pagination

Now you’ll create a PHP script that connects to the MySQL database that you created in the previous step and list the products in a web browser. In this step, your PHP code will run without any form of pagination to demonstrate how non-split records show on a single page. Although you only have ten records for testing purposes in this tutorial, seeing the records without pagination will demonstrate why segmenting data will ultimately create a better user experience and put less burden on the server.

Create the PHP script file in the document root of your website with the following command:

sudo nano /var/www/html/pagination_test.php

Then add the following content to the file. Remember to replace PASSWORD with the correct value of the password that you assigned to the test_user in the previous step:

<?php

try {

    $pdo = new PDO("mysql:host=localhost;dbname=test_db", "test_user", "PASSWORD");
    $pdo->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
    $pdo->setAttribute(PDO::ATTR_EMULATE_PREPARES,false);

    $sql="select * from products";

    $stmt = $pdo->prepare($sql);

    $stmt->execute();

    echo "<table border='1' align='center'>";

    while ( ($row = $stmt->fetch(PDO::FETCH_ASSOC) ) !== false) {
        echo "<tr>";

        echo "<td>".$row['product_id']."</td>";

        echo "<td>".$row['product_name']."</td>";

        echo "</tr>";

    }

    echo "</table>";

}

  catch(PDOException $e)

{
    echo  $e->getMessage();
}

?>

Save the file by pressing CTRL+X, Y, and ENTER.

In this script you’re connecting to the MySQL database using the PDO (PHP Data Object) library with the database credentials that you created in Step 1.

PDO is a light-weight interface for connecting to databases. The data access layer is more portable and can work on different databases with just minor code rewrites. PDO has greater security since it supports prepared statements—a feature for making queries run faster in a secure way.

Then, you instruct the PDO API to execute the select * from products statement and list products in an HTML table without pagination. The line $pdo->setAttribute(PDO::ATTR_EMULATE_PREPARES,false); ensures that the data types are returned as they appear in the database. This means that PDO will return the product_id as an integer and the product_name as a string. $pdo->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION); instructs PDO to throw an exception if an error is encountered. For easier debugging you’re catching the error inside the PHP try{}...catch{} block.

To execute the /var/www/html/pagination_test.php PHP script file that you’ve created, visit the following URL replacing your-server-IP with the public IP address of your server:

http://your-server-IP/pagination_test.php

You’ll see a page with a table of your products.

Your PHP script is working as expected; listing all products on one page. If you had thousands of products, this would result in a long loop as the products are fetched from the database and rendered on the PHP page.

To overcome this limitation, you will modify the PHP script and include the MySQL LIMIT clause and some navigation links at the bottom of the table to add pagination functionality.

Step 3 — Implementing Pagination with PHP

In this step your goal is to split the test data into multiple and manageable pages. This will not only enhance readability but also use the resources of the server more efficiently. You will modify the PHP script that you created in the previous step to accommodate pagination.

To do this, you’ll be implementing the MySQL LIMIT clause. Before adding this to the script, let’s see an example of the MySQL LIMIT syntax:

Select [column1, column2, column n...] from [table name] LIMIT offset, records;

The LIMIT clause takes two arguments as shown toward the end of this statement. The offset value is the number of records to skip before the first row. records sets the maximum number of records to display per page.

To test pagination, you’ll display three records per page. To get the total number of pages, you must divide the total records from your table with the rows that you want to display per page. You then round the resulting value to the nearest integer using PHP Ceil function as shown in the following PHP code snippet example:

$total_pages=ceil($total_records/$per_page);

Following is the modified version of the PHP script with the full pagination code. To include the pagination and navigation codes, open the /var/www/html/pagination_test.php file:

sudo nano /var/www/html/pagination_test.php

Then, add the following highlighted code to your file:

<?php

try {

    $pdo = new PDO("mysql:host=localhost;dbname=test_db", "test_user", "PASSWORD");
    $pdo->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
    $pdo->setAttribute(PDO::ATTR_EMULATE_PREPARES,false);

    /* Begin Paging Info */

    $page=1;

    if (isset($_GET['page'])) {
        $page=filter_var($_GET['page'], FILTER_SANITIZE_NUMBER_INT);
    }

    $per_page=3;

    $sqlcount="select count(*) as total_records from products";
    $stmt = $pdo->prepare($sqlcount);
    $stmt->execute();
    $row = $stmt->fetch();
    $total_records= $row['total_records'];

    $total_pages=ceil($total_records/$per_page);

    $offset=($page-1)*$per_page;

    /* End Paging Info */

    $sql="select * from products limit $offset,$per_page";

    $stmt = $pdo->prepare($sql);

    $stmt->execute();

    echo "<table border='1' align='center'>";

    while ( ($row = $stmt->fetch(PDO::FETCH_ASSOC) ) !== false) {
        echo "<tr>";

        echo "<td>".$row['product_id']."</td>";

        echo "<td>".$row['product_name']."</td>";

        echo "</tr>";

    }

    echo "</table>";

    /* Begin Navigation */

    echo "<table border='1' align='center'>";

    echo "<tr>";

    if( $page-1>=1) {
        echo "<td><a href=".$_SERVER['PHP_SELF']."?page=".($page-1).">Previous</a></td>";
    }

    if( $page+1<=$total_pages) {
        echo "<td><a href=".$_SERVER['PHP_SELF']."?page=".($page+1).">Next</a></td>";
    }

    echo "</tr>";

    echo "</table>";

    /* End Navigation */

}

catch(PDOException $e) {
        echo  $e->getMessage();
}

?>

In your file you’ve used additional parameters to execute paging:

  • $page : This variable holds the current page in your script. When moving between the pages, your script retrieves a URL parameter named page using the $_GET['page'] variable.
  • $per_page: This variable holds the maximum records that you want to be displayed per page. In your case, you want to list three products on each page.
  • $total_records: Before you list the products, you’re executing a SQL statement to get a total count of records in your target table and assigning it to the $total_records variable.
  • $offset: This variable represents the total records to skip before the first row. This value is calculated dynamically by your PHP script using the formula $offset=($page-1)*$per_page. You may adapt this formula to your PHP pagination projects. Remember you can change the $per_page variable to suit your needs. For instance, you might change it to a value of 50 to display fifty items per page if you’re running a website or another amount for a mobile device.

Again, visit your IP address in a browser and replace your_server_ip with the public IP address of your server:

http://your_server_ip/pagination_test.php

You’ll now see some navigation buttons at the bottom of the page. On the first page, you will not get a Previous button. The same case happens on the last page where you will not get the Next page button. Also, note how the page URL parameter changes as you visit each page.

The navigation links at the bottom of the page are achieved using the following PHP code snippet from your file:

. . .
    if( $page-1>=1) {
        echo "<td><a href=".$_SERVER['PHP_SELF']."?page=".($page-1).">Previous</a></td>";
    }

    if( $page+1<=$total_pages) {
        echo "<td><a href=".$_SERVER['PHP_SELF']."?page=".($page+1).">Next</a></td>";
    }
. . .

Here, the $page variable represents the current page number. Then, to get the previous page, the code will minus 1 from the variable. So, if you’re on page 2, the formula (2-1) will give you a result of 1 and this will be the previous page to appear in the link. However, keep in mind that it will only show the previous page if there is a result greater or equal to 1.

Similarly, to get to the next page, you add one to the $page variable and you must also make sure that the $page result that we append to the page URL parameter is not greater than the total pages that you’ve calculated in your PHP code.

At this point, your PHP script is working with pagination and you are able to implement the MySQL LIMIT clause for better record navigation.

Conclusion

In this tutorial, you implemented paging in MySQL with PHP on an Ubuntu 18.04 server. You can use these steps with a larger recordset using the PHP script to include pagination. By using pagination on your website or application you can create better user navigation and optimum resource utilization on your server.