How to setting an ssh connection on a Virtualbox Ubuntu

How to setting an ssh connection on a Virtualbox Ubuntu

How To setting SSH connection to Ubuntu on Virtualbox. How to setting an ssh connection on a Virtualbox Ubuntu.

Create an Ubuntu Virtualbox

Create a new Virtual Machine and check Create a new virtual disk

Name your virtualmachine, select Linux as OS type and Ubuntu (64 bit) as linux version.

Set the virtual RAM at half of the your real RAM, in my case I have 8gb on my laptop, so i give to virtual machine 4gb

Create a new virtual disk:

Select the source for install your Ubuntu OS, select VDI (VirtualBox Disk Image)

Select Dynamic allocation:

Set the size of your virtual Hard Disk and click on create:

Start your Machine:

Select the iso image of ubuntu (You must download it from ubuntu official web site).

Install Ubuntu following the installation wizard:

After have installed Ubuntu, login your virtualmachine ubuntu with username and password inserted during the installation:

Install openssh-server

sudo apt-get update
sudo apt-get install openssh-server

stop your virtualmachine.

Set the net of your virtualmachine:

click on settings & net:

Select NAT as connection and click on advanced, and set as in the picture below.

Click on port forwarding:

and set the SSH parameter:

Name: SSH

Protocol: TCP

Host IP: 127.0.0.1

Host Port: 2222

IP Guest: Empty

Port Guest: 22

Start your Virtual Machine:

When your VM is started, open your terminal and try to connect:

ssh [email protected] -p 2222

now you shoud be inside your virtualmachine.

Problems and Possible Solutions:

If you can’t connect try to disable or change settings to ubuntu firewall:

sudo ufw disable

or try to connect to the ip of VM:

click on global tools:

Click on create:

Back to Settings > Network and select adapter 1

Select Adapter only Host.

go inside the VM and get the ip number with ifconfig and get ip address:

ifconfig

In my case the ip is 192.168.56.101

now from terminal I try to connect in ssh to the ip 192.168.56.101

ssh [email protected]

30s ad

Vue.js Essentials - 3 Course Bundle

Sıfırdan İleri Seviye Vue.JS Eğitimi ve Uygulama Geliştirme

Vue.js 2 Basics in just 1 hour FREE

Programador FullStack JS Vue Node: Proj Galeria Vídeo e CRUD

Learn by Doing: Vue JS 2.0 the Right Way

Build Docker Images and Host a Docker Image Repository with GitLab

Build Docker Images and Host a Docker Image Repository with GitLab

In this tutorial, you'll learn how to build Docker images and host a Docker image repository with GitLab. We set up a new GitLab runner to build Docker images, created a private Docker registry to store them in, and updated a Node.js app to be built and tested inside of Docker containers.

In this tutorial, you'll learn how to build Docker images and host a Docker image repository with GitLab. We set up a new GitLab runner to build Docker images, created a private Docker registry to store them in, and updated a Node.js app to be built and tested inside of Docker containers.

Introduction

Containerization is quickly becoming the most accepted method of packaging and deploying applications in cloud environments. The standardization it provides, along with its resource efficiency (when compared to full virtual machines) and flexibility, make it a great enabler of the modern DevOps mindset. Many interesting cloud native deployment, orchestration, and monitoring strategies become possible when your applications and microservices are fully containerized.

Docker containers are by far the most common container type today. Though public Docker image repositories like Docker Hub are full of containerized open source software images that you can docker pull and use today, for private code you'll need to either pay a service to build and store your images, or run your own software to do so.

GitLab Community Edition is a self-hosted software suite that provides Git repository hosting, project tracking, CI/CD services, and a Docker image registry, among other features. In this tutorial we will use GitLab's continuous integration service to build Docker images from an example Node.js app. These images will then be tested and uploaded to our own private Docker registry.

Prerequisites

Before we begin, we need to set up a secure GitLab server, and a GitLab CI runner to execute continuous integration tasks. The sections below will provide links and more details.

A GitLab Server Secured with SSL

To store our source code, run CI/CD tasks, and host the Docker registry, we need a GitLab instance installed on an Ubuntu 16.04 server. GitLab currently recommends a server with at least 2 CPU cores and 4GB of RAM. Additionally, we'll secure the server with SSL certificates from Let's Encrypt. To do so, you'll need a domain name pointed at the server.

A GitLab CI Runner

Set Up Continuous Integration Pipelines with GitLab CI on Ubuntu 16.04 will give you an overview of GitLab's CI service, and show you how to set up a CI runner to process jobs. We will build on top of the demo app and runner infrastructure created in this tutorial.

Step 1 — Setting Up a Privileged GitLab CI Runner

In the prerequisite GitLab continuous integration tutorial, we set up a GitLab runner using sudo gitlab-runner register and its interactive configuration process. This runner is capable of running builds and tests of software inside of isolated Docker containers.

However, in order to build Docker images, our runner needs full access to a Docker service itself. The recommended way to configure this is to use Docker's official docker-in-docker image to run the jobs. This requires granting the runner a special privileged execution mode, so we'll create a second runner with this mode enabled.

Note: Granting the runner privileged mode basically disables all of the security advantages of using containers. Unfortunately, the other methods of enabling Docker-capable runners also carry similar security implications. Please look at the official GitLab documentation on Docker Build to learn more about the different runner options and which is best for your situation.

Read Also: How to Create Docker Image with MySQL Database

Because there are security implications to using a privileged runner, we are going to create a project-specific runner that will only accept Docker jobs on our hello_hapi project (GitLab admins can always manually add this runner to other projects at a later time). From your hello_hapi project page, click Settings at the bottom of the left-hand menu, then click CI/CD in the submenu:

Build Docker Images and Host a Docker Image Repository with GitLab

Now click the Expand button next to the Runners settings section:

Build Docker Images and Host a Docker Image Repository with GitLab

There will be some information about setting up a Specific Runner, including a registration token. Take note of this token. When we use it to register a new runner, the runner will be locked to this project only.

Build Docker Images and Host a Docker Image Repository with GitLab

While we're on this page, click the Disable shared Runners button. We want to make sure our Docker jobs always run on our privileged runner. If a non-privileged shared runner was available, GitLab might choose to use that one, which would result in build errors.

Log in to the server that has your current CI runner on it. If you don't have a machine set up with runners already, go back and complete the Installing the GitLab CI Runner Service

section of the prerequisite tutorial before proceeding.

Now, run the following command to set up the privileged project-specific runner:

    sudo gitlab-runner register -n \
      --url https://gitlab.example.com/ \
      --registration-token your-token \
      --executor docker \
      --description "docker-builder" \
      --docker-image "docker:latest" \
      --docker-privileged

Output

Registering runner... succeeded                     runner=61SR6BwV
Runner registered successfully. Feel free to start it, but if it's running already the config should be automatically reloaded!

Be sure to substitute your own information. We set all of our runner options on the command line instead of using the interactive prompts, because the prompts don't allow us to specify --docker-privileged mode.

Your runner is now set up, registered, and running. To verify, switch back to your browser. Click the wrench icon in the main GitLab menu bar, then click Runners in the left-hand menu. Your runners will be listed:

Build Docker Images and Host a Docker Image Repository with GitLab

Now that we have a runner capable of building Docker images, let's set up a private Docker registry for it to push images to.

Read Also: Docker All The Things

Step 2 — Setting Up GitLab's Docker Registry

Setting up your own Docker registry lets you push and pull images from your own private server, increasing security and reducing the dependencies your workflow has on outside services.

GitLab will set up a private Docker registry with just a few configuration updates. First we'll set up the URL where the registry will reside. Then we will (optionally) configure the registry to use an S3-compatible object storage service to store its data.

SSH into your GitLab server, then open up the GitLab configuration file:

sudo nano /etc/gitlab/gitlab.rb

Scroll down to the Container Registry settings section. We're going to uncomment the registry_external_url line and set it to our GitLab hostname with a port number of 5555:

/etc/gitlab/gitlab.rb

registry_external_url 'https://gitlab.example.com:5555'

Next, add the following two lines to tell the registry where to find our Let's Encrypt certificates:

/etc/gitlab/gitlab.rb

registry_nginx['ssl_certificate'] = "/etc/letsencrypt/live/gitlab.example.com/fullchain.pem"
registry_nginx['ssl_certificate_key'] = "/etc/letsencrypt/live/gitlab.example.com/privkey.pem"

Save and close the file, then reconfigure GitLab:

sudo gitlab-ctl reconfigure

Output

gitlab Reconfigured!

Update the firewall to allow traffic to the registry port:

sudo ufw allow 5555

Now switch to another machine with Docker installed, and log in to the private Docker registry. If you don’t have Docker on your local development computer, you can use whichever server is set up to run your GitLab CI jobs, as it has Docker installed already:

docker login gitlab.example.com:5555

You will be prompted for your username and password. Use your GitLab credentials to log in.

Output
Login Succeeded 

Success! The registry is set up and working. Currently it will store files on the GitLab server's local filesystem. If you'd like to use an object storage service instead, continue with this section. If not, skip down to Step 3.

To set up an object storage backend for the registry, we need to know the following information about our object storage service:

  • Access Key
  • Secret Key
  • Region (us-east-1) for example, if using Amazon S3, or Region Endpoint if using an S3-compatible service ([https://nyc.digitaloceanspaces.com](https://nyc.digitaloceanspaces.com))
  • Bucket Name

If you're using DigitalOcean Spaces, you can find out how to set up a new Space and get the above information by reading How To Create a DigitalOcean Space and API Key.

When you have your object storage information, open the GitLab configuration file:

sudo nano /etc/gitlab/gitlab.rb

Once again, scroll down to the container registry section. Look for the registry['storage'] block, uncomment it, and update it to the following, again making sure to substitute your own information where appropriate:

/etc/gitlab/gitlab.rb

registry['storage'] = {
  's3' => {
    'accesskey' => 'your-key',
    'secretkey' => 'your-secret',
    'bucket' => 'your-bucket-name',
    'region' => 'nyc3',
    'regionendpoint' => 'https://nyc3.digitaloceanspaces.com'
  }
}

If you're using Amazon S3, you only need region and not regionendpoint. If you're using an S3-compatible service like Spaces, you'll need regionendpoint. In this case region doesn't actually configure anything and the value you enter doesn't matter, but it still needs to be present and not blank.

Save and close the file.

Note: There is currently a bug where the registry will shut down after thirty seconds if your object storage bucket is empty. To avoid this, put a file in your bucket before running the next step. You can remove it later, after the registry has added its own objects.

If you are using DigitalOcean Spaces, you can drag and drop to upload a file using the Control Panel interface.

Reconfigure GitLab one more time:

sudo gitlab-ctl reconfigure

On your other Docker machine, log in to the registry again to make sure all is well:

docker login gitlab.example.com:5555

You should get a Login Succeeded message.

Now that we've got our Docker registry set up, let's update our application's CI configuration to build and test our app, and push Docker images to our private registry.

Step 3 — Updating gitlab-ci.yaml and Building a Docker Image

Note: If you didn't complete the prerequisite article on GitLab CI you'll need to copy over the example repository to your GitLab server. Follow the Copying the Example Repository From GitHub section to do so.

To get our app building in Docker, we need to update the .gitlab-ci.yml file. You can edit this file right in GitLab by clicking on it from the main project page, then clicking the Edit button. Alternately, you could clone the repo to your local machine, edit the file, then git push it back to GitLab. That would look like this:

    git clone [email protected]:sammy/hello_hapi.git
    cd hello_hapi
    # edit the file w/ your favorite editor
    git commit -am "updating ci configuration"
    git push

First, delete everything in the file, then paste in the following configuration:

.gitlab-ci.yml

image: docker:latest
services:
- docker:dind

stages:
- build
- test
- release

variables:
  TEST_IMAGE: gitlab.example.com:5555/sammy/hello_hapi:$CI_COMMIT_REF_NAME
  RELEASE_IMAGE: gitlab.example.com:5555/sammy/hello_hapi:latest

before_script:
  - docker login -u gitlab-ci-token -p $CI_JOB_TOKEN gitlab.example.com:5555

build:
  stage: build
  script:
    - docker build --pull -t $TEST_IMAGE .
    - docker push $TEST_IMAGE

test:
  stage: test
  script:
    - docker pull $TEST_IMAGE
    - docker run $TEST_IMAGE npm test

release:
  stage: release
  script:
    - docker pull $TEST_IMAGE
    - docker tag $TEST_IMAGE $RELEASE_IMAGE
    - docker push $RELEASE_IMAGE
  only:
    - master

Be sure to update the highlighted URLs and usernames with your own information, then save with the Commit changes button in GitLab. If you're updating the file outside of GitLab, commit the changes and git push back to GitLab.

This new config file tells GitLab to use the latest docker image (image: docker:latest) and link it to the docker-in-docker service (docker:dind). It then defines build, test, and release stages. The build stage builds the Docker image using the Dockerfile provided in the repo, then uploads it to our Docker image registry. If that succeeds, the test stage will download the image we just built and run the npm test command inside it. If the test stage is successful, the release stage will pull the image, tag it as hello_hapi:latest and push it back to the registry.

Depending on your workflow, you could also add additional test stages, or even deploy stages that push the app to a staging or production environment.

Updating the configuration file should have triggered a new build. Return to the hello_hapi project in GitLab and click on the CI status indicator for the commit:

Build Docker Images and Host a Docker Image Repository with GitLab

On the resulting page you can then click on any of the stages to see their progress:

Build Docker Images and Host a Docker Image Repository with GitLab

Build Docker Images and Host a Docker Image Repository with GitLab

Eventually, all stages should indicate they were successful by showing green check mark icons. We can find the Docker images that were just built by clicking the Registry item in the left-hand menu:

Build Docker Images and Host a Docker Image Repository with GitLab

If you click the little "document" icon next to the image name, it will copy the appropriate docker pull ... command to your clipboard. You can then pull and run your image:

    docker pull gitlab.example.com:5555/sammy/hello_hapi:latest
    docker run -it --rm -p 3000:3000 gitlab.example.com:5555/sammy/hello_hapi:latest

Output

> [email protected] start /usr/src/app
> node app.js

Server running at: http://56fd5df5ddd3:3000

The image has been pulled down from the registry and started in a container. Switch to your browser and connect to the app on port 3000 to test. In this case we're running the container on our local machine, so we can access it via localhost at the following URL:

http://localhost:3000/hello/test

Output

Hello, test!

Success! You can stop the container with CTRL-C. From now on, every time we push new code to the master branch of our repository, we'll automatically build and test a new hello_hapi:latest image.

Conclusion

In this tutorial we set up a new GitLab runner to build Docker images, created a private Docker registry to store them in, and updated a Node.js app to be built and tested inside of Docker containers.

How To Set Up Django with Postgres, Nginx, and Gunicorn on Ubuntu 16.04

How To Set Up Django with Postgres, Nginx, and Gunicorn on Ubuntu 16.04

In this guide, we will demonstrate how to install and configure some components on Ubuntu 16.04 to support and serve Django applications. We will be setting up a PostgreSQL database instead of using the default SQLite database. We will configure the Gunicorn application server to interface with our applications. We will then set up Nginx to reverse proxy to Gunicorn, giving us access to its security and performance features to serve our apps.

In this guide, we will demonstrate how to install and configure some components on Ubuntu 16.04 to support and serve Django applications. We will be setting up a PostgreSQL database instead of using the default SQLite database. We will configure the Gunicorn application server to interface with our applications. We will then set up Nginx to reverse proxy to Gunicorn, giving us access to its security and performance features to serve our apps.

Introduction

Django is a powerful web framework that can help you get your Python application or website off the ground. Django includes a simplified development server for testing your code locally, but for anything even slightly production related, a more secure and powerful web server is required.

Prerequisites and Goals

In order to complete this guide, you should have a fresh Ubuntu 16.04 server instance with a non-root user with sudo privileges configured. You can learn how to set this up by running through our initial server setup guide.

We will be installing Django within a virtual environment. Installing Django into an environment specific to your project will allow your projects and their requirements to be handled separately.

Once we have our database and application up and running, we will install and configure the Gunicorn application server. This will serve as an interface to our application, translating client requests in HTTP to Python calls that our application can process. We will then set up Nginx in front of Gunicorn to take advantage of its high performance connection handling mechanisms and its easy-to-implement security features.

Let’s get started.

Install the Packages from the Ubuntu Repositories

To begin the process, we’ll download and install all of the items we need from the Ubuntu repositories. We will use the Python package manager pip to install additional components a bit later.

We need to update the local apt package index and then download and install the packages. The packages we install depend on which version of Python your project will use.

If you are using Python 2, type:

sudo apt-get update
sudo apt-get install python-pip python-dev libpq-dev postgresql postgresql-contrib nginx


If you are using Django with Python 3, type:

sudo apt-get update
sudo apt-get install python3-pip python3-dev libpq-dev postgresql postgresql-contrib nginx


This will install pip, the Python development files needed to build Gunicorn later, the Postgres database system and the libraries needed to interact with it, and the Nginx web server.

Create the PostgreSQL Database and User

We’re going to jump right in and create a database and database user for our Django application.

By default, Postgres uses an authentication scheme called “peer authentication” for local connections. Basically, this means that if the user’s operating system username matches a valid Postgres username, that user can login with no further authentication.

During the Postgres installation, an operating system user named postgres was created to correspond to the postgres PostgreSQL administrative user. We need to use this user to perform administrative tasks. We can use sudo and pass in the username with the -u option.

Log into an interactive Postgres session by typing:

sudo -u postgres psql


You will be given a PostgreSQL prompt where we can set up our requirements.

First, create a database for your project:

CREATE DATABASE myproject;


Every Postgres statement must end with a semi-colon, so make sure that your command ends with one if you are experiencing issues.

Next, create a database user for our project. Make sure to select a secure password:

CREATE USER myprojectuser WITH PASSWORD 'password';


Afterwards, we’ll modify a few of the connection parameters for the user we just created. This will speed up database operations so that the correct values do not have to be queried and set each time a connection is established.

We are setting the default encoding to UTF-8, which Django expects. We are also setting the default transaction isolation scheme to “read committed”, which blocks reads from uncommitted transactions. Lastly, we are setting the timezone. By default, our Django projects will be set to use UTC. These are all recommendations from the Django project itself:

ALTER ROLE myprojectuser SET client_encoding TO 'utf8';
ALTER ROLE myprojectuser SET default_transaction_isolation TO 'read committed';
ALTER ROLE myprojectuser SET timezone TO 'UTC';


Now, we can give our new user access to administer our new database:

GRANT ALL PRIVILEGES ON DATABASE myproject TO myprojectuser;


When you are finished, exit out of the PostgreSQL prompt by typing:

\q


Create a Python Virtual Environment for your Project

Now that we have our database, we can begin getting the rest of our project requirements ready. We will be installing our Python requirements within a virtual environment for easier management.

To do this, we first need access to the virtualenv command. We can install this with pip.

If you are using Python 2, upgrade pip and install the package by typing:

sudo -H pip install --upgrade pip
sudo -H pip install virtualenv


If you are using Python 3, upgrade pip and install the package by typing:

sudo -H pip3 install --upgrade pip
sudo -H pip3 install virtualenv


With virtualenv installed, we can start forming our project. Create and move into a directory where we can keep our project files:

mkdir ~/myproject
cd ~/myproject


Within the project directory, create a Python virtual environment by typing:

virtualenv myprojectenv


This will create a directory called myprojectenv within your myproject directory. Inside, it will install a local version of Python and a local version of pip. We can use this to install and configure an isolated Python environment for our project.

Before we install our project’s Python requirements, we need to activate the virtual environment. You can do that by typing:

source myprojectenv/bin/activate


Your prompt should change to indicate that you are now operating within a Python virtual environment. It will look something like this: (myprojectenv)[email protected]:~/myproject$.

With your virtual environment active, install Django, Gunicorn, and the psycopg2 PostgreSQL adaptor with the local instance of pip:

Note

Regardless of which version of Python you are using, when the virtual environment is activated, you should use the pip command (not pip3).

pip install django gunicorn psycopg2


You should now have all of the software needed to start a Django project.

Create and Configure a New Django Project

With our Python components installed, we can create the actual Django project files.

Create the Django Project

Since we already have a project directory, we will tell Django to install the files here. It will create a second level directory with the actual code, which is normal, and place a management script in this directory. The key to this is that we are defining the directory explicitly instead of allowing Django to make decisions relative to our current directory:

django-admin.py startproject myproject ~/myproject


At this point, your project directory (~/myproject in our case) should have the following content:

  • ~/myproject/manage.py: A Django project management script.
  • ~/myproject/myproject/: The Django project package. This should contain the __init__.py, settings.py, urls.py, and wsgi.py files.
  • ~/myproject/myprojectenv/: The virtual environment directory we created earlier.

Adjust the Project Settings

The first thing we should do with our newly created project files is adjust the settings. Open the settings file in your text editor:

nano ~/myproject/myproject/settings.py


Start by locating the ALLOWED_HOSTS directive. This defines a list of the server’s addresses or domain names may be used to connect to the Django instance. Any incoming requests with a Host header that is not in this list will raise an exception. Django requires that you set this to prevent a certain class of security vulnerability.

In the square brackets, list the IP addresses or domain names that are associated with your Django server. Each item should be listed in quotations with entries separated by a comma. If you wish requests for an entire domain and any subdomains, prepend a period to the beginning of the entry. In the snippet below, there are a few commented out examples used to demonstrate:

~/myproject/myproject/settings.py

. . .
# The simplest case: just add the domain name(s) and IP addresses of your Django server
# ALLOWED_HOSTS = [ 'example.com', '203.0.113.5']
# To respond to 'example.com' and any subdomains, start the domain with a dot
# ALLOWED_HOSTS = ['.example.com', '203.0.113.5']
ALLOWED_HOSTS = ['your_server_domain_or_IP', 'second_domain_or_IP', . . .]


Next, find the section that configures database access. It will start with DATABASES. The configuration in the file is for a SQLite database. We already created a PostgreSQL database for our project, so we need to adjust the settings.

Change the settings with your PostgreSQL database information. We tell Django to use the psycopg2 adaptor we installed with pip. We need to give the database name, the database username, the database user’s password, and then specify that the database is located on the local computer. You can leave the PORT setting as an empty string:

~/myproject/myproject/settings.py

. . .

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql_psycopg2',
        'NAME': 'myproject',
        'USER': 'myprojectuser',
        'PASSWORD': 'password',
        'HOST': 'localhost',
        'PORT': '',
    }
}

. . .


Next, move down to the bottom of the file and add a setting indicating where the static files should be placed. This is necessary so that Nginx can handle requests for these items. The following line tells Django to place them in a directory called static in the base project directory:

~/myproject/myproject/settings.py

. . .

STATIC_URL = '/static/'
STATIC_ROOT = os.path.join(BASE_DIR, 'static/')


Save and close the file when you are finished.

Complete Initial Project Setup

Now, we can migrate the initial database schema to our PostgreSQL database using the management script:

~/myproject/manage.py makemigrations
~/myproject/manage.py migrate


Create an administrative user for the project by typing:

~/myproject/manage.py createsuperuser


You will have to select a username, provide an email address, and choose and confirm a password.

We can collect all of the static content into the directory location we configured by typing:

~/myproject/manage.py collectstatic


You will have to confirm the operation. The static files will then be placed in a directory called static within your project directory.

If you followed the initial server setup guide, you should have a UFW firewall protecting your server. In order to test the development server, we’ll have to allow access to the port we’ll be using.

Create an exception for port 8000 by typing:

sudo ufw allow 8000


Finally, you can test our your project by starting up the Django development server with this command:

~/myproject/manage.py runserver 0.0.0.0:8000


In your web browser, visit your server’s domain name or IP address followed by :8000:

http://server_domain_or_IP:8000


You should see the default Django index page:

If you append /admin to the end of the URL in the address bar, you will be prompted for the administrative username and password you created with the createsuperuser command:

After authenticating, you can access the default Django admin interface:

When you are finished exploring, hit CTRL-C in the terminal window to shut down the development server.

Testing Gunicorn’s Ability to Serve the Project

The last thing we want to do before leaving our virtual environment is test Gunicorn to make sure that it can serve the application. We can do this by entering our project directory and using gunicorn to load the project’s WSGI module:

cd ~/myproject
gunicorn --bind 0.0.0.0:8000 myproject.wsgi


This will start Gunicorn on the same interface that the Django development server was running on. You can go back and test the app again.

Note: The admin interface will not have any of the styling applied since Gunicorn does not know about the static CSS content responsible for this.

We passed Gunicorn a module by specifying the relative directory path to Django’s wsgi.py file, which is the entry point to our application, using Python’s module syntax. Inside of this file, a function called application is defined, which is used to communicate with the application. To learn more about the WSGI specification, click here.

When you are finished testing, hit CTRL-C in the terminal window to stop Gunicorn.

We’re now finished configuring our Django application. We can back out of our virtual environment by typing:

deactivate


The virtual environment indicator in your prompt will be removed.

Create a Gunicorn systemd Service File

We have tested that Gunicorn can interact with our Django application, but we should implement a more robust way of starting and stopping the application server. To accomplish this, we’ll make a systemd service file.

Create and open a systemd service file for Gunicorn with sudo privileges in your text editor:

sudo nano /etc/systemd/system/gunicorn.service


Start with the [Unit] section, which is used to specify metadata and dependencies. We’ll put a description of our service here and tell the init system to only start this after the networking target has been reached:

/etc/systemd/system/gunicorn.service

[Unit]
Description=gunicorn daemon
After=network.target


Next, we’ll open up the [Service] section. We’ll specify the user and group that we want to process to run under. We will give our regular user account ownership of the process since it owns all of the relevant files. We’ll give group ownership to the www-data group so that Nginx can communicate easily with Gunicorn.

We’ll then map out the working directory and specify the command to use to start the service. In this case, we’ll have to specify the full path to the Gunicorn executable, which is installed within our virtual environment. We will bind it to a Unix socket within the project directory since Nginx is installed on the same computer. This is safer and faster than using a network port. We can also specify any optional Gunicorn tweaks here. For example, we specified 3 worker processes in this case:

/etc/systemd/system/gunicorn.service

[Unit]
Description=gunicorn daemon
After=network.target

[Service]
User=sammy
Group=www-data
WorkingDirectory=/home/sammy/myproject
ExecStart=/home/sammy/myproject/myprojectenv/bin/gunicorn --access-logfile - --workers 3 --bind unix:/home/sammy/myproject/myproject.sock myproject.wsgi:application


Finally, we’ll add an [Install] section. This will tell systemd what to link this service to if we enable it to start at boot. We want this service to start when the regular multi-user system is up and running:

/etc/systemd/system/gunicorn.service

[Unit]
Description=gunicorn daemon
After=network.target

[Service]
User=sammy
Group=www-data
WorkingDirectory=/home/sammy/myproject
ExecStart=/home/sammy/myproject/myprojectenv/bin/gunicorn --access-logfile - --workers 3 --bind unix:/home/sammy/myproject/myproject.sock myproject.wsgi:application

[Install]
WantedBy=multi-user.target


With that, our systemd service file is complete. Save and close it now.

We can now start the Gunicorn service we created and enable it so that it starts at boot:

sudo systemctl start gunicorn
sudo systemctl enable gunicorn


We can confirm that the operation was successful by checking for the socket file.

Check for the Gunicorn Socket File

Check the status of the process to find out whether it was able to start:

sudo systemctl status gunicorn


Next, check for the existence of the myproject.sock file within your project directory:

ls /home/sammy/myproject

Outputmanage.py  myproject  myprojectenv  myproject.sock  static


If the systemctl status command indicated that an error occurred or if you do not find the myproject.sock file in the directory, it’s an indication that Gunicorn was not able to start correctly. Check the Gunicorn process logs by typing:

sudo journalctl -u gunicorn


Take a look at the messages in the logs to find out where Gunicorn ran into problems. There are many reasons that you may have run into problems, but often, if Gunicorn was unable to create the socket file, it is for one of these reasons:

  • ~/myproject/manage.py: A Django project management script.
  • ~/myproject/myproject/: The Django project package. This should contain the __init__.py, settings.py, urls.py, and wsgi.py files.
  • ~/myproject/myprojectenv/: The virtual environment directory we created earlier.

If you make changes to the /etc/systemd/system/gunicorn.service file, reload the daemon to reread the service definition and restart the Gunicorn process by typing:

sudo systemctl daemon-reload
sudo systemctl restart gunicorn


Make sure you troubleshoot any of the above issues before continuing.

Configure Nginx to Proxy Pass to Gunicorn

Now that Gunicorn is set up, we need to configure Nginx to pass traffic to the process.

Start by creating and opening a new server block in Nginx’s sites-available directory:

sudo nano /etc/nginx/sites-available/myproject


Inside, open up a new server block. We will start by specifying that this block should listen on the normal port 80 and that it should respond to our server’s domain name or IP address:

/etc/nginx/sites-available/myproject

server {
    listen 80;
    server_name server_domain_or_IP;
}


Next, we will tell Nginx to ignore any problems with finding a favicon. We will also tell it where to find the static assets that we collected in our ~/myproject/static directory. All of these files have a standard URI prefix of “/static”, so we can create a location block to match those requests:

/etc/nginx/sites-available/myproject

server {
    listen 80;
    server_name server_domain_or_IP;

    location = /favicon.ico { access_log off; log_not_found off; }
    location /static/ {
        root /home/sammy/myproject;
    }
}


Finally, we’ll create a location / {} block to match all other requests. Inside of this location, we’ll include the standard proxy_params file included with the Nginx installation and then we will pass the traffic to the socket that our Gunicorn process created:

/etc/nginx/sites-available/myproject

server {
    listen 80;
    server_name server_domain_or_IP;

    location = /favicon.ico { access_log off; log_not_found off; }
    location /static/ {
        root /home/sammy/myproject;
    }

    location / {
        include proxy_params;
        proxy_pass http://unix:/home/sammy/myproject/myproject.sock;
    }
}


Save and close the file when you are finished. Now, we can enable the file by linking it to the sites-enabled directory:

sudo ln -s /etc/nginx/sites-available/myproject /etc/nginx/sites-enabled


Test your Nginx configuration for syntax errors by typing:

sudo nginx -t


If no errors are reported, go ahead and restart Nginx by typing:

sudo systemctl restart nginx


Finally, we need to open up our firewall to normal traffic on port 80. Since we no longer need access to the development server, we can remove the rule to open port 8000 as well:

sudo ufw delete allow 8000
sudo ufw allow 'Nginx Full'


You should now be able to go to your server’s domain or IP address to view your application.

Note

After configuring Nginx, the next step should be securing traffic to the server using SSL/TLS. This is important because without it, all information, including passwords are sent over the network in plain text.

If you have a domain name, the easiest way get an SSL certificate to secure your traffic is using Let’s Encrypt. Follow this guide to set up Let’s Encrypt with Nginx on Ubuntu 16.04.

If you do not have a domain name, you can still secure your site for testing and learning with a self-signed SSL certificate.

Troubleshooting Nginx and Gunicorn

If this last step does not show your application, you will need to troubleshoot your installation.

Nginx Is Showing the Default Page Instead of the Django Application

If Nginx displays the default page instead of proxying to your application, it usually means that you need to adjust the server_name within the /etc/nginx/sites-available/myproject file to point to your server’s IP address or domain name.

Nginx uses the server_name to determine which server block to use to respond to requests. If you are seeing the default Nginx page, it is a sign that Nginx wasn’t able to match the request to a sever block explicitly, so it’s falling back on the default block defined in /etc/nginx/sites-available/default.

The server_name in your project’s server block must be more specific than the one in the default server block to be selected.

Nginx Is Displaying a 502 Bad Gateway Error Instead of the Django Application

A 502 error indicates that Nginx is unable to successfully proxy the request. A wide range of configuration problems express themselves with a 502 error, so more information is required to troubleshoot properly.

The primary place to look for more information is in Nginx’s error logs. Generally, this will tell you what conditions caused problems during the proxying event. Follow the Nginx error logs by typing:

sudo tail -F /var/log/nginx/error.log


Now, make another request in your browser to generate a fresh error (try refreshing the page). You should see a fresh error message written to the log. If you look at the message, it should help you narrow down the problem.

You might see some of the following message:

connect() to unix:/home/sammy/myproject/myproject.sock failed (2: No such file or directory)

This indicates that Nginx was unable to find the myproject.sock file at the given location. You should compare the proxy_pass location defined within /etc/nginx/sites-available/myproject file to the actual location of the myproject.sock file generated in your project directory.

If you cannot find a myproject.sock file within your project directory, it generally means that the gunicorn process was unable to create it. Go back to the section on checking for the Gunicorn socket file to step through the troubleshooting steps for Gunicorn.

connect() to unix:/home/sammy/myproject/myproject.sock failed (13: Permission denied)

This indicates that Nginx was unable to connect to the Gunicorn socket because of permissions problems. Usually, this happens when the procedure is followed using the root user instead of a sudo user. While the Gunicorn process is able to create the socket file, Nginx is unable to access it.

This can happen if there are limited permissions at any point between the root directory (/) the myproject.sock file. We can see the permissions and ownership values of the socket file and each of its parent directories by passing the absolute path to our socket file to the namei command:

namei -nom /home/sammy/myproject/myproject.sock

Outputf: /home/sammy/myproject/myproject.sock
 drwxr-xr-x root  root     /
 drwxr-xr-x root  root     home
 drwxr-xr-x sammy sammy    sammy
 drwxrwxr-x sammy sammy    myproject
 srwxrwxrwx sammy www-data myproject.sock


The output displays the permissions of each of the directory components. By looking at the permissions (first column), owner (second column) and group owner (third column), we can figure out what type of access is allowed to the socket file.

In the above example, the socket file and each of the directories leading up to the socket file have world read and execute permissions (the permissions column for the directories end with r-x instead of ---). The Nginx process should be able to access the socket successfully.

If any of the directories leading up to the socket do not have world read and execute permission, Nginx will not be able to access the socket without allowing world read and execute permissions or making sure group ownership is given to a group that Nginx is a part of. For sensitive locations like the /root directory, both of the above options are dangerous. It’s better to move the project files outside of the directory, where you can safely control access without compromising security.

Django Is Displaying: “could not connect to server: Connection refused”

One message that you may see from Django when attempting to access parts of the application in the web browser is:

OperationalError at /admin/login/
could not connect to server: Connection refused
    Is the server running on host "localhost" (127.0.0.1) and accepting
    TCP/IP connections on port 5432?


This indicates that Django is unable to connect to the Postgres database. Make sure that the Postgres instance is running by typing:

sudo systemctl status postgresql


If it is not, you can start it and enable it to start automatically at boot (if it is not already configured to do so) by typing:

sudo systemctl start postgresql
sudo systemctl enable postgresql


If you are still having issues, make sure the database settings defined in the ~/myproject/myproject/settings.py file are correct.

Further Troubleshooting

For additional troubleshooting, the logs can help narrow down root causes. Check each of them in turn and look for messages indicating problem areas.

The following logs may be helpful:

  • ~/myproject/manage.py: A Django project management script.
  • ~/myproject/myproject/: The Django project package. This should contain the __init__.py, settings.py, urls.py, and wsgi.py files.
  • ~/myproject/myprojectenv/: The virtual environment directory we created earlier.

As you update your configuration or application, you will likely need to restart the processes to adjust to your changes.

If you update your Django application, you can restart the Gunicorn process to pick up the changes by typing:

sudo systemctl restart gunicorn


If you change gunicorn systemd service file, reload the daemon and restart the process by typing:

sudo systemctl daemon-reload
sudo systemctl restart gunicorn


If you change the Nginx server block configuration, test the configuration and then Nginx by typing:

sudo nginx -t && sudo systemctl restart nginx


These commands are helpful for picking up changes as you adjust your configuration.

Conclusion

In this guide, we’ve set up a Django project in its own virtual environment. We’ve configured Gunicorn to translate client requests so that Django can handle them. Afterwards, we set up Nginx to act as a reverse proxy to handle client connections and serve the correct project depending on the client request.

Django makes creating projects and applications simple by providing many of the common pieces, allowing you to focus on the unique elements. By leveraging the general tool chain described in this article, you can easily serve the applications you create from a single server.

Learn More

Creating Web Sites using Python and Flask

Complete Python: Go from zero to hero in Python

An A-Z of useful Python tricks

A Complete Machine Learning Project Walk-Through in Python

Learning Python: From Zero to Hero

MongoDB with Python Crash Course - Tutorial for Beginners

Introduction to PyTorch and Machine Learning

Python Tutorial for Beginners (2019) - Learn Python for Machine Learning and Web Development

Python and Django Full Stack Web Developer Bootcamp

Django 2.1 & Python | The Ultimate Web Development Bootcamp

Python Django Dev To Deployment

Build a Backend REST API with Python & Django - Advanced

How To Implement Pagination in MySQL with PHP on Ubuntu 18.04

How To Implement Pagination in MySQL with PHP on Ubuntu 18.04

In this tutorial, you’ll build a PHP script to connect to your database and implement pagination to your script using the MySQL LIMIT clause.

Introduction

Pagination is the concept of constraining the number of returned rows in a recordset into separate, orderly pages to allow easy navigation between them, so when there is a large dataset you can configure your pagination to only return a specific number of rows on each page. For example, pagination can help to avoid overwhelming users when a web store contains thousands of products by reducing the number of items listed on a page, as it’s often unlikely a user will need to view every product. Another example is an application that shows records on a mobile device; enabling pagination in such a case would split records into multiple pages that can fit better on a screen.

Besides the visual benefits for end-users, pagination makes applications faster because it reduces the number of records that are returned at a time. This limits the data that needs to be transmitted between the client and the server, which helps preserve server resources such as RAM.

In this tutorial, you’ll build a PHP script to connect to your database and implement pagination to your script using the MySQL LIMIT clause.

Step 1 — Creating a Database User and a Test Database

In this tutorial you’ll create a PHP script that will connect to a MySQL database, fetch records, and display them in an HTML page within a table. You’ll test the PHP script in two different ways from your web browser. First, creating a script without any pagination code to see how the records are displayed. Second, adding page navigation code in the PHP file to understand how pagination works practically.

The PHP code requires a MySQL user for authentication purposes and a sample database to connect to. In this step you’ll create a non-root user for your MySQL database, a sample database, and a table to test the PHP script.

To begin log in to your server. Then log in to your MySQL server with the following command:

sudo mysql -u root -p

Enter the root password of your MySQL server and hit ENTER to continue. Then, you’ll see the MySQL prompt. To create a sample database, which we will call test_db in this tutorial, run the following command:

Create database test_db;

You will see the following output:

OutputQuery OK, 1 row affected (0.00 sec)

Then, create a test_user and grant the user all privileges to the test_db. Replace PASSWORD with a strong value:

GRANT ALL PRIVILEGES ON test_db.* TO 'test_user'@'localhost' IDENTIFIED BY 'PASSWORD';

OutputQuery OK, 1 row affected (0.00 sec)

Reload the MySQL privileges with:

FLUSH PRIVILEGES;

OutputQuery OK, 1 row affected (0.00 sec)

Next, switch to the test_db database to start working directly on the test_db database:

Use test_db;

OutputDatabase changed

Now create a products table. The table will hold your sample products—for this tutorial you’ll require only two columns for the data. The product_id column will serve as the primary key to uniquely identify each record. You’ll use the product_name field to differentiate each item by name:

Create table products (product_id BIGINT PRIMARY KEY, product_name VARCHAR(50) NOT NULL ) Engine = InnoDB;

OutputQuery OK, 0 rows affected (0.02 sec)

To add ten test products to the products table run the following SQL statements:

Insert into products(product_id, product_name) values ('1', 'WIRELESS MOUSE');
Insert into products(product_id, product_name) values ('2', 'BLUETOOTH SPEAKER');
Insert into products(product_id, product_name) values ('3', 'GAMING KEYBOARD');
Insert into products(product_id, product_name) values ('4', '320GB FAST SSD');
Insert into products(product_id, product_name) values ('5', '17 INCHES TFT');
Insert into products(product_id, product_name) values ('6', 'SPECIAL HEADPHONES');
Insert into products(product_id, product_name) values ('7', 'HD GRAPHIC CARD');
Insert into products(product_id, product_name) values ('8', '80MM THERMAL PRINTER');
Insert into products(product_id, product_name) values ('9', 'HDMI TO VGA CONVERTER');
Insert into products(product_id, product_name) values ('10', 'FINGERPRINT SCANNER');

You’ll see this output:

OutputQuery OK, 1 row affected (0.02 sec)

Verify that the products were inserted to the table by running:

select * from products;

You’ll see the products in your output within the two columns:

Output+------------+-----------------------+
| product_id | product_name          |
+------------+-----------------------+
|          1 | WIRELESS MOUSE        |
|          2 | BLUETOOTH SPEAKER     |
|          3 | GAMING KEYBOARD       |
|          4 | 320GB FAST SSD        |
|          5 | 17 INCHES TFT         |
|          6 | SPECIAL HEADPHONES    |
|          7 | HD GRAPHIC CARD       |
|          8 | 80MM THERMAL PRINTER  |
|          9 | HDMI TO VGA CONVERTER |
|         10 | FINGERPRINT SCANNER   |
+------------+-----------------------+
10 rows in set (0.00 sec)

Exit MySQL:

quit;

With the sample database, table, and test data in place, you can now create a PHP script to display data on a web page.

Step 2 — Displaying MySQL Records Without Pagination

Now you’ll create a PHP script that connects to the MySQL database that you created in the previous step and list the products in a web browser. In this step, your PHP code will run without any form of pagination to demonstrate how non-split records show on a single page. Although you only have ten records for testing purposes in this tutorial, seeing the records without pagination will demonstrate why segmenting data will ultimately create a better user experience and put less burden on the server.

Create the PHP script file in the document root of your website with the following command:

sudo nano /var/www/html/pagination_test.php

Then add the following content to the file. Remember to replace PASSWORD with the correct value of the password that you assigned to the test_user in the previous step:

<?php

try {

    $pdo = new PDO("mysql:host=localhost;dbname=test_db", "test_user", "PASSWORD");
    $pdo->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
    $pdo->setAttribute(PDO::ATTR_EMULATE_PREPARES,false);

    $sql="select * from products";

    $stmt = $pdo->prepare($sql);

    $stmt->execute();

    echo "<table border='1' align='center'>";

    while ( ($row = $stmt->fetch(PDO::FETCH_ASSOC) ) !== false) {
        echo "<tr>";

        echo "<td>".$row['product_id']."</td>";

        echo "<td>".$row['product_name']."</td>";

        echo "</tr>";

    }

    echo "</table>";

}

  catch(PDOException $e)

{
    echo  $e->getMessage();
}

?>

Save the file by pressing CTRL+X, Y, and ENTER.

In this script you’re connecting to the MySQL database using the PDO (PHP Data Object) library with the database credentials that you created in Step 1.

PDO is a light-weight interface for connecting to databases. The data access layer is more portable and can work on different databases with just minor code rewrites. PDO has greater security since it supports prepared statements—a feature for making queries run faster in a secure way.

Then, you instruct the PDO API to execute the select * from products statement and list products in an HTML table without pagination. The line $pdo->setAttribute(PDO::ATTR_EMULATE_PREPARES,false); ensures that the data types are returned as they appear in the database. This means that PDO will return the product_id as an integer and the product_name as a string. $pdo->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION); instructs PDO to throw an exception if an error is encountered. For easier debugging you’re catching the error inside the PHP try{}...catch{} block.

To execute the /var/www/html/pagination_test.php PHP script file that you’ve created, visit the following URL replacing your-server-IP with the public IP address of your server:

http://your-server-IP/pagination_test.php

You’ll see a page with a table of your products.

MySQL Records Displayed with a PHP script - No Pagination

Your PHP script is working as expected; listing all products on one page. If you had thousands of products, this would result in a long loop as the products are fetched from the database and rendered on the PHP page.

To overcome this limitation, you will modify the PHP script and include the MySQL LIMIT clause and some navigation links at the bottom of the table to add pagination functionality.

Step 3 — Implementing Pagination with PHP

In this step your goal is to split the test data into multiple and manageable pages. This will not only enhance readability but also use the resources of the server more efficiently. You will modify the PHP script that you created in the previous step to accommodate pagination.

To do this, you’ll be implementing the MySQL LIMIT clause. Before adding this to the script, let’s see an example of the MySQL LIMIT syntax:

Select [column1, column2, column n...] from [table name] LIMIT offset, records;

The LIMIT clause takes two arguments as shown toward the end of this statement. The offset value is the number of records to skip before the first row. records sets the maximum number of records to display per page.

To test pagination, you’ll display three records per page. To get the total number of pages, you must divide the total records from your table with the rows that you want to display per page. You then round the resulting value to the nearest integer using PHP Ceil function as shown in the following PHP code snippet example:

$total_pages=ceil($total_records/$per_page);

Following is the modified version of the PHP script with the full pagination code. To include the pagination and navigation codes, open the /var/www/html/pagination_test.php file:

sudo nano /var/www/html/pagination_test.php

Then, add the following highlighted code to your file:

<?php

try {

    $pdo = new PDO("mysql:host=localhost;dbname=test_db", "test_user", "PASSWORD");
    $pdo->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
    $pdo->setAttribute(PDO::ATTR_EMULATE_PREPARES,false);

    /* Begin Paging Info */

    $page=1;

    if (isset($_GET['page'])) {
        $page=filter_var($_GET['page'], FILTER_SANITIZE_NUMBER_INT);
    }

    $per_page=3;

    $sqlcount="select count(*) as total_records from products";
    $stmt = $pdo->prepare($sqlcount);
    $stmt->execute();
    $row = $stmt->fetch();
    $total_records= $row['total_records'];

    $total_pages=ceil($total_records/$per_page);

    $offset=($page-1)*$per_page;

    /* End Paging Info */

    $sql="select * from products limit $offset,$per_page";

    $stmt = $pdo->prepare($sql);

    $stmt->execute();

    echo "<table border='1' align='center'>";

    while ( ($row = $stmt->fetch(PDO::FETCH_ASSOC) ) !== false) {
        echo "<tr>";

        echo "<td>".$row['product_id']."</td>";

        echo "<td>".$row['product_name']."</td>";

        echo "</tr>";

    }

    echo "</table>";

    /* Begin Navigation */

    echo "<table border='1' align='center'>";

    echo "<tr>";

    if( $page-1>=1) {
        echo "<td><a href=".$_SERVER['PHP_SELF']."?page=".($page-1).">Previous</a></td>";
    }

    if( $page+1<=$total_pages) {
        echo "<td><a href=".$_SERVER['PHP_SELF']."?page=".($page+1).">Next</a></td>";
    }

    echo "</tr>";

    echo "</table>";

    /* End Navigation */

}

catch(PDOException $e) {
        echo  $e->getMessage();
}

?>

In your file you’ve used additional parameters to execute paging:

  • $page : This variable holds the current page in your script. When moving between the pages, your script retrieves a URL parameter named page using the $_GET['page'] variable.
  • $per_page: This variable holds the maximum records that you want to be displayed per page. In your case, you want to list three products on each page.
  • $total_records: Before you list the products, you’re executing a SQL statement to get a total count of records in your target table and assigning it to the $total_records variable.
  • $offset: This variable represents the total records to skip before the first row. This value is calculated dynamically by your PHP script using the formula $offset=($page-1)*$per_page. You may adapt this formula to your PHP pagination projects. Remember you can change the $per_page variable to suit your needs. For instance, you might change it to a value of 50 to display fifty items per page if you’re running a website or another amount for a mobile device.

Again, visit your IP address in a browser and replace your_server_ip with the public IP address of your server:

http://your_server_ip/pagination_test.php

You’ll now see some navigation buttons at the bottom of the page. On the first page, you will not get a Previous button. The same case happens on the last page where you will not get the Next page button. Also, note how the page URL parameter changes as you visit each page.

MySQL Records Displayed with a PHP script with Pagination - Page 1

MySQL Records Displayed with a PHP script with Pagination - Page 2

Final page of MySQL Records Displayed with a PHP script with Pagination - Page 4

The navigation links at the bottom of the page are achieved using the following PHP code snippet from your file:

. . .
    if( $page-1>=1) {
        echo "<td><a href=".$_SERVER['PHP_SELF']."?page=".($page-1).">Previous</a></td>";
    }

    if( $page+1<=$total_pages) {
        echo "<td><a href=".$_SERVER['PHP_SELF']."?page=".($page+1).">Next</a></td>";
    }
. . .

Here, the $page variable represents the current page number. Then, to get the previous page, the code will minus 1 from the variable. So, if you’re on page 2, the formula (2-1) will give you a result of 1 and this will be the previous page to appear in the link. However, keep in mind that it will only show the previous page if there is a result greater or equal to 1.

Similarly, to get to the next page, you add one to the $page variable and you must also make sure that the $page result that we append to the page URL parameter is not greater than the total pages that you’ve calculated in your PHP code.

At this point, your PHP script is working with pagination and you are able to implement the MySQL LIMIT clause for better record navigation.

Conclusion

In this tutorial, you implemented paging in MySQL with PHP on an Ubuntu 18.04 server. You can use these steps with a larger recordset using the PHP script to include pagination. By using pagination on your website or application you can create better user navigation and optimum resource utilization on your server.

How To Install Linux, Nginx, MySQL, PHP (LEMP stack) on Ubuntu 18.04?

How To Install Linux, Nginx, MySQL, PHP (LEMP stack) on Ubuntu 18.04?

This guide demonstrates how to install a LEMP stack on an Ubuntu 18.04 server. The Ubuntu operating system takes care of the first requirement. We will describe how to get the rest of the components up and running.

Introduction

The LEMP software stack is a group of software that can be used to serve dynamic web pages and web applications. This is an acronym that describes a Linux operating system, with an Nginx (pronounced like “Engine-X”) web server. The backend data is stored in the MySQL database and the dynamic processing is handled by PHP.

This guide demonstrates how to install a LEMP stack on an Ubuntu 18.04 server. The Ubuntu operating system takes care of the first requirement. We will describe how to get the rest of the components up and running.

Step 1 – Installing the Nginx Web Server

In order to display web pages to our site visitors, we are going to employ Nginx, a modern, efficient web server.

All of the software used in this procedure will come from Ubuntu’s default package repositories. This means we can use the apt package management suite to complete the necessary installations.

Since this is our first time using apt for this session, start off by updating your server’s package index. Following that, install the server:

sudo apt update
sudo apt install nginx

On Ubuntu 18.04, Nginx is configured to start running upon installation.

If you have the ufw firewall running, as outlined in the initial setup guide, you will need to allow connections to Nginx. Nginx registers itself with ufw upon installation, so the procedure is rather straightforward.

It is recommended that you enable the most restrictive profile that will still allow the traffic you want. Since you haven’t configured SSL for your server in this guide, you will only need to allow traffic on port 80.

Enable this by typing:

sudo ufw allow 'Nginx HTTP'

You can verify the change by running:

sudo ufw status

This command’s output will show that HTTP traffic is allowed:

OutputStatus: active

To                         Action      From
--                         ------      ----
OpenSSH                    ALLOW       Anywhere
Nginx HTTP                 ALLOW       Anywhere
OpenSSH (v6)               ALLOW       Anywhere (v6)
Nginx HTTP (v6)            ALLOW       Anywhere (v6)

With the new firewall rule added, you can test if the server is up and running by accessing your server’s domain name or public IP address in your web browser.

If you do not have a domain name pointed at your server and you do not know your server’s public IP address, you can find it by running the following command:

ip addr show eth0 | grep inet | awk '{ print $2; }' | sed 's/\/.*$//'

This will print out a few IP addresses. You can try each of them in turn in your web browser.

As an alternative, you can check which IP address is accessible, as viewed from other locations on the internet:

curl -4 icanhazip.com

Type the address that you receive in your web browser and it will take you to Nginx’s default landing page:

http://server_domain_or_IP

Nginx default page

If you see the above page, you have successfully installed Nginx.

Step 2 – Installing MySQL to Manage Site Data

Now that you have a web server, you need to install MySQL (a database management system) to store and manage the data for your site.

Install MySQL by typing:

sudo apt install mysql-server

The MySQL database software is now installed, but its configuration is not yet complete.

To secure the installation, MySQL comes with a script that will ask whether we want to modify some insecure defaults. Initiate the script by typing:

sudo mysql_secure_installation

This script will ask if you want to configure the VALIDATE PASSWORD PLUGIN.

Warning: Enabling this feature is something of a judgment call. If enabled, passwords which don’t match the specified criteria will be rejected by MySQL with an error. This will cause issues if you use a weak password in conjunction with software which automatically configures MySQL user credentials, such as the Ubuntu packages for phpMyAdmin. It is safe to leave validation disabled, but you should always use strong, unique passwords for database credentials.

Answer Y for yes, or anything else to continue without enabling.

VALIDATE PASSWORD PLUGIN can be used to test passwords
and improve security. It checks the strength of password
and allows the users to set only those passwords which are
secure enough. Would you like to setup VALIDATE PASSWORD plugin?

Press y|Y for Yes, any other key for No:

If you’ve enabled validation, the script will also ask you to select a level of password validation. Keep in mind that if you enter 2 – for the strongest level – you will receive errors when attempting to set any password which does not contain numbers, upper and lowercase letters, and special characters, or which is based on common dictionary words.

There are three levels of password validation policy:

LOW    Length >= 8
MEDIUM Length >= 8, numeric, mixed case, and special characters
STRONG Length >= 8, numeric, mixed case, special characters and dictionary                  file

Please enter 0 = LOW, 1 = MEDIUM and 2 = STRONG: 1

Next, you’ll be asked to submit and confirm a root password:

Please set the password for root here.

New password:

Re-enter new password:

For the rest of the questions, you should press Y and hit the ENTER key at each prompt. This will remove some anonymous users and the test database, disable remote root logins, and load these new rules so that MySQL immediately respects the changes we have made.

Note that in Ubuntu systems running MySQL 5.7 (and later versions), the root MySQL user is set to authenticate using the auth_socket plugin by default rather than with a password. This allows for some greater security and usability in many cases, but it can also complicate things when you need to allow an external program (e.g., phpMyAdmin) to access the user.

If using the auth_socket plugin to access MySQL fits with your workflow, you can proceed to Step 3. If, however, you prefer to use a password when connecting to MySQL as root, you will need to switch its authentication method from auth_socket to mysql_native_password. To do this, open up the MySQL prompt from your terminal:

sudo mysql

Next, check which authentication method each of your MySQL user accounts use with the following command:

SELECT user,authentication_string,plugin,host FROM mysql.user;

Output+------------------+-------------------------------------------+-----------------------+-----------+
| user             | authentication_string                     | plugin                | host      |
+------------------+-------------------------------------------+-----------------------+-----------+
| root             |                                           | auth_socket           | localhost |
| mysql.session    | *THISISNOTAVALIDPASSWORDTHATCANBEUSEDHERE | mysql_native_password | localhost |
| mysql.sys        | *THISISNOTAVALIDPASSWORDTHATCANBEUSEDHERE | mysql_native_password | localhost |
| debian-sys-maint | *CC744277A401A7D25BE1CA89AFF17BF607F876FF | mysql_native_password | localhost |
+------------------+-------------------------------------------+-----------------------+-----------+
4 rows in set (0.00 sec)

In this example, you can see that the root user does in fact authenticate using the auth_socket plugin. To configure the root account to authenticate with a password, run the following ALTER USER command. Be sure to change password to a strong password of your choosing:

ALTER USER 'root'@'localhost' IDENTIFIED WITH mysql_native_password BY 'password';

Then, run FLUSH PRIVILEGES which tells the server to reload the grant tables and put your new changes into effect:

FLUSH PRIVILEGES;

Check the authentication methods employed by each of your users again to confirm that root no longer authenticates using the auth_socket plugin:

SELECT user,authentication_string,plugin,host FROM mysql.user;

Output+------------------+-------------------------------------------+-----------------------+-----------+
| user             | authentication_string                     | plugin                | host      |
+------------------+-------------------------------------------+-----------------------+-----------+
| root             | *3636DACC8616D997782ADD0839F92C1571D6D78F | mysql_native_password | localhost |
| mysql.session    | *THISISNOTAVALIDPASSWORDTHATCANBEUSEDHERE | mysql_native_password | localhost |
| mysql.sys        | *THISISNOTAVALIDPASSWORDTHATCANBEUSEDHERE | mysql_native_password | localhost |
| debian-sys-maint | *CC744277A401A7D25BE1CA89AFF17BF607F876FF | mysql_native_password | localhost |
+------------------+-------------------------------------------+-----------------------+-----------+
4 rows in set (0.00 sec)

You can see in this example output that the root MySQL user now authenticates using a password. Once you confirm this on your own server, you can exit the MySQL shell:

exit

Note: After configuring your root MySQL user to authenticate with a password, you’ll no longer be able to access MySQL with the sudo mysql command used previously. Instead, you must run the following:

mysql -u root -p

After entering the password you just set, you will see the MySQL prompt.

At this point, your database system is now set up and you can move on to installing PHP.

Step 3 – Installing PHP and Configuring Nginx to Use the PHP Processor

You now have Nginx installed to serve your pages and MySQL installed to store and manage your data. However, you still don’t have anything that can generate dynamic content. This is where PHP comes into play.

Since Nginx does not contain native PHP processing like some other web servers, you will need to install php-fpm, which stands for “fastCGI process manager”. We will tell Nginx to pass PHP requests to this software for processing.

Note: Depending on your cloud provider, you may need to add Ubuntu’s universe repository, which includes free and open-source software maintained by the Ubuntu community, before installing the php-fpm package. You can do this by typing:

sudo add-apt-repository universe

Install the php-fpm module along with an additional helper package, php-mysql, which will allow PHP to communicate with your database backend. The installation will pull in the necessary PHP core files. Do this by typing:

sudo apt install php-fpm php-mysql

You now have all of the required LEMP stack components installed, but you still need to make a few configuration changes in order to tell Nginx to use the PHP processor for dynamic content.

This is done on the server block level (server blocks are similar to Apache’s virtual hosts). To do this, open a new server block configuration file within the /etc/nginx/sites-available/ directory. In this example, the new server block configuration file is named example.com, although you can name yours whatever you’d like:

sudo nano /etc/nginx/sites-available/example.com

By editing a new server block configuration file, rather than editing the default one, you’ll be able to easily restore the default configuration if you ever need to.

Add the following content, which was taken and slightly modified from the default server block configuration file, to your new server block configuration file:

server {
        listen 80;
        root /var/www/html;
        index index.php index.html index.htm index.nginx-debian.html;
        server_name example.com;

        location / {
                try_files $uri $uri/ =404;
        }

        location ~ \.php$ {
                include snippets/fastcgi-php.conf;
                fastcgi_pass unix:/var/run/php/php7.2-fpm.sock;
        }

        location ~ /\.ht {
                deny all;
        }
}

Here’s what each of these directives and location blocks do:

  • listen — Defines what port Nginx will listen on. In this case, it will listen on port 80, the default port for HTTP.
  • root — Defines the document root where the files served by the website are stored.
  • index — Configures Nginx to prioritize serving files named index.php when an index file is requested, if they’re available.
  • server_name — Defines which server block should be used for a given request to your server. Point this directive to your server’s domain name or public IP address.
  • location / — The first location block includes a try_files directive, which checks for the existence of files matching a URI request. If Nginx cannot find the appropriate file, it will return a 404 error.
  • location ~ \.php$ — This location block handles the actual PHP processing by pointing Nginx to the fastcgi-php.conf configuration file and the php7.2-fpm.sock file, which declares what socket is associated with php-fpm.
  • location ~ /\.ht — The last location block deals with .htaccess files, which Nginx does not process. By adding the deny all directive, if any .htaccess files happen to find their way into the document root they will not be served to visitors.

After adding this content, save and close the file. Enable your new server block by creating a symbolic link from your new server block configuration file (in the /etc/nginx/sites-available/ directory) to the /etc/nginx/sites-enabled/ directory:

sudo ln -s /etc/nginx/sites-available/example.com /etc/nginx/sites-enabled/

Then, unlink the default configuration file from the /sites-enabled/ directory:

sudo unlink /etc/nginx/sites-enabled/default

Note: If you ever need to restore the default configuration, you can do so by recreating the symbolic link, like this:

sudo ln -s /etc/nginx/sites-available/default /etc/nginx/sites-enabled/

Test your new configuration file for syntax errors by typing:

sudo nginx -t

If any errors are reported, go back and recheck your file before continuing.

When you are ready, reload Nginx to make the necessary changes:

sudo systemctl reload nginx

This concludes the installation and configuration of your LEMP stack. However, it’s prudent to confirm that all of the components can communicate with one another.

Step 4 – Creating a PHP File to Test Configuration

Your LEMP stack should now be completely set up. You can test it to validate that Nginx can correctly hand .php files off to the PHP processor.

To do this, use your text editor to create a test PHP file called info.php in your document root:

sudo nano /var/www/html/info.php

Enter the following lines into the new file. This is valid PHP code that will return information about your server:

<?php
phpinfo();

When you are finished, save and close the file.

Now, you can visit this page in your web browser by visiting your server’s domain name or public IP address followed by /info.php:

http://your_server_domain_or_IP/info.php

You should see a web page that has been generated by PHP with information about your server:

PHP page info

If you see a page that looks like this, you’ve set up PHP processing with Nginx successfully.

After verifying that Nginx renders the page correctly, it’s best to remove the file you created as it can actually give unauthorized users some hints about your configuration that may help them try to break in. You can always regenerate this file if you need it later.

For now, remove the file by typing:

sudo rm /var/www/html/info.php

With that, you now have a fully-configured and functioning LEMP stack on your Ubuntu 18.04 server.

Conclusion

A LEMP stack is a powerful platform that will allow you to set up and serve nearly any website or application from your server.

How to install NodeJs, MongoDB on Ubuntu 18.04

How to install NodeJs, MongoDB on Ubuntu 18.04

In this article, you'll learn how to install NodeJs, MongoDB on Ubuntu 18.04

In this article, you'll learn how to install NodeJs, MongoDB on Ubuntu 18.04

Node.js is a platform built on Chrome’s JavaScript runtime for scalable network applications and easily building and fast building applications.

The latest version node js PPA is maintaining by its official website and we can add this PPA to your Ubuntu systems and install node js on ubuntu with easy commands.

Node js package is available in the current release and LTS release. So, It is your choice to select which version you want to install on the system as per your requirements.

To install Nodejs on Ubuntu, let’s add the PPA to your system.

$ sudo apt-get install curl python-software-properties

$ curl -sL https://deb.nodesource.com/setup_12.x | sudo -E bash –

Now you successfully added node js PPA in your Ubuntu system.

Now execute the below command to install Node on and Ubuntu using apt-get.

This will also install NPM with Node.js.

This command also installs many other dependent packages on your Ubuntu system.

$ sudo apt-get install nodejs

After installation please verify your versions by following commands.

$ node -v

v12.1.0

And also check npm version

$ npm -v

v6.9.0

Now you successfully installed node js in your machine. Now you can test your node.js by creating Hello World text.

Create a file with servertext.js and write the following code in it.

var http = require(‘http’);

http.createServer(function (req, res) {

 res.writeHead(200, {‘Content-Type’: ‘text/plain’});

 res.end(‘Hello World\n’);

}).listen(4000, “127.0.0.1”);

console.log(‘Server running at ‘)

Install MongoDB on Ubuntu

MongoDB is a NoSQL database that offers high performance, high availability and automatic scaling of the enterprise level database.

MongoDB is a NoSQL database, so you cannot use SQL to retrieve and insert data, and it does not store data in tables like MySQL.

The data is stored in a “document” structure in JSON format (called BSON in MongoDB). MongoDB introduced in 2009 and is currently being developed by MongoDB Inc.

MongoDB only offers packages for 64-bit LTS Ubuntu versions. For example 16.04 LTS (xenial), 14.04 LTS (trusty), 18.04 LTS (bionic) and so on.

Now, I will install MongoDB 4.0 on Ubuntu 18.04 LTS.

GPG keys of the software are required for the Ubuntu package manager apt (Advanced Package Tool) to ensure the authenticity and consistency of the package.

Execute the following command to import MongoDB keys to your server.

$ sudo apt-key adv –keyserver hkp://keyserver.ubuntu.com:80 –recv 68818C72E52529D4

Create a source list for MongoDB by the following command.

$ sudo echo “deb http://repo.mongodb.org/apt/ubuntu bionic/mongodb-org/4.0 multiverse” | sudo tee /etc/apt/sources.list.d/mongodb-org-4.0.list

Now update the repository with the following apt command

 $ sudo apt-get update

Install MongoDB in Ubuntu using the following command

$ sudo apt-get install -y mongodb-org

Now start MongoDB and add it as a service and it started when the system is boot.

$ sudo systemctl start mongodb

$ sudo systemctl enable mongodb

$ sudo systemctl stop mongodb

Now check your MongoDB version using the following command

$ mongod –version

Node and MongoDB getting started:

Install the npm MongoDB package using the following command.

$ sudo npm install mongodb

const mongo = require(“mongodb”);

How to create a database?

To create a database in MongoDB, we can use MongoClient object and then specify the connection URL with IP address along with the database name as follows.

const MongoCli = require(‘mongodb’).MongoClient;
const url = “mongodb://127.0.0.1:27017/testdb”;

 
MongoCli.connect(url, (err, db) => {
 if (err) throw err;
 console.log(“Database created!”);
 db.close();
 });

How to create a collection?

use createCollection() to create collection.

const MongoCli = require(‘mongodb’).MongoClient;
const url = “mongodb://127.0.0.1:27017/”;

 
MongoCli.connect(url, (err, db) => {
 if (err) throw err;
 var dbo = db.db(“testdb”);
 dbo.createCollection(“users”, (err, res) => {
  if (err) throw err;
  console.log(“Collection created!”);
  db.close();
 });
 });

How to Insert record?

Insert a document into users collection. We can insert a record using **insertOne()**method.

const MongoCli = require(‘mongodb’).MongoClient;
const url = “mongodb://localhost:27017/”;

MongoCli.connect(url, (err, db) => {
 if (err) throw err;
 var dbo = db.db(“testdb”);
 var myobj = { name: “Stringify Inc”, address: “Road No 37” };
 dbo.collection(“users”).insertOne(myobj, (err, res) => {
  if (err) throw err;
  console.log(“1 document inserted”);
  db.close()
 });
 });

How to do an update?

We can update a record or document by using the updateOne() method.

const MongoCli = require(‘mongodb’).MongoClient;
const url = “mongodb://127.0.0.1:27017/”;

MongoCli.connect(url, (err, db) => {
 if (err) throw err;
 var dbo = db.db(“testdb”);
 var myquery = { address: “Road No 37” };
 var newvalues = { $set: {name: “Mickey”, address: “Canyon 123” } };
 dbo.collection(“users”).updateOne(myquery, newvalues, (err, res) => {
  if (err) throw err;
  console.log(“1 document updated”);
  db.close();
 });
 })

How to do delete?

We can delete a record or document by using deleteOne() method.

const MongoClient = require(‘mongodb’).MongoClient;
const url = “mongodb://localhost:27017/”;

MongoClient.connect(url, (err, db) => {
 if (err) throw err;
 var dbo = db.db(“testdb”);
 var myquery = { address: ‘Mountain 21’ };
 dbo.collection(“users”).deleteOne(myquery, (err, obj) => {
  if (err) throw err;
  console.log(“1 document deleted”);
  db.close();
 });
 });

How to do sort?

We can use sort() method to sort the result in ascending order or descending order. It will take one parameter and an object decide the sorting order.

const MongoCli = require(‘mongodb’).MongoClient;
const url = “mongodb://localhost:27017/”;

MongoClient.connect(url, (err, db) => {
 if (err) throw err;
 var dbo = db.db(“testdb”);
 var mysort = { name: 1 };
 dbo.collection(“users”).find().sort(mysort).toArray((err, result) => {
  if (err) throw err;
  console.log(result);
  db.close();
 });
 });

Below the code for descending order.

const MongoCli = require(‘mongodb’).MongoClient;
const url = “mongodb://localhost:27017/”;

MongoCli.connect(url, (err, db) => {
 if (err) throw err;
 var dbo = db.db(“testdb”);
 var mysort = { name: -1 };
 dbo.collection(“users”).find().sort(mysort).toArray((err, result) => {
  if (err) throw err;
  console.log(result);
  db.close();
 });
 });

How to do Limit?

We can do limit with** limit() **method.

const MongoCli = require(‘mongodb’).MongoClient;
const url = “mongodb://localhost:27017/”;

MongoCli.connect(url, (err, db) => {
 if (err) throw err;
 var dbo = db.db(“testdb”);
 dbo.collection(“users”).find().limit(5).toArray((err, result) => {
  if (err) throw err;
  console.log(result);
  db.close();
 });
 });

How to do Join?

MongoDB is not like a relational database, but we can do left outer join by using $lookup stage.

Let’s assume we have “orders” and “products” collections

const MongoCli = require(‘mongodb’).MongoClient;
const url = “mongodb://127.0.0.1:27017/”;

MongoClient.connect(url, (err, db) => {
 if (err) throw err;
 var dbo = db.db(“testdb”);
 dbo.collection(‘orders’).aggregate([
  { $lookup:
   {
    from: ‘products’,
    localField: ‘product_id’,
    foreignField: ‘_id’,
    as: ‘orderdetails’
   }
  }
  ]).toArray((err, res) => {
  if (err) throw err;
  console.log(JSON.stringify(res));
  db.close();
 });
 });