Dokku is a PaaS implementation built on docker, it can give you a Heroku-like environment
which is also open source and free.
You can use it on AWS EC2 or VPS such as Digital Ocean to save your time on many DevOps work.
In this Dokku tutorial, I will talk about how to deploy Django project with Dokku, it would use Postgres db and Amazon S3 to store data and media files.
After reaading this tutorial, you will get:
The source code of this post can be found on Django Dokku example
First, you can check Dokku doc for the latest system requirment. (you can ignore this step if you choose the Ubuntu LTS version)
Then you can get the install script in the Dokku release
In this tutorial, below is the script I use to install Dokku. Please make sure run this command on your server
# Please use ssh to login to your server
root@ubuntu:# wget https://raw.githubusercontent.com/dokku/dokku/v0.18.3/bootstrap.sh
root@ubuntu:# sudo DOKKU_TAG=v0.18.3 bash bootstrap.sh
The script would also instal Docker and other stuff on your server.
After Dokku is installed on your server, now you need config SSH public key for Dokku service.
Let’s say the IP of your server is 165.22.153.24, then you need to visit http://165.22.153.24 in your browser.
You need paste the ssh public key. (you can skip the hostname config here because we would do it in a bit)
After you are done, please check on your server to make sure the relevant service is not running (because this is risky)
root@ubuntu:# ps auxf | grep dokku-installer
root 14776 0.0 0.0 12944 920 pts/0 S+ 07:34 0:00 \_ grep --color=auto dokku-installer
The install service would be terminated after you config in web browser, as you can see in the output.
It is annoying to type the ip address to login each time, so we can config our ssh client.
Please do this in local env
You can add the server to your local ~/.ssh/config
to help you ssh in more easy way.
Please add code below to your ~/.ssh/config
Host dokku_server
ForwardAgent yes
Hostname 165.22.153.24
Port 22
ServerAliveInterval 60
ServerAliveCountMax 60
Now you can use ssh [[email protected]](/cdn-cgi/l/email-protection)_server
instead of ssh [[email protected]](/cdn-cgi/l/email-protection)
to login to your server
Now we start to create and config our Dokku app.
Please note that, the dokku project has name django_dokku_example
.
root@ubuntu:# dokku apps:create django_dokku_example
-----> Creating django_dokku_example... done
Here we put the Postgres db on our server, but you can also use 3-party DB server like Amazon RDS if you like.
Dokku has many plugins and here we use postgres plugin
to help us.
# install the plugin
root@ubuntu:# sudo dokku plugin:install https://github.com/dokku/dokku-postgres.git
# we can also specify the Postgres db version, if you do not specify here, the latest version would be used.
root@ubuntu:# export POSTGRES_IMAGE="postgres"
root@ubuntu:# export POSTGRES_IMAGE_VERSION="10.7"
# let's create db, I add suffix '_db'
root@ubuntu:# sudo dokku postgres:create django_dokku_example_db
Status: Downloaded newer image for postgres:10.7
docker.io/library/postgres:10.7
Waiting for container to be ready
Creating container database
Securing connection to database
=====> Postgres container created: django_dokku_example_db
=====> Container Information
Config dir: /var/lib/dokku/services/postgres/django_dokku_example_db/config
Data dir: /var/lib/dokku/services/postgres/django_dokku_example_db/data
Dsn: postgres://postgres:dc7f4aa4f2d4856068c88b4187afadbf@dokku-postgres-django-dokku-example-db:5432/django_dokku_example_db
Exposed ports: -
Id: a92fff08e4e36f60bb250bccc5816f576bc8a97b4398a0c6e228c24749ef1397
Internal ip: 172.17.0.2
Links: -
Service root: /var/lib/dokku/services/postgres/django_dokku_example_db
Status: running
Version: postgres:10.7
As you can see, POSTGRES_IMAGE
and POSTGRES_IMAGE_VERSION
is used to download the docker image. So you can use them to specify the Postgres db version you want to use.
Now link the db to your dokku app, and it would add a new env varialbe DATABASE_URL
to the dokku app.
root@ubuntu:~# dokku postgres:link django_dokku_example_db django_dokku_example
-----> Setting config vars
DATABASE_URL: postgres://postgres:dc7f4aa4f2d4856068c88b4187afadbf@dokku-postgres-django-dokku-example-db:5432/django_dokku_example_db
-----> Restarting app django_dokku_example
! App django_dokku_example has not been deployed
Let’s print out the env of our Dokku app.
root@ubuntu:~# dokku config django_dokku_example
=====> django_dokku_example env vars
DATABASE_URL: postgres://postgres:dc7f4aa4f2d4856068c88b4187afadbf@dokku-postgres-django-dokku-example-db:5432/django_dokku_example_db
dokku support store media files in local disk, you can check Dokku Persistent Storage for more detail
If you have no Amazon service account, please go to Amazon S3 and click the Get started with Amazon S3
to signup.
Login AWS Management Console
In the top right, click your company name and then click My Security Credentials
Click the Access Keys
section
Create New Access Key
, please copy the AMAZON_S3_KEY
and AMAZON_S3_SECRET
to notebook.
If you are new to Amazon and have no idea what is IAM
user, you can skip it and set permissions later.
Next, we start to create Amazon bucket on S3 Management Console, please copy Bucket name
to notebook.
Bucket
in Amazon S3 is like top-level container, every site should have its own bucket
, and the bucket name are unique across all Amazon s3, and the url of the media files have domain like {bucket_name}.s3.amazonaws.com
.
Now we add AWS_STORAGE_BUCKET_NAME
, AWS_ACCESS_KEY_ID
and AWS_SECRET_ACCESS_KEY
env variable to our Dokku app.
$ dokku config:set --no-restart django_dokku_example AWS_STORAGE_BUCKET_NAME='django-dokku-example'
$ dokku config:set --no-restart django_dokku_example AWS_ACCESS_KEY_ID=''
$ dokku config:set --no-restart django_dokku_example AWS_SECRET_ACCESS_KEY=''
By default, Dokku would restart the app if you changed env variable, here we use --no-restart
to tell Dokku not restart it.
There are still some env variables we need to config, please use dokku config:set --no-restart django_dokku_example
to add them to Dokku app.
DJANGO_ALLOWED_HOSTS: *
DJANGO_SECRET_KEY: {please generate new one}
DJANGO_SETTINGS_MODULE: config.settings.production
Here we set DJANGO_ALLOWED_HOSTS
to *
and later we would config nginx to make domain work.
Next, we start to config domain.
root@ubuntu:~# dokku domains:add django_dokku_example dokku.accordbox.com
-----> Added dokku.accordbox.com to django_dokku_example
! No web listeners specified for django_dokku_example
Here we add to dokku.accordbox.com
to our Dokku app django_dokku_example
, we can add more than one domain to Dokku app.
Now the Dokku app env is ready, before pushing code to Dokku server, let’s config our Django project.
You would also see we need to add some config files, which are very similar with Heroku’s config file, Dokku would scan and read them to decide some deployment workflow.
Now let’s config Django project to let it use Amazon s3.
# please remember to update requirements.txt or Pipfile
$ pip install boto3
$ pip install django-storages
Add storages
to INSTALLED_APPS
in settings/base.py
Add config below to settings/production.py
AWS_STORAGE_BUCKET_NAME = os.environ.get('AWS_STORAGE_BUCKET_NAME')
AWS_S3_CUSTOM_DOMAIN = '%s.s3.amazonaws.com' % AWS_STORAGE_BUCKET_NAME
AWS_ACCESS_KEY_ID = os.environ.get('AWS_ACCESS_KEY_ID')
AWS_SECRET_ACCESS_KEY = os.environ.get('AWS_SECRET_ACCESS_KEY')
AWS_S3_FILE_OVERWRITE = False
MEDIA_URL = "https://%s/" % AWS_S3_CUSTOM_DOMAIN
DEFAULT_FILE_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'
We use whitenoise
to help us serve static files on server.
# please remember to update requirements.txt or Pipfile
$ pip install whitenoise
Edit MIDDLEWARE
in settings/base.py
, put WhiteNoiseMiddleware
above all other middleware apart from Django’s SecurityMiddleware
:
MIDDLEWARE = [
'django.middleware.security.SecurityMiddleware',
'whitenoise.middleware.WhiteNoiseMiddleware',
# ...
]
Set in settings/production.py
STATICFILES_STORAGE = 'whitenoise.storage.CompressedManifestStaticFilesStorage'
Now we add code to settings/production.py
to make it can understand DATABASE_URL
value.
# please remember to update requirements.txt or Pipfile
$ pip install django-environ
$ pip install psycopg2
import environ
env = environ.Env()
if 'DATABASE_URL' in env:
DATABASES["default"] = env.db("DATABASE_URL") # noqa F405
DATABASES["default"]["ATOMIC_REQUESTS"] = True # noqa F405
DATABASES["default"]["CONN_MAX_AGE"] = env.int("CONN_MAX_AGE", default=60) # noqa F405
This file exists in root directory of our project. It contains entry command of our service.
web: gunicorn config.wsgi:application
release: python manage.py migrate --noinput
Here web
means the web service, Dokku would use the command to run for our web service.
And release
is the command which would be run in release stage, here we use the command to migrate our databae in release stage.
If you used Celery worker in your project, you can add worker: ...
to do that.
This file exists in root directory of our project, it defines container number
of our services.
web=1
Here we set web=1
so there would be one web container
Please make sure your project has requirements.txt
or Pipfile
so Dokku can check if it is Python project and would install dependencies when deploying.
Please commit code first and then keep reading.
First we add a remote branch dokku
to our Git repo.
$ git remote add dokku dokku@dokku_server:django_dokku_example
dokku_server
here can also be the ip address of the server (dokku_server is the hostname we config in ~/.ssh/config
), django_dokku_example
is the dokku app we just created on our server.
Then we push our code to remote branch
$ git push dokku master
Below is the output, Dokku would check language we use and download relevant Heroku buildpack to deploy the project. Which is very cool!
$ git push dokku master
Counting objects: 177, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (162/162), done.
Writing objects: 100% (177/177), 49.38 KiB | 3.53 MiB/s, done.
Total 177 (delta 45), reused 0 (delta 0)
-----> Cleaning up...
-----> Building django_dokku_example from herokuish...
-----> Adding BUILD_ENV to build environment...
-----> Warning: Multiple default buildpacks reported the ability to handle this app. The first buildpack in the list below will be used.
Detected buildpacks: multi python
-----> Multipack app detected
=====> Downloading Buildpack: https://github.com/heroku/heroku-buildpack-python.git
=====> Detected Framework: Python
-----> Installing python-3.6.9
-----> Installing pip
-----> Installing SQLite3
-----> Installing requirements with pip
Collecting pytz==2019.2 (from -r /tmp/build/requirements/./base.txt (line 1))
Downloading https://files.pythonhosted.org/packages/87/76/46d697698a143e05f77bec5a526bf4e56a0be61d63425b68f4ba553b51f2/pytz-2019.2-py2.py3-none-any.whl (508kB)
Successfully installed MarkupSafe-1.1.1 Pillow-6.1.0 Unidecode-1.1.1 Willow-1.1 argon2-cffi-19.1.0 beautifulsoup4-4.6.0 boto3-1.8.0 botocore-1.11.9 certifi-2019.9.11 cffi-1.12.3 chardet-3.0.4 coreapi-2.3.3 coreschema-0.0.4 defusedxml-0.6.0 django-2.2.5 django-allauth-0.40.0 django-anymail-6.1.0 django-crispy-forms-1.7.2 django-environ-0.4.5 django-model-utils-3.2.0 django-modelcluster-4.4 django-redis-4.10.0 django-storages-1.6.6 django-taggit-0.24.0 django-treebeard-4.3 djangorestframework-3.10.3 docutils-0.15.2 draftjs-exporter-2.1.7 gunicorn-19.9.0 html5lib-1.0.1 idna-2.8 itypes-1.1.0 jinja2-2.10.1 jmespath-0.9.4 oauthlib-3.1.0 psycopg2-2.8.3 pycparser-2.19 python-dateutil-2.8.0 python-slugify-3.0.4 python3-openid-3.1.0 pytz-2019.2 redis-3.3.8 requests-2.22.0 requests-oauthlib-1.2.0 s3transfer-0.1.13 six-1.12.0 sqlparse-0.3.0 text-unidecode-1.3 uritemplate-3.0.0 urllib3-1.25.6 wagtail-2.6.2 webencodings-0.5.1 whitenoise-4.1.3
-----> $ python manage.py collectstatic --noinput
288 static files copied to '/tmp/build/staticfiles', 840 post-processed.
Using release configuration from last framework (Python).
-----> Discovering process types
Procfile declares types -> release, web
-----> Releasing django_dokku_example (dokku/django_dokku_example:latest)...
-----> Deploying django_dokku_example (dokku/django_dokku_example:latest)...
! Release command declared: 'python manage.py migrate --noinput'
Operations to perform:
Apply all migrations: account, admin, auth, contenttypes, sessions, sites, socialaccount, taggit, users, wagtailadmin, wagtailcore, wagtaildocs, wagtailimages, wagtailsearch, wagtailusers
Running migrations:
Applying wagtailusers.0007_userprofile_current_time_zone... OK
Applying wagtailusers.0008_userprofile_avatar... OK
Applying wagtailusers.0009_userprofile_verbose_name_plural... OK
-----> App Procfile file found (/home/dokku/django_dokku_example/DOKKU_PROCFILE)
DOKKU_SCALE declares scale -> web=1
-----> Attempting pre-flight checks
For more efficient zero downtime deployments, create a file CHECKS.
See http://dokku.viewdocs.io/dokku/deployment/zero-downtime-deploys/ for examples
CHECKS file not found in container: Running simple container check...
-----> Waiting for 10 seconds ...
-----> Default container check successful!
-----> Running post-deploy
-----> Overriding default nginx.conf with detected nginx.conf.sigil
-----> Configuring dokku.accordbox.com...(using app-supplied template)
-----> Creating http nginx.conf
-----> Running nginx-pre-reload
Reloading nginx
-----> Renaming containers
Renaming container (47bec7f2e474) pensive_hoover to django_dokku_example.web.1
=====> Application deployed:
http://dokku.accordbox.com
To 165.22.153.24:django_dokku_example
* [new branch] master -> master
If the push command did not raise error, you should see something like this and now our app is live on http://dokku.accordbox.com
Let’s make https
work for our site.
$ dokku plugin:install https://github.com/dokku/dokku-letsencrypt.git
$ dokku config:set --no-restart --global DOKKU_LETSENCRYPT_EMAIL=michaelyin@accordbox.com
$ dokku letsencrypt django_dokku_example
# this would setup cron job to update letsencrypt certificate
$ dokku letsencrypt:cron-job --add
Please replace DOKKU_LETSENCRYPT_EMAIL
with your email.
After https
can work on your site, you can setup http->https
to improve security. Please check SECURITY
section of config/settings/production.py
for more detail.
It is common if your Django projects return 500 error page in some cases, so how to troubleshoot?
I strongly recommend to use Sentry
to help you solve this. And I already wrote post talking abou this Error log with Sentry
Or you can check the running logs using this command dokku logs django_dokku_example
Sometimes, we need to customize our Nginx settings.
For example, the default Nginx settings has small client_max_body_size
, so I need to change it to make upload feature work in my Django project.
Dokku
generate nginx config from nginx.conf.sigil
, you can first download it from Dokku repo (better download from specific Git tag instead of master branch).
And then you can add more modify your settings.
You can add that file to root directory of your project and Dokku would use it to generate new nginx config.
When Dokku deploy, it would start container which has latest code and then wait for 10 secs to make sure the service is ok to run.
To shorten the time, we can add add CHECKS
file and Dokku would use that file to check if our web server is ok to serve.
/ django_dokku_example
This tell Dokku to visit /
url and check if the response contains django_dokku_example
-----> Attempting pre-flight checks
-----> Attempt 1/5 Waiting for 5 seconds ...
CHECKS expected result:
http://localhost/ => "django_dokku_example"
-----> All checks successful!
=====> django_dokku_example web container output:
[2019-10-08 04:41:55 +0000] [9] [INFO] Starting gunicorn 19.9.0
[2019-10-08 04:41:55 +0000] [9] [INFO] Listening at: http://0.0.0.0:5000 (9)
[2019-10-08 04:41:55 +0000] [9] [INFO] Using worker: sync
[2019-10-08 04:41:55 +0000] [204] [INFO] Booting worker with pid: 204
172.17.0.1 - - [08/Oct/2019:04:42:04 +0000] "GET / HTTP/1.1" 200 3590 "-" "curl/7.47.0"
As you can see, Dokku use curl to check and if the check succeed, it would send traffic to the new container immediatly.
If you want to run some command on web server, for example, run Django shell, you can use command in this way
dokku --rm run django_dokku_example python manage.py shell
Here Dokku would create a new container for you to use and this can avoid some risky operation. --rm
means the container would be delted after you exit.
I always use this to run createsuperuser
If you want to setup cron jobs for your project, it is better to do it in host machine.
# we set crontjob as dokku user
$ crontab -u dokku -e
@daily dokku --rm run django_dokku_example python manage.py clearsessions
As you can see, we setup clearsessions
job to run each day and it can helps us clean out expired sessions in db.
Dokku is a very good option if you have limited budget but you still like the Heroku experience or you want to use Heroku on AWS EC2
There is no silver bullet in this world, so I will also talk about limitations of Dokku here.
Because of the design of Dokku, it is not easy to scale the app. So if your project needs HA (high availability), Dokku might not be a good choice here
In this Dokku tutorial, I showed you how to deploy Django project to Dokku.
The source code of this post can be found on Django Dokku example
What should you go next? There are still some funny things for you to check.
dokku postgres:backup
can be used to uppload db backup to AWS S3.
dokku ps:report
can give you app report and you can stop, scale and restart as you like.
If you have any question, please feel free to contact us. Thanks for reading!
#django #python #dokku #postgresdb