Docker + Jupyter for Machine Learning

Docker + Jupyter for Machine Learning

Many data science applications require an isolated model training/development environment from your host environment. The lightweight solution for this would be to integrate Jupyter with Docker.

The best practice for setting up such a container is using a docker file, which I have written following the best practices in less than 1 minute. I hope this would help anyone engaging in data science applications with docker.
The project is structured as follows

├── Prject_folder           
│ ├── Dockerfile           #Primary imga building 
│ ├── docker-compose.yml   #Describing the files to mount etc     
│ ├── requirements.txt     #Required python packages for the image
Step 1:

Download the following 3 scripts to your project directory on your computer. The GitHub repo can be found here.

FROM "ubuntu:bionic"

MAINTAINER [email protected]

RUN useradd -ms /bin/bash docker
RUN su docker

ENV LOG_DIR_DOCKER="/root/dockerLogs"
ENV LOG_INSTALL_DOCKER="/root/dockerLogs/install-logs.log"

RUN mkdir -p ${LOG_DIR_DOCKER} \
 && touch ${LOG_INSTALL_DOCKER}  \
 && echo "Logs directory and file created"  | sed -e "s/^/$(date +%Y%m%d-%H%M%S) :  /" 2>&1 | tee -a ${LOG_INSTALL_DOCKER}

RUN apt-get update | sed -e "s/^/$(date +%Y%m%d-%H%M%S) :  /" 2>&1 | tee -a ${LOG_INSTALL_DOCKER} \
  && apt-get install -y python3-pip python3-dev | sed -e "s/^/$(date +%Y%m%d-%H%M%S) :  /" 2>&1 | tee -a ${LOG_INSTALL_DOCKER} \
  && ln -s /usr/bin/python3 /usr/local/bin/python | sed -e "s/^/$(date +%Y%m%d-%H%M%S) :  /" 2>&1 | tee -a ${LOG_INSTALL_DOCKER} \
  && pip3 install --upgrade pip | sed -e "s/^/$(date +%Y%m%d-%H%M%S) :  /" 2>&1 | tee -a ${LOG_INSTALL_DOCKER}

COPY requirements.txt /root/datascience/requirements.txt
WORKDIR /root/datascience
RUN pip3 install -r requirements.txt
 
RUN jupyter notebook --generate-config --allow-root
RUN echo "c.NotebookApp.password = u'sha1:6a3f528eec40:6e896b6e4828f525a6e20e5411cd1c8075d68619'" >> /root/.jupyter/jupyter_notebook_config.py

CMD ["jupyter", "notebook", "--allow-root", "--notebook-dir=.", "--ip=0.0.0.0", "--port=8888", "--no-browser"]

Dockerfile

object_detection:
  image: datascience
  container_name: datascience
  restart: always
  environment:
     - TERM=xterm
  hostname: '127.0.0.1'
  ports:
     - "8888:8888"         #JupyterNB

  volumes:
     - /Users/chamalgomes/Documents/Python/GitHub/medium/JupyterWithDocker/test.txt:/root/datascience/test.txt

docker-compose.yml

Update it as {absolutePATH_to_yourFile}/{fileNameame}:/root/datascience/{fileName} to mount you preferred files.

tensorflow==1.14.0 
jupyter==1.0.0

requirements.txt

Update it to include the packages you require i.e matplotlib==1.0.0

Step 2:

Install Docker on your computer https://docs.docker.com/install/,

Step 3:
docker build -t datascience .
Step 4:

Update the docker-compose.yml file to mount files you require from your host to the container following the example I have given already with the test.txt file.

Step 5:
docker-compose up 

DONE Now Visit http://localhost:8888, The default password is set as “root”. Feel free to change it.

Addendum

If you run into trouble run the following command to download the log file to your current directory. Post this as a response to this article and I will get back to with a solution as soon as possible.

docker cp <container-name>:/root/dockerLogs/install-logs.log .

If you want to start an interactive session inside the container type in the following command

docker exec -it <container name> /bin/bash

If you want delete the image completely from your computer

Step 1: Stop the container

docker container stop <container-id>

Step 2: Remove the container

docker container rm <container-id>

Step 3: Delete the image

docker image rm datascience

Thanks for reading !

Originally published by Chamal Gomes at towardsdatascience.com

Angular 9 Tutorial: Learn to Build a CRUD Angular App Quickly

What's new in Bootstrap 5 and when Bootstrap 5 release date?

What’s new in HTML6

How to Build Progressive Web Apps (PWA) using Angular 9

What is new features in Javascript ES2020 ECMAScript 2020

Using Docker and Kubernetes to Simplify Machine Learning

Using Docker and Kubernetes to Simplify Machine Learning

Using Docker and Kubernetes to Simplify Machine Learning: Managing the hardware, drivers, libraries and packages that make up a ML development environment can be hard. In this talk, I will introduce how Docker can be used to simplify the process of setting up a local ML development environment, and how we can use Kubernetes and Kubeflow to scale that standardised environment to provide scalable, web-based Jupyter environments for a large number of users, that can be served from both public cloud providers and from on-premise clusters.

Managing the hardware, drivers, libraries and packages that make up a ML development environment can be hard.

In this talk, I will introduce how Docker can be used to simplify the process of setting up a local ML development environment, and how we can use Kubernetes and Kubeflow to scale that standardised environment to provide scalable, web-based Jupyter environments for a large number of users, that can be served from both public cloud providers and from on-premise clusters.

Thanks for reading

If you liked this post, share it with all of your programming buddies!

Follow us on Facebook | Twitter

Further reading about Docker, Kubernetes and Machine Learning

Machine Learning A-Z™: Hands-On Python & R In Data Science

Python for Data Science and Machine Learning Bootcamp

Machine Learning, Data Science and Deep Learning with Python

Deep Learning A-Z™: Hands-On Artificial Neural Networks

Artificial Intelligence A-Z™: Learn How To Build An AI

A Complete Machine Learning Project Walk-Through in Python

Machine Learning In Node.js With TensorFlow.js

Docker for Absolute Beginners

An illustrated guide to Kubernetes Networking

Machine Learning Full Course - Learn Machine Learning

Machine Learning Full Course - Learn Machine Learning

This complete Machine Learning full course video covers all the topics that you need to know to become a master in the field of Machine Learning.

Machine Learning Full Course | Learn Machine Learning | Machine Learning Tutorial

It covers all the basics of Machine Learning (01:46), the different types of Machine Learning (18:32), and the various applications of Machine Learning used in different industries (04:54:48).This video will help you learn different Machine Learning algorithms in Python. Linear Regression, Logistic Regression (23:38), K Means Clustering (01:26:20), Decision Tree (02:15:15), and Support Vector Machines (03:48:31) are some of the important algorithms you will understand with a hands-on demo. Finally, you will see the essential skills required to become a Machine Learning Engineer (04:59:46) and come across a few important Machine Learning interview questions (05:09:03). Now, let's get started with Machine Learning.

Below topics are explained in this Machine Learning course for beginners:

  1. Basics of Machine Learning - 01:46

  2. Why Machine Learning - 09:18

  3. What is Machine Learning - 13:25

  4. Types of Machine Learning - 18:32

  5. Supervised Learning - 18:44

  6. Reinforcement Learning - 21:06

  7. Supervised VS Unsupervised - 22:26

  8. Linear Regression - 23:38

  9. Introduction to Machine Learning - 25:08

  10. Application of Linear Regression - 26:40

  11. Understanding Linear Regression - 27:19

  12. Regression Equation - 28:00

  13. Multiple Linear Regression - 35:57

  14. Logistic Regression - 55:45

  15. What is Logistic Regression - 56:04

  16. What is Linear Regression - 59:35

  17. Comparing Linear & Logistic Regression - 01:05:28

  18. What is K-Means Clustering - 01:26:20

  19. How does K-Means Clustering work - 01:38:00

  20. What is Decision Tree - 02:15:15

  21. How does Decision Tree work - 02:25:15 

  22. Random Forest Tutorial - 02:39:56

  23. Why Random Forest - 02:41:52

  24. What is Random Forest - 02:43:21

  25. How does Decision Tree work- 02:52:02

  26. K-Nearest Neighbors Algorithm Tutorial - 03:22:02

  27. Why KNN - 03:24:11

  28. What is KNN - 03:24:24

  29. How do we choose 'K' - 03:25:38

  30. When do we use KNN - 03:27:37

  31. Applications of Support Vector Machine - 03:48:31

  32. Why Support Vector Machine - 03:48:55

  33. What Support Vector Machine - 03:50:34

  34. Advantages of Support Vector Machine - 03:54:54

  35. What is Naive Bayes - 04:13:06

  36. Where is Naive Bayes used - 04:17:45

  37. Top 10 Application of Machine Learning - 04:54:48

  38. How to become a Machine Learning Engineer - 04:59:46

  39. Machine Learning Interview Questions - 05:09:03

Machine Learning Tutorial - Learn Machine Learning - Intellipaat

Machine Learning Tutorial - Learn Machine Learning - Intellipaat

This Machine Learning tutorial for beginners will enable you to learn Machine Learning algorithms with python examples. Become a pro in Machine Learning.

Mastering the Machine Learning Course would easily develop one's career. This is the reason why studying Machine Learning Tutorial becomes so important in the career of a particular student.
Making a part of the machine learning course would enact and studying the Machine Learning Tutorial would make one carve out a new niche.