Docker + Jupyter for Machine Learning

Docker + Jupyter for Machine Learning

Many data science applications require an isolated model training/development environment from your host environment. The lightweight solution for this would be to integrate Jupyter with Docker.

The best practice for setting up such a container is using a docker file, which I have written following the best practices in less than 1 minute. I hope this would help anyone engaging in data science applications with docker. The project is structured as follows

├── Prject_folder           
│ ├── Dockerfile           #Primary imga building 
│ ├── docker-compose.yml   #Describing the files to mount etc     
│ ├── requirements.txt     #Required python packages for the image

Step 1:

Download the following 3 scripts to your project directory on your computer. The GitHub repo can be found here.

FROM "ubuntu:bionic"

MAINTAINER [email protected]

RUN useradd -ms /bin/bash docker
RUN su docker

ENV LOG_DIR_DOCKER="/root/dockerLogs"
ENV LOG_INSTALL_DOCKER="/root/dockerLogs/install-logs.log"

RUN mkdir -p ${LOG_DIR_DOCKER} \
 && touch ${LOG_INSTALL_DOCKER}  \
 && echo "Logs directory and file created"  | sed -e "s/^/$(date +%Y%m%d-%H%M%S) :  /" 2>&1 | tee -a ${LOG_INSTALL_DOCKER}

RUN apt-get update | sed -e "s/^/$(date +%Y%m%d-%H%M%S) :  /" 2>&1 | tee -a ${LOG_INSTALL_DOCKER} \
  && apt-get install -y python3-pip python3-dev | sed -e "s/^/$(date +%Y%m%d-%H%M%S) :  /" 2>&1 | tee -a ${LOG_INSTALL_DOCKER} \
  && ln -s /usr/bin/python3 /usr/local/bin/python | sed -e "s/^/$(date +%Y%m%d-%H%M%S) :  /" 2>&1 | tee -a ${LOG_INSTALL_DOCKER} \
  && pip3 install --upgrade pip | sed -e "s/^/$(date +%Y%m%d-%H%M%S) :  /" 2>&1 | tee -a ${LOG_INSTALL_DOCKER}

COPY requirements.txt /root/datascience/requirements.txt
WORKDIR /root/datascience
RUN pip3 install -r requirements.txt

RUN jupyter notebook --generate-config --allow-root
RUN echo "c.NotebookApp.password = u'sha1:6a3f528eec40:6e896b6e4828f525a6e20e5411cd1c8075d68619'" >> /root/.jupyter/jupyter_notebook_config.py

CMD ["jupyter", "notebook", "--allow-root", "--notebook-dir=.", "--ip=0.0.0.0", "--port=8888", "--no-browser"]

Dockerfile

object_detection:
  image: datascience
  container_name: datascience
  restart: always
  environment:
     - TERM=xterm
  hostname: '127.0.0.1'
  ports:
     - "8888:8888"         #JupyterNB

  volumes:
     - /Users/chamalgomes/Documents/Python/GitHub/medium/JupyterWithDocker/test.txt:/root/datascience/test.txt

docker-compose.yml

Update it as {absolutePATH_to_yourFile}/{fileNameame}:/root/datascience/{fileName} to mount you preferred files.

tensorflow==1.14.0 
jupyter==1.0.0

requirements.txt

Update it to include the packages you require i.e matplotlib==1.0.0

Step 2:

Install Docker on your computer https://docs.docker.com/install/,

Step 3:

docker build -t datascience .

Step 4:

Update the docker-compose.yml file to mount files you require from your host to the container following the example I have given already with the test.txt file.

Step 5:

docker-compose up 

DONE Now Visit http://localhost:8888, The default password is set as “root”. Feel free to change it.

Addendum

If you run into trouble run the following command to download the log file to your current directory. Post this as a response to this article and I will get back to with a solution as soon as possible.

docker cp <container-name>:/root/dockerLogs/install-logs.log .

If you want to start an interactive session inside the container type in the following command

docker exec -it <container name> /bin/bash

If you want delete the image completely from your computer

Step 1: Stop the container

docker container stop <container-id>

Step 2: Remove the container

docker container rm <container-id>

Step 3: Delete the image

docker image rm datascience

Thanks for reading !

Originally published by Chamal Gomes at towardsdatascience.com

docker machine-learning

Bootstrap 5 Complete Course with Examples

Bootstrap 5 Tutorial - Bootstrap 5 Crash Course for Beginners

Nest.JS Tutorial for Beginners

Hello Vue 3: A First Look at Vue 3 and the Composition API

Building a simple Applications with Vue 3

Deno Crash Course: Explore Deno and Create a full REST API with Deno

How to Build a Real-time Chat App with Deno and WebSockets

Convert HTML to Markdown Online

HTML entity encoder decoder Online

What is Supervised Machine Learning

What is neuron analysis of a machine? Learn machine learning by designing Robotics algorithm. Click here for best machine learning course models with AI

Pros and Cons of Machine Learning Language

AI, Machine learning, as its title defines, is involved as a process to make the machine operate a task automatically to know more join CETPA

How To Get Started With Machine Learning With The Right Mindset

You got intrigued by the machine learning world and wanted to get started as soon as possible, read all the articles, watched all the videos, but still isn’t sure about where to start, welcome to the club.

What is Machine learning and Why is it Important?

Machine learning is quite an exciting field to study and rightly so. It is all around us in this modern world. From Facebook’s feed to Google Maps for navigation, machine learning finds its application in almost every aspect of our lives. It is quite frightening and interesting to think of how our lives would have been without the use of machine learning. That is why it becomes quite important to understand what is machine learning, its applications and importance.

Machine Learning Guide Full Book PDF

Machine Learning is an utilization of Artificial Intelligence (AI) that provides frameworks the capacity to naturally absorb and improve as a matter of fact without being expressly modified. AI centers round the improvement of PC programs which will get to information and use it learn for themselves.The way toward learning starts with perceptions or information, for instance , models, direct understanding, or guidance, so on look for designs in information and choose better choices afterward hooked in to the models that we give. The essential point is to allow the PCs adapt consequently without human intercession or help and modify activities as needs be.