Dylan North

Dylan North

1570182738

Python Django with Docker and Gitlab CI

Python Django with Docker and Gitlab CI - In this article i will describe how we set up Gitlab CI to run tests for Django project. But first couple of words about what tools we were using…

For a project I was specifically asked to build an API using Python Django. So, my first starting point was to google “django cookiecutter” which immediately brought me to this amazing cookiecutter project. What I am going to demonstrate here is how to quickly setup the project (for the sake of completeness) and use Gitlab Continuous Integration to automatically unit test, run linters, generate documentation, build a container and release it.

Setup the project

We start with initiating the project using the mentioned cookiecutter project,although you can also use another cookiecutter or build on your existing project; you probably need to make some alterations here and there. Here is a small list of prerequisites:

  • you have docker installed locally
  • you have Python installed locally
  • you have a Gitlab account and you can push using ssh keys

Now, install cookiecutter and generate the project:

pip install "cookiecutter>=1.4.0"
cookiecutter https://github.com/pydanny/cookiecutter-django

Provide the options in any way you like, so the Django project will be created. Type y when asked to include Docker (because that is why we are here!!!).

Walk trough the options for the Django Cookiecutter

Enter the project, create a git repo and push it there:

cd my_django_api
git init
git add .
git commit -m "first awesome commit"
git remote add origin git@gitlab.com:jwdobken/my-django-api.git
git push -u origin master

Obviously replace my-django-api with your project name and jwdobken with your own Gitlab account name.

You can read here how to develop running docker locally. It is something I do with all my projects of any type; the dev and production environments are more alike and it has been years since I worked with something like virtual environments and I am not missing it!

Add a test environment

Make a test environment by copying the local environment:

cp local.yml test.yml
cp requirements/local.txt requirements/test.txt
cp -r compose/local compose/test

In compose/test/django/Dockerfile change requirements/local.txt to requirements/test.txt . You can make more alterations to the test environment later.

The Gitlab-CI file

Finally we get to the meat. Here is the .gitlab-ci.yml file:

image: docker:latest
	services:
	  - docker:dind
	

	variables:
	  DOCKER_HOST: tcp://docker:2375
	  DOCKER_DRIVER: overlay2
	  CONTAINER_TEST_IMAGE: $CI_REGISTRY_IMAGE:$CI_BUILD_REF_SLUG
	  CONTAINER_RELEASE_IMAGE: $CI_REGISTRY_IMAGE:latest
	

	stages:
	  - test
	  - build
	  - release
	

	test:
	  stage: test
	  image: tiangolo/docker-with-compose
	  script:
	    - docker-compose -f test.yml build
	    # - docker-compose -f test.yml run --rm django pydocstyle
	    - docker-compose -f test.yml run --rm django flake8
	    - docker-compose -f test.yml run django coverage run -m pytest
	    - docker-compose -f local.yml run --rm django coverage html
	    - docker-compose -f local.yml run --rm django /bin/sh -c "cd docs && apk add make && make html"
	    - docker-compose -f local.yml run django coverage report
	  coverage: "/TOTAL.+ ([0-9]{1,3}%)/"
	  artifacts:
	    paths:
	      - htmlcov
	      - docs/_build
	    expire_in: 5 days
	

	build:
	  stage: build
	  script:
	    - docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN $CI_REGISTRY
	    - docker build -t $CONTAINER_TEST_IMAGE -f compose/production/django/Dockerfile .
	    - docker push $CONTAINER_TEST_IMAGE
	

	release:
	  stage: release
	  script:
	    - docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN $CI_REGISTRY
	    - docker pull $CONTAINER_TEST_IMAGE
	    - docker tag $CONTAINER_TEST_IMAGE $CONTAINER_RELEASE_IMAGE
	    - docker push $CONTAINER_RELEASE_IMAGE
	  only:
	    - master
	

	pages:
	  stage: release
	  script:
	    - mkdir -p public/coverage
	    - mv htmlcov/* public/coverage
	    - mkdir -p public/docs
	    - mv -v docs/_build/html/* public/docs
	  artifacts:
	    paths:
	      - public
	    expire_in: 30 days
	  only:
	    - master

The test stage builds the container stack in the test environment, runs the unit tests with flake8, copies the html coverage report and catches the total coverage. Also, we misuse the test build to generate the sphinxdocumentation for which we need to install Make.

The build stage builds the production container and pushes it to the Gitlab container registry.

The release stage pulls the build container and tags it as the latest release before pushing it to the container registry.

The page part publishes the test and documentation artifacts with Gitlab Pages.

Push your code to Gitlab where you should find a running pipeline.

the pipeline is running

the pipeline has succesfully finished

In the container registry of the project you can find two images: the latest master image and the latest release image. The page itself explains how to pull images from here to anywhere.

Badges

Gitlab enables badges on the repo page to give any specific information. On the Gitlab project page, go to Settings, go to Badges. Here you can add the following badges:

Pipeline status of the master branch:

Test coverage and report:

Documentation:

Note that the URL link of Gitlab Pages, for the test coverage report and documentation, is not straightforward. Replace your username with a groupname if you work in a group. In the case of a subgroup, provide the full path. Usually I end up with a bit of trial-and-error; this article explains most of it.

status badges shown on the repo page

Pydocstyle

Finally, I highly recommend to check the existence and quality of your docstrings using pydocstyle. Add the following line to requirements/test.txtand requirements/local.txt in the Code quality section:

pydocstyle==3.0.0  # https://github.com/PyCQA/pydocstyle

Add the following lines to setup.cfg to configure pydocstyle:

[pydocstyle]
match = (?!\d{4}_).*\.py

And finally add the following line to .gitlab-ci.yml in the script section of the test stage (just after the build):

- docker-compose -f test.yml run — rm django pydocstyle

Be warned that the project does not comply with pydocstyle by default, so you will have to complete the code with docstrings to pass the test again.

Finally

We now have a fresh Django project with a neat CI pipeline on Gitlab for automated unit tests, documentation and container image release. You can later include Continuous Deployment to the pipeline; I left it out of the scope, because it depends too much on your production environment. You can read more about Gitlab CI here.

Currently the pipeline is quite slow mainy caused by the build of the images. The running time can be accelerated by caching dependencies.

There is a soft (10GB) size restriction for registry on GitLab.com, as part of the repository size limit. Therefore, when the number of images increases, you probably need to archive old images manually.

===================================================================

Thanks for reading :heart: If you liked this post, share it with all of your programming buddies! Follow me on Facebook | Twitter

#python #django

What is GEEK

Buddha Community

Python Django with Docker and Gitlab CI

Hot to trigger build on production server after release stage?

Is it really necessary to use ‘cookiecutter’ ? Can you please elaborate on using it ?

Ahebwe  Oscar

Ahebwe Oscar

1620177818

Django admin full Customization step by step

Welcome to my blog , hey everyone in this article you learn how to customize the Django app and view in the article you will know how to register  and unregister  models from the admin view how to add filtering how to add a custom input field, and a button that triggers an action on all objects and even how to change the look of your app and page using the Django suit package let’s get started.

Database

Custom Titles of Django Admin

Exclude in Django Admin

Fields in Django Admin

#django #create super user django #customize django admin dashboard #django admin #django admin custom field display #django admin customization #django admin full customization #django admin interface #django admin register all models #django customization

Shardul Bhatt

Shardul Bhatt

1626775355

Why use Python for Software Development

No programming language is pretty much as diverse as Python. It enables building cutting edge applications effortlessly. Developers are as yet investigating the full capability of end-to-end Python development services in various areas. 

By areas, we mean FinTech, HealthTech, InsureTech, Cybersecurity, and that's just the beginning. These are New Economy areas, and Python has the ability to serve every one of them. The vast majority of them require massive computational abilities. Python's code is dynamic and powerful - equipped for taking care of the heavy traffic and substantial algorithmic capacities. 

Programming advancement is multidimensional today. Endeavor programming requires an intelligent application with AI and ML capacities. Shopper based applications require information examination to convey a superior client experience. Netflix, Trello, and Amazon are genuine instances of such applications. Python assists with building them effortlessly. 

5 Reasons to Utilize Python for Programming Web Apps 

Python can do such numerous things that developers can't discover enough reasons to admire it. Python application development isn't restricted to web and enterprise applications. It is exceptionally adaptable and superb for a wide range of uses.

Robust frameworks 

Python is known for its tools and frameworks. There's a structure for everything. Django is helpful for building web applications, venture applications, logical applications, and mathematical processing. Flask is another web improvement framework with no conditions. 

Web2Py, CherryPy, and Falcon offer incredible capabilities to customize Python development services. A large portion of them are open-source frameworks that allow quick turn of events. 

Simple to read and compose 

Python has an improved sentence structure - one that is like the English language. New engineers for Python can undoubtedly understand where they stand in the development process. The simplicity of composing allows quick application building. 

The motivation behind building Python, as said by its maker Guido Van Rossum, was to empower even beginner engineers to comprehend the programming language. The simple coding likewise permits developers to roll out speedy improvements without getting confused by pointless subtleties. 

Utilized by the best 

Alright - Python isn't simply one more programming language. It should have something, which is the reason the business giants use it. Furthermore, that too for different purposes. Developers at Google use Python to assemble framework organization systems, parallel information pusher, code audit, testing and QA, and substantially more. Netflix utilizes Python web development services for its recommendation algorithm and media player. 

Massive community support 

Python has a steadily developing community that offers enormous help. From amateurs to specialists, there's everybody. There are a lot of instructional exercises, documentation, and guides accessible for Python web development solutions. 

Today, numerous universities start with Python, adding to the quantity of individuals in the community. Frequently, Python designers team up on various tasks and help each other with algorithmic, utilitarian, and application critical thinking. 

Progressive applications 

Python is the greatest supporter of data science, Machine Learning, and Artificial Intelligence at any enterprise software development company. Its utilization cases in cutting edge applications are the most compelling motivation for its prosperity. Python is the second most well known tool after R for data analytics.

The simplicity of getting sorted out, overseeing, and visualizing information through unique libraries makes it ideal for data based applications. TensorFlow for neural networks and OpenCV for computer vision are two of Python's most well known use cases for Machine learning applications.

Summary

Thinking about the advances in programming and innovation, Python is a YES for an assorted scope of utilizations. Game development, web application development services, GUI advancement, ML and AI improvement, Enterprise and customer applications - every one of them uses Python to its full potential. 

The disadvantages of Python web improvement arrangements are regularly disregarded by developers and organizations because of the advantages it gives. They focus on quality over speed and performance over blunders. That is the reason it's a good idea to utilize Python for building the applications of the future.

#python development services #python development company #python app development #python development #python in web development #python software development

akshay L

akshay L

1572860016

Python Django Tutorial | Django Course

This Python Django tutorial will help you learn what is django web development & application, what is django and introduction to django framework, how to install django and start programming, how to create a django project and how to build django app. There is a short django project as well to master this python django framework.

Why should you watch this Django tutorial?

You can learn Django much faster than any other programming language and this Django tutorial helps you do just that. Our Django tutorial has been created with extensive inputs from the industry so that you can learn Django and apply it for real world scenarios.

#Python Django Tutorial #Django Course #Python Django Training #Python Django Course #intellipaat

Art  Lind

Art Lind

1602968400

Python Tricks Every Developer Should Know

Python is awesome, it’s one of the easiest languages with simple and intuitive syntax but wait, have you ever thought that there might ways to write your python code simpler?

In this tutorial, you’re going to learn a variety of Python tricks that you can use to write your Python code in a more readable and efficient way like a pro.

Let’s get started

Swapping value in Python

Instead of creating a temporary variable to hold the value of the one while swapping, you can do this instead

>>> FirstName = "kalebu"
>>> LastName = "Jordan"
>>> FirstName, LastName = LastName, FirstName 
>>> print(FirstName, LastName)
('Jordan', 'kalebu')

#python #python-programming #python3 #python-tutorials #learn-python #python-tips #python-skills #python-development

Art  Lind

Art Lind

1602666000

How to Remove all Duplicate Files on your Drive via Python

Today you’re going to learn how to use Python programming in a way that can ultimately save a lot of space on your drive by removing all the duplicates.

Intro

In many situations you may find yourself having duplicates files on your disk and but when it comes to tracking and checking them manually it can tedious.

Heres a solution

Instead of tracking throughout your disk to see if there is a duplicate, you can automate the process using coding, by writing a program to recursively track through the disk and remove all the found duplicates and that’s what this article is about.

But How do we do it?

If we were to read the whole file and then compare it to the rest of the files recursively through the given directory it will take a very long time, then how do we do it?

The answer is hashing, with hashing can generate a given string of letters and numbers which act as the identity of a given file and if we find any other file with the same identity we gonna delete it.

There’s a variety of hashing algorithms out there such as

  • md5
  • sha1
  • sha224, sha256, sha384 and sha512

#python-programming #python-tutorials #learn-python #python-project #python3 #python #python-skills #python-tips