Guide from a Python project to an open source package

Guide from a Python project to an open source package

Excited about your python project? Do you want to share Python open source projects for beginners? Packaging, publishing your python projects source code is a great way to get other people interesting python projects. Here is a simple step by step guide to build and publish your first python package

Excited about your python project? Do you want to share Python open source projects for beginners? Packaging, publishing your python projects source code is a great way to get other people interesting python projects. Here is a simple step by step guide to build and publish your first python package

We’ll describe all the steps we’ve been through to create Scitime, a package that gives an estimate of algorithm training times (you can learn more about Scitime here).

Note: We will assume that you already have a python projects source code in GitHub that you would like to package and publish.

Step 0: Have your project licensed

Before doing anything else, your project should have a license since it’s going to be open source. Depending on how you’d like your package to be used, many licenses are available. Some usual licenses for open source projects are MIT or BSD.

To add a license to your project, simply add a [LICENSE]( file to the repository root by following the steps described here.

Step 1: make your code publish-ready

There are a few preliminary things you need to do to have your Python project package-able:

  • Your project structure needs to be in place. Usually, the root of the repository contains a folder with the name of the repository — this is where the core project code should be. What’s outside of this folder is the extra code that’s necessary to run and build the package (tests, documentation, etc.)
  • The core folder should include one (or more) module(s) and an [****]( file which imports the classes/functions you’ll want the end user to have access to.
  • Ideally, a proper logging system (instead of prints) should be set by using the logging package.
  • Ideally, your core code should be organized in one or more classes.
from .estimate import Estimator
import logging

class LogMixin(object):
    def logger(self):
        name = '.'.join([self.__module__, self.__class__.__name__])
        FORMAT = '%(name)s:%(levelname)s:%(message)s'
        logging.basicConfig(format=FORMAT, level=logging.DEBUG)
        logger = logging.getLogger(name)
        return logger

Step 2: Create a with setuptools

Once your project has a set structure, you should then add a file at the root of the repository. This mostly helps to automate all the publishing and version maintaining processes. Here’s an example of what a should look like (source here).

from setuptools import setup
from os import path

DIR = path.dirname(path.abspath(__file__))
INSTALL_PACKAGES = open(path.join(DIR, 'requirements.txt')).read().splitlines()

with open(path.join(DIR, '')) as f:
    README =

    description="Training time estimator for scikit-learn algorithms",
    author='Gabriel Lerner & Nathan Toubiana',
    author_email='[email protected]',
    keywords=['machine-learning', 'scikit-learn', 'training-time'],
        # include json and pkl files
        '': ['*.json', 'models/*.pkl', 'models/*.json'],

A few notes:

  • If your package has dependencies, a clean way to handle these is to add them in the setup file, through the install_requires argument (if the list is long you can always point to a requirement.txt file as done above)
  • If you want metadata (from the repository) to be downloaded when anyone installs the package, you should add these through the package_data argument
  • More information about the setup() function can be found here

Note: steps 3 to 6 are optional (but highly recommended), however you can jump straight to step 7 if you want to publish your package right now.

Step 3: Set local tests and test coverage checks

If it is not already the case, at this point, your project should definitely have unit tests. Although there are many frameworks that can help you do that, one simple way of doing it is using pytest. All tests should be in a dedicated folder (named tests/ or testing/ for example). In that folder, put all the test files you need in order to cover as much of your core code as possible. Here’s an example of how to write a unit test. Here’s also one of our Scitime’s test files.

Once in place, you can run your tests locally by running python -m pytest from the repository root.

Once your tests are created, you should also be able to estimate the coverage. This is important as you want to maximize the amount of code that’s tested in your project (to reduce the amount of unexpected bugs).

A lot of frameworks are available for this as well, for Scitime we used codecov. You can decide on a threshold of minimum coverage allowed by creating a .codecov.yml file and also decide on what files to include in the coverage analysis by creating a .coveragerc file.

comment: false

        target: auto
        threshold: 10%
        target: auto
        threshold: 10%

Example of a .codecov.yml file

branch = True
source = scitime
include = */scitime/*
omit =

Example of a .coveragerc file

Step 4: Standardize syntax and code style

You’ll also need to make sure that your code follows the PEP8 guidelines (i.e has a standard style and that the syntax is correct). Again, a lot of tools can help here. We used flake8.

Step 5: Create a proper documentation

Now that your project has tests and a good to structure, it should be a good time to add a proper documentation. First thing is to have a nice readme file that will show up on your Github repo root. Once this is done, here are a few additional “nice to have”:

  • Pull Request & issue templates: when a new PR or issue is created, these files permit you to template the descriptions as you wish. Follow these steps for PRs and these steps for issues to create them. Here are Scitime’s PR template and issue template.
  • A contribution guide. This should be a short guide on how you want external users to contribute to your package. Here’s Scitime’s contribution guide.
  • A code of conduct (here’s Scitime’s).
  • Tags and a description (see screenshot below).
  • Badges in the readme file (here’s a nice article on how to set these up).

As the readme file should be quite synthesized, it’s also pretty standard to have a more in-depth documentation. You can use sphinx to do so, and then host the documentation on readthedocs (here’s Scitime’s example). The documentation related files are usually in a docs/ folder (as done for Scitime). Here’s a nice sphinx & readthedocs tutorial.

python project for beginners

Example of a repo that has tags and a description

Step 6: Set up continuous integration

At this point, your project is not far away from being ready to be published. However, it seems a little overwhelming to have to deal with updating the docs, running the tests and checking the style and coverage after each commit. Fortunately, Continuous Integration (CI) helps you with that. You can use webhooks to GitHub to automate all these things after each commit. Here’s the set of CI tools we used for Scitime:

  • For running tests, we used travis-ci and appveyor (for tests on windows platforms). For travis-ci, besides setting up the webhooks on the repository, you also have to create a .travis.yml file in which you can not only run tests but also upload updated coverage outputs and check style and format. Same goes for appveyor by creating a appveyor.yml file
  • Codecov and readthdocs also have dedicated webhooks
language: python
  - "3.6"
# command to install dependencies
  - pip install -r requirements.txt
  - pip install flake8
  - pip install pytest-cov
  - pip install codecov
# command to run tests
  - python -m pytest --cov=scitime
  - ./build_tools/
  - codecov 

Example of a .travis.yml file: note that here, for each commit, the tests are run along with the test coverage checks. But there’s also a flake8 check (the logic being defined in the file)



    - PYTHON: "C:\\Python36-x64"

  # We need wheel installed to build wheels
  - "%PYTHON%\\python.exe -m pip install -r requirements.txt"
  - "%PYTHON%\\python.exe -m pip install pytest==3.2.1"

build: off


  - "%PYTHON%\\python.exe -m pytest"

Example of a appveyor.yml file: here, we only run the tests

This should make the whole process of updating the repository a lot easier.

python projects source code

Example of commit history with integrated webhooks

Step 7: Create your first release and publication

At this point, your soon-to-be package should look similar to this:


It’s now ready to be published! The first thing to do is to create your first release on GitHub — this is to keep track of the state of your project at a given point in time, a release should be created each time the version changes. You can follow this to create the release.

Once done, the only thing left to do is to publish your package. The most common platforms to publish a python package are PyPI and Conda. We will describe how to publish in both:

  • For PyPI, you first need to create an account, and then follow these steps using twine. This should be fairly straightforward, and PyPI also provides a test environment that can be used right before the actual deploy. The latter basically consists of creating a source distribution (python sdist) and uploading it with twine (twine upload dist/*). Once done, there should be a PyPI page corresponding to your package (here’s Scitime’s) and anyone should be able to install your package running the pip command.
  • For conda, we recommend publishing your package through conda-forge which is a community that helps you publish and maintain your package through their conda channel. You can follow these steps to add your package to the community, you will then be added to the conda-forge GitHub organization and be able to maintain your package quite easily (here’s Scitime’s conda-forge page) — anyone should then be able to install your package running the conda command.

All done!

You should now have your package finally published and available to anyone! Although most of the work is done, you’ll still have to maintain your project as you need to make some updates: this basically means changing the versions each time you make significant changes, creating new releases and going through step 7 again.

Originally published on


Bootstrap 5 Complete Course with Examples

Bootstrap 5 Tutorial - Bootstrap 5 Crash Course for Beginners

Nest.JS Tutorial for Beginners

Hello Vue 3: A First Look at Vue 3 and the Composition API

Building a simple Applications with Vue 3

Deno Crash Course: Explore Deno and Create a full REST API with Deno

How to Build a Real-time Chat App with Deno and WebSockets

Convert HTML to Markdown Online

HTML entity encoder decoder Online

Basic Data Types in Python | Python Web Development For Beginners

In the programming world, Data types play an important role. Each Variable is stored in different data types and responsible for various functions. Python had two different objects, and They are mutable and immutable objects.

How To Compare Tesla and Ford Company By Using Magic Methods in Python

Magic Methods are the special methods which gives us the ability to access built in syntactical features such as ‘<’, ‘>’, ‘==’, ‘+’ etc.. You must have worked with such methods without knowing them to be as magic methods. Magic methods can be identified with their names which start with __ and ends with __ like __init__, __call__, __str__ etc. These methods are also called Dunder Methods, because of their name starting and ending with Double Underscore (Dunder).

Python Programming: A Beginner’s Guide

Python is an interpreted, high-level, powerful general-purpose programming language. You may ask, Python’s a snake right? and Why is this programming language named after it?

Hire Python Developers

Are you looking for experienced, reliable, and qualified Python developers? If yes, you have reached the right place. At **[]( "")**, our full-stack Python development services...

Python any: How to Check If Element is Iterable or Not

Python any() function returns True if any element of an iterable is True otherwise any() function returns False. The syntax is any().