Morioh
Login
Poppy Cooke

Poppy Cooke

5 years ago

Writing Python packages and deploying them to PyPI is one way of sharing libraries to the open-source community. For beginners, submitting a package to PyPI is not an easy task. This blog guides you through the complete process including testing and continuous integration and deployment (CI/CD).

Register first!

As a potential open-source software contributor, you need to create the following accounts.

PyPI

PyPi is the main package repository for Python. Register at https://pypi.org/.

Test PyPI

Test PyPI let’s you deploy test packages before deploying them to PyPi. Register at https://test.pypi.org/.

Github

Github is a code repository. You should already be familiar with git. Create an account at https://github.com/. You may choose other git repositories like BitBucket but they will not be covered on this blog.

Travis CI

Travis CI let’s you test software on different platforms. It is a full-pledged continuous integration and deployment (CI/CD) platform. Register at https://travis-ci.org/ using your Github account. You may choose to use other CI/CD platforms but they will not be covered on this blog.

CodeCov

CoveCov provides visualization for code coverage which helps you find untested code. Register at https://codecov.io/gh using your Github account.

Create a Github project

Go to https://github.com/new and create a project. Notice that I have enabled Initialize this repository with a README, used MIT License and integrated Travis CI.

Clone your repository

git clone git@github.com:roniemartinez/amortization.git
git checkout -b staging

Write your package

We will use our example from my previous blog, Calculating Amortization with Python. Below is the improved version (amortization.py). I have added a main() function to use our package from command line after installation using pip.

def calculate_amortization_amount(principal, interest_rate, period):
    """
    Calculates Amortization Amount per period

    :param principal: Principal amount
    :param interest_rate: Interest rate per period
    :param period: Total number of periods
    :return: Amortization amount per period
    """
    x = (1 + interest_rate) ** period
    return principal * (interest_rate * x) / (x - 1)


def amortization_schedule(principal, interest_rate, period):
    """
    Generates amortization schedule

    :param principal: Principal amount
    :param interest_rate: Interest rate per period
    :param period: Total number of periods
    :return: Rows containing period, interest, principal, balance, etc
    """
    amortization_amount = calculate_amortization_amount(principal, interest_rate, period)
    number = 1
    balance = principal
    while number <= period:
        interest = balance * interest_rate
        principal = amortization_amount - interest
        balance -= principal
        yield number, amortization_amount, interest, principal, balance if balance > 0 else 0
        number += 1


def main():  # pragma: no cover
    import argparse
    from tabulate import tabulate

    parser = argparse.ArgumentParser(
        description='Python library for calculating amortizations and generating amortization schedules')
    # required parameters
    required = parser.add_argument_group('required arguments')
    required.add_argument('-P', '--principal', dest='principal', type=float, required=True, help='Principal amount')
    required.add_argument('-n', '--period', dest='period', type=int, required=True, help='Total number of periods')
    required.add_argument('-r', '--interest-rate', dest='interest_rate', type=float, required=True,
                          help='Interest rate per period')
    # optional parameters
    parser.add_argument('-s', '--schedule', dest='schedule', default=False, action='store_true',
                        help='Generate amortization schedule')
    arguments = parser.parse_args()
    if arguments.schedule:
        table = (x for x in amortization_schedule(arguments.principal, arguments.interest_rate, arguments.period))
        print(
            tabulate(
                table,
                headers=["Number", "Amount", "Interest", "Principal", "Balance"],
                floatfmt=",.2f",
                numalign="right"
            )
        )
    else:
        amount = calculate_amortization_amount(arguments.principal, arguments.interest_rate, arguments.period)
        print("Amortization amount: {:,.2f}".format(amount))


if __name__ == '__main__':   # pragma: no cover
    main()

Write a setup.py

setup.py contains all the definitions related to our package.

from setuptools import setup

VERSION = '0.1.0'

setup(
    name='amortization',
    version=VERSION,
    py_modules=['amortization'],
    url='https://github.com/roniemartinez/amortization',
    download_url='https://github.com/roniemartinez/amortization/tarball/{}'.format(VERSION),
    license='MIT',
    author='Ronie Martinez',
    author_email='ronmarti18@gmail.com',
    description='Python library for calculating amortizations and generating amortization schedules',
    long_description=open('README.md').read(),
    long_description_content_type='text/markdown',
    install_requires=['tabulate==0.8.3'],
    entry_points={
        'console_scripts': ['amortization=amortization:main'],
    },
    classifiers=[
        'Development Status :: 4 - Beta',
        'License :: OSI Approved :: MIT License',
        'Topic :: Office/Business :: Financial',
        'Topic :: Scientific/Engineering :: Mathematics',
        'Topic :: Software Development :: Libraries :: Python Modules',
        'Programming Language :: Python :: 2',
        'Programming Language :: Python :: 2.7',
        'Programming Language :: Python :: 3',
        'Programming Language :: Python :: 3.5',
        'Programming Language :: Python :: 3.6',
        'Programming Language :: Python :: 3.7',
        'Programming Language :: Python :: Implementation :: CPython'
    ]
)

Here are the definitions:

name - name of library

version - version of library

py_modules - list of Python modules (.py) to include in our package. Use packages if we are including directory modules

url - package link, use Github repository link

download_url - link to download package aside from PyPI, use Github releases link

license - license of library

author - name of author

author_email - email of author

description - short description of library

long_description - long description of library, use README

long_description_content_type - tells PyPI the format of long description

install_requires - third party libraries needed

entry_points - used to define scripts

classifiers - this is use to classify our package, see https://pypi.org/pypi?%3Aaction=list_classifiers

Bonus: Write a setup.cfg

This is used to include other files to the resulting package and to define universal wheel.

[metadata]
description-file=README.md
license_files=LICENSE

[bdist_wheel]
universal = 1

Upload using twine

To install <a href="https://github.com/pypa/twine" target="_blank">twine</a>, run the following command:

pip install twine

Save your PyPI credentials

NOTE: Saving your credentials in plain text is not safe. Do not share them anywhere.
Save the following to $HOME/.pypirc:

[distutils]
index-servers=
    pypi
    testpypi

[pypi]
username=<username>
password=<password>

[testpypi]
repository=https://test.pypi.org/legacy/
username=<username>
password=<password>

Build your package

To build your package, run the following command

python setup.py sdist bdist_wheel

The following packages will be created. The first package amortization-<version>.tar.gz is the source distribution. The last package is a wheel distribution.

amortization-<version>.tar.gz
amortization-<version>-py2.py3-none-any.whl

Uploading

To upload, run the following command:

twine upload --repository testpypi dist/*  # uploads to Test PyPI
twine upload --repository pypi dist/*      # uploads to PyPI

Do not deploy untested code!

Writing code is one thing, deploying libaries for everyone is a different thing. You must test first before deployment. Below is the complete guide to automated testing and deployment.

Write your tests

We will use pytest package for testing. To install pytest, run the following command:

pip install pytest

First step is to create an empty conftest.py in the same directory where amortization.py is located. This will help pytest find our module. Create tests directory and write test_amortization.py containing the required tests.

from amortization import calculate_amortization_amount, amortization_schedule


def test_amortization_amount():
    principal = 150000
    period = 36
    interest_rate = 0.1
    amortization = principal * (interest_rate * (1 + interest_rate) ** period)/((1 + interest_rate) ** period - 1)
    assert calculate_amortization_amount(principal, interest_rate, period) == amortization


def test_amortization_schedule():
    principal = 150000
    period = 36
    interest_rate = 0.1

    amortization_amount = calculate_amortization_amount(principal, interest_rate, period)

    number = 1
    balance = principal

    for n, a, i, p, b in amortization_schedule(principal, interest_rate, period):
        interest = balance * interest_rate
        principal = amortization_amount - interest
        balance -= principal

        assert number == n
        assert amortization_amount == a
        assert interest == i
        assert principal == p
        assert balance == b

        number += 1

Running tests locally

To run tests, execute the following command.

pytest

Integrating with Travis CI

To define the jobs for Travis, write .travis.yml file. We will be testing on 4 different Python versions, 2.7, 3.5, 3.6, and 3.7. For managing dependencies, we will be using pipenv. See this Pipfile for the modules to be installed. We cannot deploy to PyPI on all these platforms, we will select 3.6 for deployment.

language: python

cache: pip

python:
  - 2.7
  - 3.5
  - 3.6

matrix:
  include:
  - python: 3.7
    dist: xenial
    sudo: true

branches:
  except:
    - /^[0-9]+\.[0-9]+\.[0-9]+/

install:
  - pip install pipenv
  - pipenv install --dev --skip-lock

script:
  pipenv run pytest --cov=amortization --cov-report=xml -v

after_success:
  pipenv run codecov

deploy:
  provider: pypi
  user: $TEST_PYPI_USER
  password: $TEST_PYPI_PASS
  server: https://test.pypi.org/legacy/
  distributions: "sdist bdist_wheel"
  on:
    branch: staging
    condition: $TRAVIS_PYTHON_VERSION = "3.6"

Notice the variables, $TEST_PYPI_USER and $TEST_PYPI_PASS.

Travis CI supports deployment to PyPI, however, it is not safe to save

the credentials to our code repository. We will need to save the

credentials to Travis Environment Variables. Go

to https://travis-ci.org///settings.

As our .travis.yml was only written for staging branch and to be deployed to Test PyPI, we need to modify it to work with master branch. From master branch, we will deploy to PyPI.

deploy:
  provider: pypi
  user: $PYPI_USER
  password: $PYPI_PASS
  distributions: "sdist bdist_wheel"
  on:
    branch: master
    condition: $TRAVIS_PYTHON_VERSION = "3.6"

Add project to CodeCov

To visualize code coverage, add Github project to CodeCov and obtain an upload token. Add CODECOV_TOKEN to Travis Environment Variables.

Better README

To add visualization about builds and versions and other information. Use shields provided by Travis CI, CodeCov, etc.

<table>
    <tr>
        <td>License</td>
        <td><img src='https://img.shields.io/pypi/l/amortization.svg'></td>
        <td>Version</td>
        <td><img src='https://img.shields.io/pypi/v/amortization.svg'></td>
    </tr>
    <tr>
        <td>Travis CI</td>
        <td><img src='https://travis-ci.org/roniemartinez/amortization.svg?branch=master'></td>
        <td>Coverage</td>
        <td><img src='https://codecov.io/gh/roniemartinez/amortization/branch/master/graph/badge.svg'></td>
    </tr>
    <tr>
        <td>Wheel</td>
        <td><img src='https://img.shields.io/pypi/wheel/amortization.svg'></td>
        <td>Implementation</td>
        <td><img src='https://img.shields.io/pypi/implementation/amortization.svg'></td>
    </tr>
    <tr>
        <td>Status</td>
        <td><img src='https://img.shields.io/pypi/status/amortization.svg'></td>
        <td>Downloads</td>
        <td><img src='https://img.shields.io/pypi/dm/amortization.svg'></td>
    </tr>
    <tr>
        <td>Supported versions</td>
        <td><img src='https://img.shields.io/pypi/pyversions/amortization.svg'></td>
    </tr>
</table>

These shields will result to a nice looking README.

Test and deploy without hassle

After completing all the above steps, deployment will be easier and faster. Just use git, update your code base and push.

git push

Installing our open-source library

We can now install and use our Python library easily.

pip install amortization

Since we added an entry point to our setup.py, we can call the package from command line.

amortization -P 150000 -n 36 -r 0.1 -s

Conclusion

Contributing libraries to PyPI is simple and easy. Open-source tools like Github and Travis CI, reduces time to production.

Source code for this project is available on Github.

#python

How to Submit a Package to PyPI

blog.easyaspy.org

How to Submit a Package to PyPI

Complete guide to Python package creation, automated testing and deployment

3 Likes16.75 GEEK