Pyenv and pipenv for the perfect Python projects

Pyenv and pipenv for the perfect Python projects

This is an opinionated way of developing with Python locally. Pyenv and pipenv for the perfect Python projects . You’ve probably discovered that it’s a pain in the ass to manage different projects with dependencies targeting different Python versions on your local machine.

To complicate things, there are multiple ways of installing Python too:

  • Preinstallation by the OS 😔
  • Using a package manager like brew or apt 😕
  • Using the binaries from www.python.org 😫
  • Using pyenv—easy way to install and manage Python installations 😎

This guide uses pyenv to manage Python installations, and Pipenv to manage project dependencies (instead of raw pip).

Installing pyenv

Let’s install via brew:

$ brew install pyenv

If you’re not on Mac, please see pyenv’s installation instructions.

Add the following to your ~/.bash_profile , or ~/.bashrc (depending on your shell) to automatically initialize pyenv when your terminal loads:

eval "$(pyenv init -)"

How does pyenv work?

See all available Python versions:

$ pyenv install --list

Let’s install Python 3.6.6

$ pyenv install 3.6.6
Installed Python-3.6.6 to /Users/dvf/.pyenv/versions/3.6.6

pyenv won’t change your global interpreter unless you tell it to:

$ python --version
Python 2.7.14
$ pyenv global 3.6.6
Python 3.6.6

pyenv allows you to install different versions of Python local to a directory. Let’s create a project targeting Python 3.7.0:

$ pyenv install 3.7.0
Installed Python-3.7.0 to /Users/dvf/.pyenv/versions/3.7.0
$ mkdir my_project && cd my_project
$ python --version
Python 3.6.6
$ pyenv local 3.7.0
$ python --version
Python 3.7.0

Now whenever you find yourself in my_project you’ll automatically use the Python 3.7.0 interpreter.

🤚Did that make sense? If not, stop here and take some time to play around with pyenv—it works by installing all Python interpreters in ~/.pyenv and dynamically adjusting your $PATH depending on your current directory.

What is Pipenv and how does it work?

Pipenv is the officially recommended way of managing project dependencies. Instead of having a requirements.txt file in your project, and managing virtualenvs, you'll now have a Pipfile in your project that does all this stuff automatically.

Start off by installing it via pip, it’s a rapidly evolving project so make sure you have the latest version (2018.10.13 at the time of writing):

$ pip install -U pipenv

Using Pipenv for the first time

Let’s set up Pipenv in your project:

$ cd my_project
$ pipenv install
Creating a virtualenv for this project…
Pipfile: /Users/dvf/my_project/Pipfile
Using /Users/dvf/.pyenv/versions/3.7.0/bin/python3.7 (3.7.0) to create virtualenv…

You’ll find two new files in your project: Pipfile and Pipfile.lock.

If you’re installing in a pre-existing project, Pipenv will convert your old requirements.txt into a Pipfile. How cool is that?

This is what your Pipfile should look like for a fresh project:

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"
[packages]
[dev-packages]
[requires]
python_version = "3.7"

Notice that we didn’t activate any virtual environments here, Pipenv takes care of virtual environments for us. So, installing new dependencies is simple:

$ pipenv install django
Installing django
...
Installing collected packages: pytz, django
Successfully installed django-2.1.2 pytz-2018.5
Adding django to Pipfile's [packages]…
Pipfile.lock (4f9dd2) out of date, updating to (a65489)…
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
Updated Pipfile.lock (4f9dd2)!
Installing dependencies from Pipfile.lock (4f9dd2)…
🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 2/2 — 00:00:01
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.

If you inspect your Pipfile you’ll notice it now contains django = "*" as a dependency.

If we wanted to install dev dependencies for use during development, for example YAPF, you’d add --dev to the install step:

$ pipenv install --dev yapf

What is **Pipfile.lock**?

Pipfile.lock is super important because it does two things:

  1. Provides good security by keeping a hash of each package installed.
  2. Pins the versions of all dependencies and sub-dependencies, giving you replicable environments.

Let’s see what it currently looks like:

{
    "_meta": {
        "hash": {
            "sha256": "627ef89...64f9dd2"
        },
        "pipfile-spec": 6,
        "requires": {
            "python_version": "3.7"
        },
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.org/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {
        "django": {
            "hashes": [
                "sha256:acdcc1...ab5bb3",
                "sha256:efbcad...d16b45"
            ],
            "index": "pypi",
            "version": "==2.1.2"
        },
        "pytz": {
            "hashes": [
                "sha256:a061aa...669053",
                "sha256:ffb9ef...2bf277"
            ],
            "version": "==2018.5"
        }
    },
    "develop": {}
}

Notice that the versions of each dependency are pinned. Without a very good reason, you would always want this file committed to your source control.

Custom Indexes

Until Pipenv it was difficult to use private Python repositories, for example if you’d like to host private Python libraries within your organization. Now all you need to do is define them as an additional sources in the Pipfile:

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[[source]]
url = "https://www.example.com"
verify_ssl = true
name = "some-repo-name"

[packages]
django = "*"
my-private-app = {version="*", index="some-repo-name"}

[dev-packages]

[requires]
python_version = "3.7"

Notice that we told my-private-app to use the private repo. If omitted, Pipenv will cycle through indexes until it finds the package.

💡Pipenv will also consume any environment variables in values, which is useful if you have sensitive credentials you don’t want sitting in source control (this was my contribution _</humblebrag>_)

Deploying

When deploying it’s important that your deploy fails if there’s a mismatch between installed dependencies and the Pipfile.lock. So you should append --deploy to your install step which does just that:

$ pipenv install --deploy

You could also check which dependencies are mismatched:

$ pipenv check

And see which sub-dependencies are installed by packages:

$ pipenv graph --reverse
pip==18.1
pytz==2018.5
  - Django==2.1.2 [requires: pytz]
setuptools==40.4.3
wheel==0.32.2
yapf==0.24.0

Once-off commands, scripts and activating venvs

If you’re actively developing a project, it’s helpful to activate the virtual environment:

$ pipenv shell
Launching subshell in virtual environment…
(my_project) ➜ my_project

Or, if you’d like to execute a command inside the venv:

$ pipenv run python manage.py runserver

You can also add scripts to Pipfile similar to npm package.json:

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"
[packages]
django = "*"
[dev-packages]
yapf = "*"
[scripts]
server = "python manage.py runserver"
[requires]
python_version = "3.7"

Now you can execute the script:

$ pipenv run server

We’ve just touched the tip of the iceberg. If you’ve like to learn more about Pipenv, I encourage you to read the great documentation.

Further reading:

Making Python code run at massive scale in the cloud

Pointers in Python: What's the Point?

Top Python IDEs for Data Science in 2019

Top 19+ Best IDEs for Web Development as of 2019

Learning Model Building in Scikit-learn : A Python Machine Learning Library

Socket Programming in Python: Client, Server, and Peer Examples

python

Bootstrap 5 Complete Course with Examples

Bootstrap 5 Tutorial - Bootstrap 5 Crash Course for Beginners

Nest.JS Tutorial for Beginners

Hello Vue 3: A First Look at Vue 3 and the Composition API

Building a simple Applications with Vue 3

Deno Crash Course: Explore Deno and Create a full REST API with Deno

How to Build a Real-time Chat App with Deno and WebSockets

Convert HTML to Markdown Online

HTML entity encoder decoder Online

Basic Data Types in Python | Python Web Development For Beginners

In the programming world, Data types play an important role. Each Variable is stored in different data types and responsible for various functions. Python had two different objects, and They are mutable and immutable objects.

How To Compare Tesla and Ford Company By Using Magic Methods in Python

Magic Methods are the special methods which gives us the ability to access built in syntactical features such as ‘<’, ‘>’, ‘==’, ‘+’ etc.. You must have worked with such methods without knowing them to be as magic methods. Magic methods can be identified with their names which start with __ and ends with __ like __init__, __call__, __str__ etc. These methods are also called Dunder Methods, because of their name starting and ending with Double Underscore (Dunder).

Python Programming: A Beginner’s Guide

Python is an interpreted, high-level, powerful general-purpose programming language. You may ask, Python’s a snake right? and Why is this programming language named after it?

Hire Python Developers

Are you looking for experienced, reliable, and qualified Python developers? If yes, you have reached the right place. At **[HourlyDeveloper.io](https://hourlydeveloper.io/ "HourlyDeveloper.io")**, our full-stack Python development services...

Python any: How to Check If Element is Iterable or Not

Python any() function returns True if any element of an iterable is True otherwise any() function returns False. The syntax is any().