In this post, I’ll touch upon the basics of Python package installation and virtual environments. I will also compare two popular tools used to manage virtual environments.

When I started working with Python, it took me some time to understand these. Through this post I intend to help new Python developers understand these concepts and avoid common mistakes.

Note: All command line examples in this post are specific to MacOS. For Linux and WSL systems, the syntax varies depending on which distribution and shell you’ve installed.

Python package installer (PIP)

Understanding how PIP works is an important prerequisite to understanding the need of virtual environments. PIP is the package manager for Python. It lets you install, upgrade and uninstall Python packages.

The command pip install _docker_ installs the latest version of the _docker_ module, along with all of its dependencies.

A common way to install multiple packages is by specifying them in a requirements.txt file. Then use pip install -r requirements.txt to install the packages. Here’s an example requirements.txt file:

pyyaml>=5.1
cryptography
aioconsole==0.1.14
termcolor==1.1.0

Challenges with PIP

Dependency Conflicts

Using the above commands installs/upgrades packages for the system-wide Python interpreter. The versions of Python, PIP, and the Python packages that ship with your OS may be required for legacy software to function. It is not advisable to change/upgrade them as it can break compatibility with legacy software. See ‘Deprecations’ section here for reference.

Also, if you have multiple projects that have conflicting dependencies (using different versions of the same package), or a multi user system where each user is likely to be building different projects (like a CI/CD build server), doing system wide package installation is not really an option.

Non-deterministic builds

The current version(20.1.1) of PIP does **not **guarantee the installation order when installing multiple packages. Also, if two packages depend on different versions of the same (transitive) package, which version of the transitive package gets installed depends on the order in which the packages are installed. Hence, two people using the same requirements.txt file can still end up with different package versions getting installed.

Virtual environments

Virtual environments are used to isolate the Python interpreter and packages for each project. Each isolated environment is protected from dependency conflicts that could arise in a shared environment. For this reason, it is highly recommended to use virtual environments while working on Python projects.

Lifecycle of a virtual environment

Creating a virtual environment installs a new Python interpreter in its directory. Once created, it needs to be activated. This updates the environment variables to point to the Python interpreter and PIP within the virtual environment. All package installs/upgrades thereafter happen locally within the virtual environment. Once you are done with installing packages and testing/building/packaging your project, simply deactivate the virtual environment. This reverts the changes made to the environment variables when the environment was activated. Common practice is to have a different virtual environment for each of your Python projects.

#python #basics