Setting up Virtual Environments in Python on Ubuntu

If you're a professional Python developer, rarely do you work on just one project at a time. Whilst you might be developing our brand new app, sometimes you'll still have to fix a bug in the legacy platform.

The legacy platform is likely to depend on different sets of packages and uses a different version of Python.

Having only one location where all your packages are stored means you can only have one development environment on your machine at any one time. It'll be very tedious to have to uninstall the newer version of Python and its packages, and re-install the older versions of them each time you want to work on the legacy platform.

Virtual Environments

That's where virtual environments comes in. Virtual environments allows you to specify and install specific versions of Python and packages within the virtual environment, and, more importantly, switch between these environments with a single command.

Creating a virtual environment

Virtual environments can be created with the virtualenv package:

$ pip install virtualenv

Python 3.3 introduced a new module - venv - that comes as part of the Python Standard Library, and is meant to replace virtualenv.

In Ubuntu, we can install it like so:

$ sudo apt-get install -y python3-venv

Use venv if you do not need to support Python 2; then, you can use python3 -m venv instead of virtualenv.

But since it's likely we'll be developing in Python 2, we'll use virtualenv going forward.

This provides us with the executable virtualenv, which we can use to create our virtual environments.

Behind the scenes, instead of installing packages in a shared directory, virtualenv installs packages in separate directories - one directory for each environment.

So first of all, we must specify which directory to place our environments in. In our case, we chose to create a new .environment directory under our home directory

$ cd
$ mkdir .environments

Typing cd without any arguments defaults to your home directory

Next, we'll create the actual virtual environment by calling virtualenv, passing in the name of our environment as the only argument.

$ cd .environments/
$ virtualenv legacy
New python executable in /home/daniel/.environments/legacy/bin/python  
Installing setuptools, pip, wheel...done.  

This will create many files and directories that are used to create the virtual environment.

$ tree -L 2
.
├── bin
│   ├── activate
│   ├── activate.csh
│   ├── activate.fish
│   ├── activate_this.py
│   ├── easy_install
│   ├── easy_install-2.7
│   ├── pip
│   ├── pip2
│   ├── pip2.7
│   ├── python
│   ├── python2 -> python
│   ├── python2.7 -> python
│   ├── python-config
│   └── wheel
├── include
│   └── python2.7 -> /usr/include/python2.7
├── lib
│   └── python2.7
├── local
│   ├── bin -> /home/daniel/.environments/legacy/bin
│   ├── include -> /home/daniel/.environments/legacy/include
│   └── lib -> /home/daniel/.environments/legacy/lib
└── pip-selfcheck.json

9 directories, 15 files  

These files and directories mimic the ones that serves Python normally.

The include directory points to /usr/include/, the local directory re-directs to the relevant directories.

$ which python2.7
/usr/bin/python2.7
$ which python
/usr/bin/python

But most importantly, under bin, new python and pip executables are created. When using this virtual environment, those are the executables you'll actually be running, not the ones stored under /usr/bin.

Selecting the Python version

By default, Ubuntu 16.04 comes with versions 2 and 3 of Python pre-installed.

$ python -V
Python 2.7.12  
$ python3 -V
Python 3.5.2  

When you created the virtual environment, it'll use the default version of Python, which, in our case, was 2.7.12.

We can specify a different version of Python; but before we do that, let's make sure our Python versions are up-to-date.

$ sudo apt-get update
$ sudo apt-get -y upgrade

Now, we can use the --python flag (or -p for short) to specify the path to the Python executable we want our virtual environment to copy and use.

$ virtualenv --python=/usr/bin/python3 current
Running virtualenv with interpreter /usr/bin/python3  
Using base prefix '/usr'  
New python executable in /home/daniel/.environments/current/bin/python3  
Also creating executable in /home/daniel/.environments/current/bin/python  
Installing setuptools, pip, wheel...done.  

Using a Virtual Environment

When we installed the virtualenv package, it also provided us with an activate and deactivate executable under bin, which activates and deactivates our virtual environment, respectively.

$ source ~/.environments/legacy/bin/activate
(legacy) $ # Now you can go work on your legacy project

When you run activate, all that does is change your $PATH variable to look for files inside the virtual environment first, i.e. the ones under bin, include, lib and local.

Now, when you install packages, it'll be stored under ENV/lib/pythonX.X/site-packages/ instead of the usual /usr/lib/pythonX.X/site-packages.

Once you're done with using this virtual environment, run deactivate.

(legacy) $ deactivate
$ # Back to normal

Removing a Virtual Environment

Because all the files associated with a virtual environment are all under one directory, you can simply remove the directory to delete the environment.

$ cd ~/.environments/
$ rm -rf legacy/

virtualenvwrapper

Whilst the virtualenv package paved the way for creating virtual environments, it is a little clunky - you need to remember where you place the environments and then run source bin/activate in the environment you want. Not a huge inconvenience, but improvements, in terms of user experience, can be made.

The virtualenvwrapper package does just that - it is a wrapper on top of virtualenv that provides executables to create, copy and delete virtual environments and activate them by name only.

Installation

First, let's install it.

$ pip install virtualenvwrapper

This will install virtualenvwrapper under /usr/local/bin/virtualenvwrapper.sh, so we need to run source to set it up.

$ export WORKON_HOME=~/.environments/
$ source /usr/local/bin/virtualenvwrapper.sh
virtualenvwrapper.user_scripts creating ...  
$ # now we can run mkvirtualenv

However, the next time we open a new terminal, mkvirtualenv will not be available, and we'd need to run the export and source commands again. To save us the hassle, we'd need to add two new entries into our ~/.bashrc to make these permanent.

Add these lines to the bottom of your ~/.bashrc file.

# For virtualenvwrapper
export WORKON_HOME=~/.environments  
source "/usr/local/bin/virtualenvwrapper.sh"  
Usage

Next, we'll create the virtual environment.

$ mkvirtualenv -p python3 experimental
(experimental) $ 

Like virtualenv, you can also create new environments with a specific version of python using mkvirtualenv -p python3 env

We already have the legacy and current environments; these will still work with virtualenvwrapper.

To switch between different environments, we can use the workon executable.

(experimental) $ workon current
(current) $

deactivate and deleting a virtual environment works the same way as with virtualenv.

For completeness's sake, here is a quick run-down of how to use venv, the Python 3.3+ replacement for virtualenv

Install
apt-get install python3-venv

Create
python3 -m venv ~/.environments/[ENV]

Activate
source ~/.environments/[ENV]/bin/activate

Deactivate
deactivate

Summary

  • Virtual environments allow you to create isolated development environment for code bases that uses different versions of Python and packages
  • virtualenv is a package that allows you to create virtual environments
  • virtualenvwrapper is a package that wraps around virtualenv to provide the mkvirtualenv and workon executable, to make managing virtual environments easier.

Now there's no excuse not to isolate your applications into different environments!

Daniel Li

Full-stack Web Developer in Hong Kong. Founder of Brew.

Hong Kong http://danyll.com