If you have difficulty understanding virtualenv, one reason is it’s misnamed. I don’t see anything virtual about it. It would be more accurate to call it an isolated python environment, which is what I will be calling it in this article.
Installation
On Debian and Ubuntu, you install virtualenv with apt-get install python-virtualenv python3-virtualenv. On Windows, pip install virtualenv.
Python 3.3 and later has the functionality built-in, in the venv standard library module. However, virtualenv is still more popular and we will use that one here, because it also works on Python 2.7 and because it is supported by virtualenvwrapper (we will check it out below).
Basics
Fire up python and give it these commands:
import sys sys.path |
It will give you a list of directories. This is where it searches for modules when you ask it to import something. When you run python interactively the first item is the empty string, which denotes the current directory. On my system, the next item is /usr/lib/python3.4, and so on.
Now, let’s create an isolated python environment:
| Unix |
|
|
|---|---|---|
| Windows |
|
What the above command does is install a slightly modified python interpreter under /tmp/test1/bin (or C:\test1\Scripts). Run it with /tmp/test1/bin/python (or C:\test1\Scripts\python) and check the python path:
import sys sys.path |
As you will see, the path is different from before. Now most of the directories Python will search when you try to import something are under /tmp/test1 (or C:\test1). There are still a few system directories listed, but these are for the standard Python library. Any modules you have installed system-wide are not in the system path.
virtualenv also installs a modified version of pip under /tmp/test1/bin (or C:\test1\Scripts). If you run /tmp/test1/bin/pip install django (or C:\test1\Scripts\pip install django), this will install django somewhere inside /tmp/test1 (or C:\test1).
This is pretty much everything virtualenv does. It installs a modified Python interpreter that has a different python path; and it installs a modified version of pip that installs stuff in the isolated environment directory instead of system-wide.
If you have both Python 2 and Python 3 installed on your system, you can specify which one you want to use with the --python option; for example, in Debian/Ubuntu:
virtualenv --python=/usr/bin/python3 |
You can remove an isolated environment simply by deleting the directory and its contents.
Activating
It’s hard to type /tmp/test1/bin/python, so there is the activate script:
| Unix |
|
|
|---|---|---|
| Cmd |
|
|
| PowerShell * |
|
(* In PowerShell, you may first need to type Set-ExecutionPolicy Unrestricted to allow the execution of unsigned scripts.)
After you run this, if you type python, /tmp/test1/bin/python will run; same with pip.
This is accomplished by changing the shell path. The shell is the software that interprets your commands; for example, when you type python and press enter, something must get your keystrokes and decide to execute python; that something is the shell. If you are on Linux, your shell is probably “bash”; if you are on Windows, it’s “cmd” or the PowerShell. Just as the Python path is a list of directories where Python searches for modules, the shell path is a list of directories where the shell searches for executable files. When you type a command such as python, the shell looks in these directories in order to locate the executable. You can see the shell path with echo $PATH in Unix, echo %path% in Windows cmd, and $Env:path in PowerShell.
So, the only thing the activate script does is change the shell path. You can use deactivate to change it back to what it was.
Virtualenvwrapper
Isolated python environments are so cool that you will soon have too many. Virtualenvwrapper is a set of shell scripts that makes it much easier to manage these environments.
On Debian/Ubuntu, you can install it with apt-get install virtualenvwrapper. On Windows you need pip install virtualenvwrapper-win for cmd and pip install virtualenvwrapper-powershell for PowerShell; I’m not certain how well these are supported, however, especially the PowerShell version, which seems unmaintained since 2012. Note that you must run these pip commands system-wide, not in an isolated environment (i.e. you need to deactivate any isolated environment first).
mkvirtualenv test2 |
This creates and activates an isolated python environment called test2. It stores it somewhere, but the thing is, you don’t need to know where. In Debian/Ubuntu it’s usually in the .virtualenvs subdirectory of your home directory. You can use deactivate to deactivate it. Activate it again like this:
workon test2 |
You can check which virtualenvs you have like this:
lsvirtualenv |
Finally, remove a virtualenv like this:
rmvirtualenv test2 |
In production servers I use plain virtualenv; virtualenvwrapper is very handy in development.
System site packages
You can add the --system-site-packages option to virtualenv or mkvirtualenv, like this:
virtualenv --system-site-packages /tmp/test1 |
or
mkvirtualenv --system-site-packages test2 |
Now, as before, if you tell it pip install django, it will install django in the isolated environment. The difference is that the python path will also include everything that has been installed system-wide. So if you have installed, for example, requests, system-wide, it will use it (unless you also install requests in the virtualenv, in which case the latter will take precedence).
Many people prefer their isolated environments to be, well, isolated, so they don’t use this option, but I use it often in order to avoid compiling packages. pip install gdal can take a long time because it needs to compile stuff, whereas apt-get install python-gdal is way faster; so I use --system-site-packages in order to be able to use the system-wide gdal (this issue is becoming less important now that most Python packages are distributed compiled, as a Python wheel). In addition, psycopg2 can be tricky to compile, and it is usually better to use the operating system’s version, which is usually guaranteed to work with the operating system’s postgresql. In Windows, however, I normally don’t use --system-site-packages, as there are no site packages packaged with the operating system.
The “go to jail” image is © 2011 Ken Teegardin from seniorliving.org.