Photo from Chile

Getting Started with virtualenv and virtualenvwrapper in Python

There are a couple of tools which can be extremely useful when developing with Python on your local system that I would encourage you to try. These are virtualenv and virtualenvwrapper, and this post will introduce you to them, including instructions for their installation and use.

What is virtualenv?

virtualenv is a tool that allows you to create isolated Python environments, which can be quite helpful when you have different projects with differing requirements. As this is the case with many Mozilla Web QA projects it can be indispensable when working on those.

What is virtualenvwrapper?

virtualenvwrapper is just that, a wrapper utility around virtualenv that makes it even easier to work with. I admit that I have never used virtualenv without virtualenvwrapper, and I do not intend to. For that reason this post will only cover working with virtualenv via virtualenvwrapper. Note also that virtualenvwrapper is a set of shell functions that are guaranteed to work in the following shells:

  • bash
  • ksh
  • zsh

It may run under other shells, and there is a Windows version available called virtualenvwrapper-win

Installation

Both virtualenv and virtualenvwrapper can be installed via pip. Install virtualenv first and then virtualenvwrapper. Use the following commands to install them:

view plain print about
1pip install virtualenv
2 pip install virtualenvwrapper

Configuration

In order to use virtualenvwrapper you should add two lines to your shell startup file (e.g., .bash_profile):

view plain print about
1export WORKON_HOME=$HOME/.virtualenvs
2 source /usr/local/bin/virtualenvwrapper.sh

The first line tells virtualenvwrapper where to store the virtualenvs that will be created and used. The example above stores them in a folder called .virtualenvs inside your home folder. The first line runs the shell script to set up the virtualenvwrapper commands and should point to the location where virtualenvwrapper was installed.

Using virtualenvwrapper to Manage and Work with Virtual Environments

There are a lot of commands available with virtualenvwrapper, all of which are well documented. In my experience, the following are the commands you will use most often:

  • mkvirtualenv - used to create a new virtual environment. When you create a new environment it automatically becomes the active environment.
  • rmvirtualenv - used to remove an existing virtual environment. The environment must be deactivated (see below) before it can be removed.
  • workon - used to activate a virtual environment. Will also list all existing virtual environments if no argument is passed.
  • deactivate - used to deactivate the currently active virtual environment. Note that workon will automatically deactivate the current environment before activating a new one.

Here's an example of how one might use virtualenvwrapper to set up and configure a virtual environment for running tests for a Mozilla Web QA project (marketplace-tests). This code assumes that you already have the Git repo cloned to your local machine and you have navigated to the root folder of the project.

view plain print about
1mkvirtualenv marketplace-tests
2 pip install -Ur requirements.txt

The first line will create and activate a new virtual environment for marketplace-tests, while the second line will install all of the required packages into this virtual environment. Now, whenever you want to work on marketplace-tests you can simply type the command:

view plain print about
1workon marketplace-tests

Using virtualenv and PyCharm

The instructions above are specific to using virtualenv when running Python from the command line. You can also use your virtual environments when running and debugging Python code from within PyCharm. First you have to add your virtual environments to PyCharm's list of Python interpreters:

  1. Open the Preferences (Settings on Windows) dialog and choose Project Interpreter.
  2. Click on the Configure Interpreters link, which will open up the Python Interpreters dialog. Here you will see all of the interpreters currently configured for PyCharm.
  3. Click the + button at the bottom of the list and choose the path to the interpreter in your virtual environment. In my experience sometimes PyCharm is able to find and list these automatically, and other times you have to choose the Local... item and browse for the path yourself. After choosing the path PyCharm will do some setup, displaying a progress indicator and will finally ask you whether you want to set this interpreter as Project Interpreter? Generally you are adding an interpreter precisely because you do want to set it as the interpreter for the current project, in which case you would answer Yes.
  4. Your interpreter will now appear in the list of Python interpreters.

If ever you want to change the interpreter for a given project just access the Project Interpreter dialog again and simply select a different interpreter from the Project Interpreter select list. This option is even available to be changed for individual run configurations, as described in the previous post on Setting Up PyCharm to Run MozWebQA Tests:

  1. Type ctrl + alt + R to open the Run configuration selector.
  2. Type 0, or choose the first option (Edit configurations...) from the select list.
  3. In the Run dialog, beside Interpreter > Python Interpreter, choose your interpreter from the list.
  4. Click Run.

Setting Up PyCharm to Run MozWebQA Tests

PyCharm is a Python IDE, released by JetBrains. I quite enjoy using it and, as I've recently started contributing some tests to Mozilla's Web QA department (mozwebqa), I wanted to use it to interactively debug some mozwebqa tests. This turned out to be a lot trickier than I had imagined, so I am documenting the steps to do so via this post.

This post assumes you have a project open in PyCharm for one of the mozwebqa projects. I am going to use marketplace-tests for this example. As I am on a Mac the instructions and screen shots will be OS X specific, but I imagine they will translate pretty closely to Windows.

Step 1 - Configure py.test as your default test runner

  1. Open up the Preferences... dialog and choose Python Integrated Tools under Project Settings.
  2. Choose py.test as the Default test runner.
  3. Click OK.

Step 2 - Set default run configuration parameters for your project

  1. Type ctrl + alt + R to open the Run configuration selector.
  2. Type 0, or choose the first option (Edit configurations...) from the select list.
  3. In the Run dialog, expand Defaults > Python's test and choose py.test.
  4. Make sure the Python interpreter that you want to use for this project is selected for Python interpreter. If you are using virtualenv you may have to configure a new Python interpreter for your virtualenv. More on that in a separate post.
  5. Choose the root of your project for Working directory.
  6. Click Apply, then Close.

Step 3 - Create a pytest.ini file in the root of your project

This is necessary to pass command line options to py.test. It would be nice if there were a way to do this via the IDE, but I was not able to do it. If anyone knows how, or can figure out how, to do that I'd love to know. I was able to pass a single option to the command line from PyCharm, but could not get it to work with multiple options.

  1. Create a file called pytest.ini in the root of your project.
  2. Add the command line options you need into that file under the [pytest] section. See below for an example from marketplace-tests.
  3. Click OK.
view plain print about
1[pytest]
2addopts = --driver=firefox --credentials=mine/credentials.yaml --destructive

Step 4 - Create a copy of credentials.yaml in a personal folder

You are going to have to edit credentials.yaml to place some actual credentials in it, so in order to not have it overwritten each time you do a pull, you should put a copy that you are going to use somewhere else. I tend to create a /mine folder under the project root and place it there, but you can put it anywhere you like. You will notice that the command line option above uses that /mine folder to locate the credentials file.

Step 5 - Run your tests

With a file that contains tests open in an editor window, type ctrl + shift + R and PyCharm will run all of the tests in the file. If you wish to run just one test, type ctrl + alt + R, followed by 0 to open the Edit configurations... dialog and then place the name of your test in the Keywords input. Click Run.