Suvash Thapaliya

~ 19 since 85 ~

Using Pythonbrew and Virtualenv(with pip) for creating sandboxed Python development environments.

As developers get started with a project, they prefer having total control over the development environment, as much as possible to perfectly match the deployment environment. The other strong requirement, especially in a team with mutiple developers would be to work on exactly replicated environments.

I have recently been looking into web development using Python(Django). Having some experience from Ruby/Rails development I have come to appreciate the bells and whistles that come along with it, some of which I believe raises the bar of productivity and helps developers get their work done in a better fashion. One of such is the proliferation of tools to create perfectly sandboxed Ruby development environments using RVM and Bundler.

Intended Audience

What follows below is my take on creating as perfect/sandboxed Python development environments as possible. This article assumes that you have some previous basic experience/understanding developing in Python. If not, you may want to go through this article by Mir Nazim to get a better understanding of the whole Python development ecosystem. In short, this article expects you to be familiar with tools such as pip and virtualenv. After you’re done with that feel free to jump back here.

One slight cautionary note would be that I work on an OSX machine, so the commands presented below might not exactly work on your enviroment. Please append “sudo” accordingly if you are on a Linux machine.

Let’s brew some Python(s) first

As mentioned in the Github project page,

pythonbrew is a program to automate the building and installation of Python in the users $HOME. pythonbrew is inspired by perlbrew and rvm.

Putting it simply, pythonbrew allows one to install(or brew) different versions of Python in the same machine in the user’s home directory, thus not requiring administrative privileges and also simplifying the removal process.

Installation of pythonbrew

The installation is pretty straightforward, one just has to execute the following in the shell as recommended.

curl -kL http://xrl.us/pythonbrewinstall | bash

pythonbrew then installs itself to ~/.pythonbrew. The following has to be added to your ~/.bashrc (or ~/.zshrc) to source the pythonbrew executable.

[[ -s $HOME/.pythonbrew/etc/bashrc ]] && source $HOME/.pythonbrew/etc/bashrc

One can also install pythonbrew to the non default location, by setting the PYTHONBREW_ROOT environment variable before installation

export PYTHONBREW_ROOT=/path/to/pythonbrew
curl -kLO http://xrl.us/pythonbrewinstall
chmod +x pythonbrewinstall
./pythonbrewinstall

To confirm that you succesfully installed pythonbrew, open a new shell/terminal session and check the pythonbrew version, which is successful should report you back accordingly.

pythonbrew --version

Next step : Brew some pythons

First, quickly query a list of what python versions are available to install

pythonbrew list -k

Select one of them and install it

pythonbrew install 2.7.2

Sometime the installation might not go as expected and throw errors(as it did on my machine), in such cases additional options might help(turning off the tests helped in my case)

pythonbrew install --verbose 2.7.2
pythonbrew install --force 2.7.2
pythonbrew install --no-test 2.7.2

More details for options can be seen on the Github project page

Switch to the python you want, or just use it temporarily

Switching to a python permanently or just using it temporarily (or even jumping back to the default system python) is pretty simple enough then.

# To switch permanently,
pythonbrew switch 2.7.2

# To use temporarily,
pythonbrew use 2.7.2

# To jump back to system python, just turn off pythonbrew
pythonbrew off

To get a better understanding, try the following after you switch/use a python

which python

Create some virtual environments (and replicate them)

Creating sandboxed environments is pretty straightforward now. pythonbrew uses the virtualenv wrapped around it’s own syntax to manage virtual/sandboxed environments.

Initialize the virtual environment first

# Initializes the virtualenv
pythonbrew venv init

Creating, using, deleting and listing all your virtual environments is quite simple now

# Create a virtual/sandboxed environment
pythonbrew venv create mycoolbundle

# Use it
pythonbrew venv use mycoolbundle

# Quitting a virtual environment to the return to the global environment
deactivate

# Delete it
pythonbrew venv delete mycoolbundle

# List all environment on your machine
pythonbrew venv list

Add your goodies to the right virtual environments

After you’ve installed pythonbrew and created the virtual environment for the project required, it’s time you install packages just specific for your project. Using pip is highly recommended and the preferred way to do this.

pip install Django
pip install simplejson
pip install ......

Freeze your environment to pass it around

Well, now that you have setup your clean environment from scratch you might want to pass it around to your fellow hackers. Doing is quite simple as well.

First, create the requirements file for your project(once you’ve switched to your project related virtual environment)

# List all the pip packages used in the virtual environment
pip freeze -l

# Dump it to a requirements file in the project folder
pip freeze -l > requirements.txt

It is highly recommended to create the requirements.txt file when working in a team of developers and check it into source control if being used.

Installing all the pip packages back from the requirements.txt file on a different virtual environment

# Install packages from requirements file
pip install -r requirements.txt

Okay ! But, I need more …

Well, I’m on a ride into the awesome Python community, and honestly speaking don’t know much about it. This post was more of an effort to bring together all the scattered knowledge out there that I needed, as I couldn’t find it all in one nice bunch.

As I progress along the way and learn more stuff, I’ll try to put it out here in digestable chunks. Until then, feel free to send me your comments below.

And if you liked the article,

Friends from HN, feel free to discuss it here

blog comments powered by Disqus

Hosted on GitHub