waqas bhatti / notes / python 2.7 and 3+ on older debians

Most of our servers have Debian 7, and thus come with Python 2.7. Some of our servers still have Debian 6, so don't have the newest Python, effectively crippling the newest versions of our photometry pipelines on these machines. To get around this, we decided to install the newest Python versions 2.7, and 3.4 (at the time) in /opt. This works very well, now that we've also migrated to using Python virtualenv for most purposes. The new Python wheel binary distribution format is also a nice perk, so we can quickly install most of the Python scientific stack without too much pain.

First, we install some required Debian packages.

[root@server:~]# apt-get install build-essential libsqlite3-dev zlib1g-dev
libncurses5-dev libgdbm-dev libbz2-dev libreadline5-dev libssl-dev
libdb-dev libmysqlclient-dev libblas-dev liblapack-dev libatlas-dev
libpng-dev curl libffi-dev

Then, download Python (whatever version you'd like from the Python website, compile, and install it to /opt. Below, we give an example of installing Python 2.7.8, which was current a couple of years ago. Python 3.5 follows a similar way of installation (replace /opt/python27 with /opt/python35).

[root@server:~]# tar xvf Python-2.7.8.tgz
[root@server:~]# cd Python-2.7.8
[root@server:~]# ./configure --prefix=/opt/python27
[root@server:~]# make -j 4
[root@server:~]# make install

After Python's installed, you can use the version of pip (the Python package manager) that comes with the distribution or bootstrap a virtualenv directly. See here for a newer perspective on this and more details, but a short version of bootstrapping a virtualenv for astronomy work from the downloaded version of the virtualenv package is given below. First, get virtualenv and unpack it somewhere, then do the following for Python 2.7:

[user@server:~]$ /opt/python27/bin/python /path/to/unzipped/virtualenv/virtualenv.py /path/to/new/venv

Python 3.5 comes with virtualenv and pip included as part of the distribution. To use it, do the following:

[user@server:~]$ /opt/python35/bin/python3 -m venv /path/to/new/venv

Now prepare the virtualenv and upgrade pip (the Python package installer):

[user@server:~]$ source /path/to/new/venv/bin/activate
(venv)[user@server:~]$ pip install pip --upgrade

Now that the virtualenv is ready, one can install the usual scientific packages and use them while the virtualenv is activated.

(venv)[user@server:~]$ pip install numpy scipy astropy matplotlib ipython[notebook]