waqas bhatti / notes / setting up python for astronomy work on rhel6


Please note: we use the system-wide Python installation here; this is Python 2.6, which isn't supported by too many packages these days. Installing a newer version of Python is fairly straightforward. See instructions for Debian—for RHEL/CentOS, replace apt-get and *-dev with yum and *-devel packages for the various libraries.

First, make sure the EPEL repository is enabled for RHEL6/CentOS 6/Scientific Linux 6. If you're using Fedora, you can skip this step as the Fedora repositories contain everything you need. As root:

[root@machine ~]$ wget http://download.fedoraproject.org/pub/epel/6/i386/epel-release-6-5.noarch.rpm
[root@machine ~]$ yum localinstall epel-release-6-5.noarch.rpm

Then, as root, install all the build dependencies for the various packages:

[root@machine ~]$ yum-builddep numpy
[root@machine ~]$ yum-builddep scipy
[root@machine ~]$ yum-builddep pyfits
[root@machine ~]$ yum-builddep python-matplotlib
[root@machine ~]$ yum-builddep ipython

some optional packages to talk to databases, etc.:

[root@machine ~]$ yum-builddep python-psycopg2 (for PostgresQL)
[root@machine ~]$ yum install mysql-devel (for MySQL)

This will pull in some older versions of the packages we need, but that's ok; we'll be setting up a Python virtualenv with the latest versions. Now we need to grab a sufficiently new version of virtualenv. The Fedora/EPEL6 repositories have a decently up-to-date version that works fine. As root:

[root@machine ~]$ yum install python-virtualenv

Now we're set to build our Python astronomy virtual environment. Switch back to whatever user you'll be when working on astronomy stuff, and then do the following:

[user@machine ~]$ virtualenv astronomy <-- or whatever directory you want to work in
[user@machine ~]$ cd astronomy
[user@machine astronomy]$ source bin/activate
(astronomy)[user@machine astronomy]$ <-- note prompt changes to indicate active virtualenv

Installing the scientific packages

Now, we'll use pip to install the newest versions of the packages we need. Use the order below; Numpy should be compiled/installed before Scipy since its installation sets some build flags that Scipy relies on to compile correctly.

(astronomy)[user@machine astronomy]$ pip install -U numpy
(astronomy)[user@machine astronomy]$ pip install -U scipy
(astronomy)[user@machine astronomy]$ pip install -U matplotlib
(astronomy)[user@machine astronomy]$ pip install -U pyfits
(astronomy)[user@machine astronomy]$ pip install -U ipython

some optional packages:

(astronomy)[user@machine astronomy]$ pip install -U psycopg2 (for PostgresQL)
(astronomy)[user@machine astronomy]$ pip install -U MySQL-python (for MySQL)
(astronomy)[user@machine astronomy]$ pip install -U pyephem (for ephemeris calculations)
(astronomy)[user@machine astronomy]$ pip install -U pyraf (if you've installed IRAF already)

Dealing with Python code you create (or local Python modules)

The easiest way I've found to make your own code (or any external modules you have to install locally) work with this virtualenv setup is the following:

(astronomy)[user@machine astronomy]$ mkdir local external

(astronomy)[user@machine astronomy]$ cd local
(astronomy)[user@machine local]$ <-- put or symlink your single file Python libraries/scripts in here

                                  (you need to point the Python interpreter in
                                   the virtualenv to these files. to do this, we
                                   create a .pth file in the virtualenv's
                                   lib/pythonX.X/site-packages directory)

(astronomy)[user@machine local]$ echo '/home/user/astronomy/local' > \

(astronomy)[user@machine local]$ cd ../external

(astronomy)[user@machine external]$ <-- put your full-blown Python packages in here as separate directories

                                     (executing python setup.py install for an
                                      external package in this directory while
                                      the virtualenv is active will make it
                                      install to the appropriate bin/, lib/,
                                      etc. directories inside the
                                      virtualenv directory)

Your local Python libraries and scripts will be picked up by the Python interpreter and should be importable or runnable with no additional path trickery needed.

Using the virtualenv

Most useful python packages live at the Python Package Index. Use pip inside the astronomy virtualenv to install them.

Activate the astronomy virtualenv before you start work:

[user@machine ~]$ cd astronomy
[user@machine astronomy]$ source bin/activate
(astronomy)[user@machine astronomy]$ <-- note prompt changes to indicate active virtualenv

(work work work...)

(once you're done, deactivate the virtualenv)

(astronomy)[user@machine astronomy]$ deactivate <-- turns off the virtualenv when no longer needed
[user@machine astronomy]$ <-- prompt returns to normal

Here's a way to check if everything works correctly:

(astronomy)[user@machine astronomy]$ uname -r -v
Linux localhost 2.6.32-431. #1 SMP Wed Oct 23 15:22:49 EDT 2013
(astronomy)[user@machine astronomy]$ cat /etc/redhat-release
CentOS release 6.5 (Final)

(astronomy)[user@machine astronomy]$ ipython
Python 2.6.6 (r266:84292, Nov 22 2013, 12:11:10)
Type "copyright", "credits" or "license" for more information.

IPython 1.1.0 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

In [1]: import numpy, scipy, matplotlib, pyfits

In [2]: numpy.__version__, scipy.__version__, matplotlib.__version__, pyfits.__version__
Out[2]: ('1.8.0', '0.13.2', '1.3.1', '3.2.dev')