Knowledge Base

Python on the met-cluster

A range of Python environments are available on the met-cluster, all of which can be accessed with the ‘module‘ command. The most widely used and maintained distribution is Enthought Canopy, but we also have Anaconda Python Distributions and a ‘python.org’ distribution available. Note that the Python environment packaged with the operating system is also installed and this is the environment you get when you type ‘python’ with no python module loaded.

Python 3 is fundamentally different from, and not compatible with Python 2. Python 2.7 is still widely used. On the cluster, all the modules running Python 2.7 are grouped under ‘python/…’. The modules loading Python 3.5 environments are grouped under ‘python3.5/…’. Check with the command ‘module avail python’ or ‘module avail python3.5’ for the full list of options on the met-cluster.

Our default versions of Anaconda and Canopy have been installed with all packages included which were available from the distributor at the time of installation. It is not possible to install additional packages in those shared environments.

If you require a custom environment, with different versions of packages, or additional packages, you will need to create a private environment. Each of the Python distributions has a different way to create such private environments, e.g.  in Canopy 1.7 you can start from an empty virtual environment,  and in Canopy 2.1 you will need to  create a personal copy of the user environment.

 

Virtual Environment in Python Canopy 1.7

If you cannot find your desired package combination in any of the standard Python setups, you can install it in your own virtual environment. This is currently available for python/canopy-1.7.2 on the cluster.

Set up your virtual environment with the following commands. You will only need to do the initial steps once. After the initial setup, your environment will load automatically when you load the python/canopy-1.7.2-myenv module.

  • Load the canopy module:

  • Create your environment:

  • Switch to your new environment:

 

Once you have set this up, the next time you log in you can just start your environment with:

Here you start from an empty environment, and you will need to install all the required modules on your own.The easiest option is the enpkg command, which is native to Canopy. In order to use this command, you’ll need to register with Enthought. Once registered, you can simply download and install packages with:

If the package you require is not found via the ‘enpkg’ command, you can also try using ‘pip’:

If both of the above options can’t find what you’re looking for, it’s still possible to install your desired package by downloading it directly from the developer and following their installation instructions whilst the python/canopy-1.7.2-myenv module is loaded.

 

User Installation in Python Canopy 2.1

Before you begin, please note that at least 2GB of free space are required in your home directory for the Canopy Python user installation. It is not possible to install this in a different location. Home directories can easily fill up, causing various computing issues, so please be mindful of this when you choose to install your own Python environment.

The following command installs your personal Python environment from the Canopy 2.1.3 distribution. You will only need to run this once and afterwards your personal environment will be automatically called when you load the ‘python/canopy-2.1.3-userenv’ module.

The installation will take quite a while and end with displaying the Canopy GUI. From here, you can install additional packages in the ‘Package Manager’. In order to use this feature, you’ll need to register with Enthought.

From now on your own Python user environment can be simply loaded with:

If you don’t want to use the Canopy package manager to install packages, you can also use the native ‘edm’ command instead.

If the package you require is not found via the ‘edm’ command, you can also try using ‘pip’:

If both of the above options can’t find what you’re looking for, it’s still possible to install your desired package by downloading it directly from the developer and following their installation instructions whilst the python/canopy-2.1.3-userenv module is loaded.