Caution: this page contains information about the OMNI cluster! The corresponding page for the HoRUS cluster can be found here.
Python 3.6.8 is installed on the cluster. Additionally, some packages like
mpi4py are available if the corresponding modules are loaded.
Caution: for compatibility reasons, 2.7.17 is still available as part of the operating system. Python 2 has been deprecated since early 2020 and should not be used! All following explanations cover only Python 3 and may not be applicable to Python 2.
A good starting point for beginner Python programmers is for example the tutorials section of the Python website.
Note: Example commands starting with
$ refer to the Linux console, while commands starting with
>>> refer to the Python console. These characters need to be omitted when typing in a command.
All information on this page refers to the default Python which is part of the operating system. This Python installation is available without loading a module. There are additional Python installations that come as part of Miniconda and Jupyter (and Miniconda allows you to install additional Python versions). Use of those Python installations is explained on the Miniconda and Jupyter pages.
You can call the Python interactive console with the command
$ python Python 3.6.8 (default, Apr 16 2020, 01:36:27) [GCC 8.3.1 20191121 (Red Hat 8.3.1-5)] on linux Type "help", "copyright", "credits" or "license" for more information. >>>
python2 will call the Python2 console.
To execute a Python script, enter
$ python <scriptname>
You can also execute individual Python commands using the option
$ python -c "print('Hello world.'); print('Use semicolons to enter multiple commands.')"
Note that the entire command or sequence of commands is enclosed by quotation marks (either single or double quotes are possible).
The commands described here can of course also be used in shell script (e.g. job scripts).
Python code is organized into packages which contain modules. Each file with the suffix
.py is a module, each directory that contains a file named
__init__.py is a package. In the central Python Package Index (PyPI) there are a large number of packages available. Some of these are already installed on the cluster.
You can display all installed packages by opening the Python console and entering
Some packages that are commonly used in scientific programming are available by loading additional modules. Specifically, those are
mpi4py. You can display these packages with
module spider py3
Installing packages youself
Note: the package manager Conda, which can install a large number of Python packages, is installed on the OMNI cluster as well. If you use Conda or the Anaconda distribution already, you should use that instead. Anaconda on the OMNI cluster is described here.
If you need a package that is not installed, you can use the program pip3 to install it yourself. To do that, enter
$ pip3 install --user <package name>
The package and any packages it depends on will then be installed in your home directory, by default in
Packages in this directory will be found by Python immediately. Note that packages that were installed in this way are only available to you, not to other users. You can use a Python module or package within Python by importing it
>>> import examplemodule
>>> from examplepackage import examplemodule
Custom software and environment variables
If you want to run computations on the cluster with software developed by you (or manually downloaded by you through means other than Pip), you need to make sure that Python will find your software. A Python script that imports your software module
runthis.py import mysoftware # Do stuff with mysoftware…
will crash with the following error message:
>>> import mysoftware Traceback (most recent call last): File "<stdin>", line 1, in <module> ImportError: No module named 'mysoftware'
That is because Python, when importing modules (or packages), only searches for the module name in predefined paths. You can see the list of these paths by entering in the Python console
>>>import sys; print(sys.path)
You can add a directory inside Python
but it is more useful to use the environment variable
PYTHONPATH by setting and exporting it at the beginning of your job script
$ export PYTHONPATH=$PYTHONPATH:/home/<YourUsername>/mysoftware
In both cases the addition to the search path is only temporary, namely as long as Python (in the case of
sys.path) or your terminal (in the case of
PYTHONPATH) is open. You can add paths permanently to
sys.path by adding a text file that ends in
.pth in a directory that is already on the path. In that file, you can add the new directories (one per line).
By default, the addition to the path is not recursive, i.e. even if you add a directory to
PYTHONPATH you cannot reach any Python modules in its subdirectories. However, if a subdirectory contains a file named
__init__.py, Python will treat the subdirectory as a package (see above) and you can import either the package or individual modules from it.