Wiss. Rechnen » Python

Caution: this page contains information about the HoRUS cluster! The corresponding page for the OMNI cluster can be found here.

Python 3.6.6 is installed on the cluster. As an operating system standard, Python 2.7.5 is also available. We recommend, if Python 2 is not explicitly necessary, to use Python 3. All following explanations will refer to Python 3 and may not be possible in Python 2.

A good starting point for beginner Python programmers is for example the tutorials section of the Python website.

Note: Example commands starting with $ refer to the Linux console, while commands starting with >>> refer to the Python console. These characters need to be omitted when typing in a command.

Calling Python

You can call the Python interactive console with the command python3:

$ python3
Python 3.6.6 (default, Mar 29 2019, 00:03:27)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-36)] on linux
Type "help", "copyright", "credits" or "license" for more information.

The command python will call the Python2 console.

To execute a Python script, enter

$ python3 <scriptname>

You can also execute individual Python commands using the option -c

$ python3 -c "print('Hello world.'); print('Use semicolons to enter multiple commands.')"

Note that the entire command or sequence of commands is enclosed by quotation marks (either single or double quotes are possible).

Of course, the commands described here can also be used in shell scripts (e.g. job scripts).

Python packages and package installation

Python code is organized into packages which contain modules. Each file with the suffix .py is a module, each directory that contains a file named __init__.py is a package. In the central Python Package Index (PyPI) there are a large number of packages available. Some of these are already installed on the cluster. You can display all installed packages by opening the python console and entering

>>> help("modules")

If you need a package that is not installed, you can use the program Pip to install it yourself. To do that, enter

$ pip3 install --user <Paketname>

The package and any packages it depends on will then be installed in your home directory, by default in


Packages in this directory will be found by Python imediately. Note that packages that were installed in this way are only available to you. You can use a Python module or package within Python by importing it

>>> import examplemodule

or alternatively

>>> from examplepackage import examplemodule

Custom software and environment variables

If you want to run computations on the cluster with software developed by you (or manually downloaded by you through means other than Pip), you need to make sure that Python will find your software. A Python script that imports your software module

import mysoftware

# Do stuff with mysoftware…

will crash with the following error message:

>>> import mysoftware 
Traceback (most recent call last): 
  File "<stdin>", line 1, in <module>
ImportError: No module named 'mysoftware'

That is because Python, when importing modules (or packages), only searches for the module name in predefined paths. You can see the list of these paths by entering in the Python console

>>> import sys; print(sys.path)

You can add a directory inside Python

>>> sys.path.append(/home/<YourUsername>/mysoftware)

but it is more useful to use the environment variable PYTHONPATH by setting and exporting it at the beginning of your job script

$ export PYTHONPATH=$PYTHONPATH:/home/<YourUsername>/mysoftware

In both cases the addition to the search path is only temporary, namely as long as Python (in the case of sys.path) or your terminal (in the case of PYTHONPATH) is open. You can add paths permanently to sys.path by adding a text file that ends in .pth in a directory that is already on the path. In that file, you can add the new directory paths (one per line).

By default, the addition to the path is not recursive, i.e. even if you add a directory to sys.path or PYTHONPATH you cannot reach any Python modules in its subdirectories. However, if a subdirectory contains a file named __init__.py, Python will treat the subdirectory as a package (see above) and you can import either the package or individual modules from it.

Aktualisiert um 12:10 am 14. August 2018 von Jan Philipp Stephan