Python 3.6.8 is installed on the cluster as the default version. In order to use it, you don’t need to load any modules. Specifically, for this Python version, some packages like
mpi4py are available if the corresponding modules are loaded as listed below:
The python version 3.11.2 can be used by loading the module
python. Note: This module
python has preinstalled compatible packages like
mpi4py, so you don’t need to load the modules of these packages separately, unlike the default version. Use this command to list pre-installed packages.
You can load
python module with the following command:
$ module load python
Loading Python 3.11.2 Environment
You can check the Python version with the following command:
$ python --version
You can also check the installation directory with the following command:
$ which python
Caution: For compatibility reasons, 2.7.18 is still available as part of the operating system. Python 2 has been deprecated since early 2020 and should not be used! All following explanations cover only Python 3 and may not be applicable to Python 2.
A good starting point for beginner Python programmers is for example the tutorials section of the Python website.
Note: Example commands starting with
$ refer to the Linux console, while commands starting with
>>> refer to the Python console. These characters need to be omitted when typing in a command.
There are additional Python installations that come as part of Miniconda and Jupyter (and Miniconda allows you to install additional Python versions). Use of those Python installations is explained on the Miniconda and Jupyter pages.
You can call the Python interactive console with the command
Python 3.6.8 (default, Apr 16 2020, 01:36:27)
[GCC 8.3.1 20191121 (Red Hat 8.3.1-5)] on linux
Type "help", "copyright", "credits" or "license" for more information.
python2 will call the Python2 console.
To execute a Python script, enter
$ python <scriptname>
You can also execute individual Python commands using the option
$ python -c "print('Hello world.'); print('Use semicolons to enter multiple commands.')"
Note that the entire command or sequence of commands is enclosed by quotation marks (either single or double quotes are possible).
The commands described here can of course also be used in shell script (e.g. job scripts).
Python code is organized into packages which contain modules. Each file with the suffix
.py is a module, each directory that contains a file named
__init__.py is a package. In the central Python Package Index (PyPI) there are a large number of packages available. Some of these are already installed on the cluster.
You can display all installed packages by opening the Python console and entering
Some packages that are commonly used in scientific programming are available by loading additional modules. Specifically, those are
mpi4py. You can display these packages with
module spider py3
Installing packages youself
Note: The package manager Conda, which can install a large number of Python packages, is installed on the OMNI cluster as well. If you use Conda or the Anaconda distribution already, you should use that instead. Anaconda on the OMNI cluster is described here.
If you need a package that is not installed, you can use the program pip3 to install it yourself. To do that, enter
$ pip3 install --user <package name>
The package and any packages it depends on will then be installed in your home directory, by default in
Packages in this directory will be found by Python immediately. Note that packages that were installed in this way are only available to you, not to other users. You can use a Python module or package within Python by importing it
>>> import examplemodule
>>> from examplepackage import examplemodule
Custom software and environment variables
If you want to run computations on the cluster with software developed by you (or manually downloaded by you through means other than Pip), you need to make sure that Python will find your software. A Python script that imports your software module
# Do stuff with mysoftware…
will crash with the following error message:
>>> import mysoftware
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named 'mysoftware'
That is because Python, when importing modules (or packages), only searches for the module name in predefined paths. You can see the list of these paths by entering in the Python console
>>>import sys; print(sys.path)
You can add a directory inside Python
but it is more useful to use the environment variable
PYTHONPATH by setting and exporting it at the beginning of your job script
$ export PYTHONPATH=$PYTHONPATH:/home/<YourUsername>/mysoftware
In both cases the addition to the search path is only temporary, namely as long as Python (in the case of
sys.path) or your terminal (in the case of
PYTHONPATH) is open. You can add paths permanently to
sys.path by adding a text file that ends in
.pth in a directory that is already on the path. In that file, you can add the new directories (one per line).
By default, the addition to the path is not recursive, i.e. even if you add a directory to
PYTHONPATH you cannot reach any Python modules in its subdirectories. However, if a subdirectory contains a file named
__init__.py, Python will treat the subdirectory as a package (see above) and you can import either the package or individual modules from it.