Wiss. Rechnen » Miniconda
 

The Anaconda distribution is a collection of software for scientific purposes. It contains a Python installation, an R installation and the package manager Conda, which can be used to install additional Anaconda packages. Since the complete Anaconda distribution with all packages takes up a lot of storage space, there is a variant called Miniconda, which only contains Python, Conda and a few basic packages. Both variants are completely free and open source.

Miniconda is available on the OMNI cluster. It is based on Python 3.8.5 and Conda 4.9.2. In order to use Miniconda, the module miniconda3 needs to be loaded.

module load miniconda3

You can find the Conda documentation here. A complete list of available first-party Anaconda packages can be found here. Note that Conda also can also install third-party packages and can manage multiple sources (called “channels” in Conda). The most important community channel is conda-forge.

Setting up Conda

You typically use Conda with the command conda. For example, the command:

conda help

can display an overview of possible Conda commands. You can find a complete list in the Conda documentation here.

When you use Conda for the first time, you need to initialize it with conda init, and then close and reopen the shell (for example by disconnecting and reconnecting to the cluster). Here is an example output for the command:

[demo_user@hpc-login01 ~]$ conda init
no change     /cm/shared/omni/apps/miniconda3/condabin/conda
no change     /cm/shared/omni/apps/miniconda3/bin/conda
no change     /cm/shared/omni/apps/miniconda3/bin/conda-env
no change     /cm/shared/omni/apps/miniconda3/bin/activate
no change     /cm/shared/omni/apps/miniconda3/bin/deactivate
no change     /cm/shared/omni/apps/miniconda3/etc/profile.d/conda.sh
no change     /cm/shared/omni/apps/miniconda3/etc/fish/conf.d/conda.fish
no change     /cm/shared/omni/apps/miniconda3/shell/condabin/Conda.psm1
no change     /cm/shared/omni/apps/miniconda3/shell/condabin/conda-hook.ps1
no change     /cm/shared/omni/apps/miniconda3/lib/python3.8/site-packages/xontrib/conda.xsh
no change     /cm/shared/omni/apps/miniconda3/etc/profile.d/conda.csh
modified      /home/js056352/.bashrc

==> For changes to take effect, close and re-open your current shell. <==

(After logout and subsequent login...)

(base) [demo_user@hpc-login01 ~]$

You can see two things: first, conda init has modified the user’s .bashrc file (you can open the file with a text editor you can see that the PATH variable has been modified). Second, the command line now shows (base) on the left. This is the name of the active Conda environment. We will get into Conda environments in the next section.

Caution: there is a danger of unintended side effects if you load or unload the miniconda3 module, especially after a conda init, because the order of directories in your PATH variable will be switched. We recommend that you have the miniconda3 module permanently active whenever you use Miniconda. This can be accomplished for example by adding module load miniconda3 to your .bashrc before the conda init (see also Linux Basics).

On the cluster there are a number of different Python installations available. When the module miniconda3 is loaded, the python and python3 commands will direct to the default Python contained in Miniconda. You can also install your own Python, this will be explained in the next section.

You have other options for configuring Conda, those are explained here.

Conda Environments

Conda Environments, much like Python’s Virtual Environments, are designed to separate software for different projects. Conda Environments have the advantage of not being limited to Python. Caution: you must create at least one custom Conda Environment because the default environment (base) does not allow you to install packages. You can show all environments with conda env list:

$ conda env list

# conda environments:
#
base                  *  /cm/shared/omni/apps/miniconda3
test                     /home/demo_user/.conda/envs/test
                         /home/demo_user/miniconda3

As you can see in this example, the base environment is in the Conda installation directory where normal users do not have write permissions. Your custom environments on the other hand are in your home directory.

You can create your own environment with the command:

conda create --name <Envname>

You need to activate an environment before you can use it. It is also helpful to deactivate the environment after use. The commands are

conda activate <Envname>

and

conda deactivate

The name of the active environment is shown in brackets on the left of the command line. If you do not want any Conda Environment active, you have to type conda deactivate a second time.

More information about Conda Environments can be found here.

Installing packages

You can install Conda packages with the command:

conda install <package name>

As mentioned before, Conda packages will be installed in a subfolder of your home directory. If you get an error message about missing permissions, make sure you are not in the base environment.

Caution: if you want to install Python packages, you should use conda install over pip install whenever possible, since the latter may lead to conflicts.

We would also like to remind you that you can also install packages from third-party sources. Conda, like most package managers, has an option to add additional channels. An important unofficial channel is conda-forgewhich contains community packages. Installing unofficial packages is done at your own risk and should be done as little as possible.

More details on Conda packages can be found in the Conda documentation here, more about Conda channels here. A complete list of available Anaconda packages is here.

Aktualisiert um 15:11 am 8. February 2021 von Jan Steiner