CSEWiki

A software environment in the context of computing on the cluster is just a set of variables that define the locations of programs, libraries, configuration files, data, or other information that a particular piece of software needs to run. Environments are usually defined by adjusting the environment variables of the login shell.

Different pieces of software (or different versions of the same software) can have different environment needs that may conflict with one another, which can cause errors or result in the software not functioning as expected – or at all. This has lead to a need for ways to manage one or more software environments.

To help minimize software environment conflicts, our clusters use the module command to alter the user's shell environment. This allows for reproducible environment changes that are easily loaded or unloaded at will.

When a user requests that a piece of software be made available to all users on the cluster, the sysadmins will often write an environment module to make it easily accessible for everyone with the module load command.

Typical module commands:

• module show - shows what modules you currently have loaded

• module avail - a list of ALL available modules

• module load <modulename> - load a module

• module unload <modulename> - unload a module

• module purge - unload ALL modules

• module whatis <modulename> - show a short description of the module, if available

• module help <modulename> - show more detailed information about a module, if available

Full help for the module command is available on the cluster by running man module, or on the web at the modules package website.

Multiple versions of a the same software package may be available. By default, module load <modulename> will usually load the latest version. If you want to use the non-default version of a module, specify the version with a / after the module name. For example, to load a specific version of Hmmer: module load hmmer/2.3.2

The environment variables that get set by a module depend on the requirements of the software. At a minimum, most modules will add the software's executable location to the user's $PATH, but it may also load other modules as dependencies, ensure that specific conflicting modules are not loaded, or take other actions to ensure that the software will run correctly.

If you find an error in a module on one of our clusters, please contact the helpdesk and cut/paste the entire command line prompt, including your username, clustername, directory, the command you attempted to run and error output you received so that we can start the troubleshooting process to fix the module.

Virtual environments are another way of defining a custom environment that can be loaded and unloaded at will, while leaving the “base” or default environment intact. Virtual environments are common when a user wants to use a software environment out of his or her home directory without requiring system-wide installation, special permissions (like root or sudo) or needing sysadmin intervention.

Miniconda is the recommended virtual environment and package manager for Python. Users can create unique virtual environments using conda based on the needs of the software they're using, including different versions of Python (Python 2 versus Python 3) and package dependencies such as numpy, scipy, matplotlib, and many others.

Please be aware that personal virtual environments do take up disk space in your home directory, because every virtual environment installs its own version of Python and related packages.

Administration of personal virtual environments installed in your home directory is left to the user.

On some clusters, admin-installed conda packages and/or conda virtual environments with particular software may be available under modules bio (miniconda 2) or bio3 (miniconda 3).