Skip to content

Using conda

This wiki is based on conda version prior to 4.6. It may not apply to 4.6 and later versions (where, for example, conda environment is activated by conda activate ). Check the manual before you start using conda.

Introduction

Anaconda is a distribution of python that contains a free, easy-to-install package manager, environment manager and Python distribution with a collection of 1,000+ open source packages with free community support. Conda is the package and environment manager that installs and updates packages and their dependencies, included in all versions of Anaconda. 

Anaconda needs to be installed in your home or project directory and so does any software which will be installed using it.  This allows users to have full control of their programs.

This wiki serves only as a mini-version of conda tutorial. You are highly recommended to read the User Guide.

conda to install and run a program called "macs2". First we go to Anaconda Cloud to search for it. Once it's found, we pick a source and click on the link to get the installation command (i.e., conda install -c bioconda macs2 below)

Installation of macs2

1
2
3
4
5
export PATH=$PATH:$HOME/anaconda3/bin
conda create --name macs2
source activate macs2
conda install -c bioconda macs2 
source deactivate

To launch it later on (for example, in your SLURM job submission script):

Using it

1
2
3
4
export PATH=$PATH:$HOME/anaconda3/bin
source activate macs2
#[your commands]
source deactivate

Installation of Anaconda

Visit https://docs.anaconda.com/anaconda/install/linux for installation instruction. And Linux installers are available from https://www.anaconda.com/download/#linux

Note:

1) Choose 64-Bit (x86) Installer.

2) Version 2 or 3? You should choose python2 or python3 depending on what your software requirements are.  Some libraries/packages only work on Python 2 so you should check with the python package you ultimately want to use.  If you don't know, select the latest version 3. 

3) Anaconda now partners with Microsoft and asks if you want to install "VSCode"  THIS WILL NOT WORK.  The installation requires administrative privileges that you don't have.  

You can download the install file directly into your home directory by first getting the URL of the download (right-click on the download link and select "copy link address" or "copy link location").  Then open a terminal to log onto a dev-node, and use this command:

curl -O <download link>

An example link can be: https://repo.anaconda.com/archive/Anaconda3-2022.05-Linux-x86_64.sh

Once the file Anaconda3-2022.05-Linux-x86_64.sh is downloaded, you can run the command: 

bash Anaconda3-2022.05-Linux-x86_64.sh

During Installation the installer will ask you

  1. Accept licence agreement : you must type yes to agree.  We can not install for the who system, you must install yourself and agree to this license
  2. Where to install : you can install in any location in your home directory, the default is Anaconda2 or Anaconda3.  Note you can have multiple installations of Anaconda
  3. Initialize Anaconda (run conda init?) : please answer "no". A more careful setup is suggested. Please follow the instruction in the next section after this installation.
  4. Install VS Code?  This is a new bundle that comes with Anaconda.   This won't work and you must answer "no"

Please remember where Anaconda is installed in the step 2 above. The installation path will be used for module setup in the next section.

In case your installed Anaconda is auto-activated on startup, please run the command:

conda config --set auto_activate_base false

after close and re-open your current shell. Please read the next section for the setup and the start-up of your Anaconda.

Module Setup for Anaconda

In order to have no conflict between user-installed Anaconda and system-installed Python versions, a setup in the $HOME/.bashrc file for module system is necessary. The .bashrcfile under your home directory can be used to do environment setting for every time you log in to a HPCC node. You can modify the file by an editor program (such as nano, vim) for Anaconda setup:

(1) Please check if any conda initialization or setup script is in the ~/.bashrc file. The script starts with # >>> conda init >>> and ends with # <<< conda init <<<:

1
2
3
4
5
6
7
# >>> conda init >>>
# !! Contents within this block are managed by 'conda init' !!
__conda_setup= ... ...
... ...
... ...
unset __conda_setup
# <<< conda init <<<

If there is, please remove or comment all of the command lines out (by adding the sign "#" in its front). If there is any export command line to add the installed Anaconda bin path to PATH environment variable, please also comment it out.

(2) If your installed Anaconda path is ~/anaconda3 or ~/anaconda2, you can skip this step and go to the next one. If not, please set the variable CONDA3PATH to be the place of installed Anaconda3 by adding the command line:

1
export CONDA3PATH=<your installed Anaconda3 path>

in the ~/.bashrc file. If you have Anaconda2 installed, you may set the variable CONDA2PATH to be the installed path of Anaconda2. After the environment setup, please save the ~/.bashrc file. Log out and log into a dev node.

(3) Whenever you want to use Anaconda3 commands, just run:

1
module load Conda/3

to initiate conda and CONDA3PATH is in your PATH variable. Loading the module will also replace any loaded Python module so there is no conflict. If CONDA2PATH variable is set up, please load Conda/2 module for using Anaconda2.

(4) If you would like to have your installed Anaconda3 to be the default Python program when your session starts up, you can add the command:

1
module load Conda/3 2> /dev/null

to the  ~/.bashrc file after export CONDA3PATH=... command line. Using Conda module can also avoid a conflict with Web-based Remote Desktop gateway. If anaconda2 is installed, you can use Conda/2 instead of Conda/3 module.

Managing conda

To manage conda, you should use a Terminal window to check the following.

You can begin by checking if conda is installed by using the command: 

Checking the version

1
conda --version

If it is installed, the number of the version you have installed should show up after the previous command in Terminal.

Version example

1
conda 4.2.9

If there’s an error message, check to see if you’re logged into the same account that you used to install Anaconda or Miniconda. You can update conda to the current version by typing in to your command line the following in line 1:

Updating conda

1
2
conda update conda
Proceed ([y]/n)?

Line 2 asks you if you wish to proceed. 

Type ‘y’ and hit Enter to update.

Managing Environments

You will use a Terminal window for the following steps.

To create an environment, use the following command:

Creating an environment

1
conda create --name testing biopython

Two things to note from this: you can replace ‘testing’ with whatever name you want, and biopython is just the program that was chosen for this. If your program is not yet installed, you will be prompted to install a new set of packages with a similar message to how you are prompted if you want to update conda. Type ‘y’ and hit "Enter" to confirm that you want to install the package. To activate this new environment, choose the appropriate command (keep in mind that our environment name is ‘testing’):

Activate an environment

1
source activate testing

To deactivate an environment and return to the root, use the following command:

Deactivating an environment

1
source deactivate testing

Now let’s say you want to create a new environment and install a different version of Python along with a couple of packages. We will use the Astroid and Babel packages.

Run this command:

Environment with multiple packages

1
conda create --name test2 python=3.5 astroid babel

Your new environment is named ‘test2’ (which can be interchanged for whatever name you like when you’re running an environment) and it has Python 3, Astroid, and Babel installed.

If you wish to find out more about the create command, run: 

create help

1
conda create --help

To display your environments, use the command: 

Displaying environments

1
conda info --envs

Your environments should be outputted, and there will be an asterisk next to the active environment.

Your current environment should be in (parentheses) or [brackets] in front of your prompt. To switch to another environment, just use the activate command with the environment name. If you wish to make a copy of an environment, use the following command: 

Copying an environment

1
conda create --name hello --clone testing

You can verify that the clone was made by displaying your environments.

You can delete an environment by using: 

Deleting an environment

1
 conda remove --name hello --all

Since we don’t want a clone of testing, we decided to delete it. You can check that it is deleted by displaying your environments.

Managing Python

Use a Terminal window again for these steps.

You can search for which versions of Python are available to be installed by using the command: 

Searching for available Python versions

1
conda search --full-name python

“--full-name” lists only the packages whose full name is exactly ‘python’.

If you want to search packages which contain the text ‘python’, but that is not its full name, use the command: 

Versions with the word "python"

1
conda search python

We will go over how to install a new verison of Python without overwriting our original Python version. Let’s start off by creating a new environment called py3. Use the create command:

Environment for installing a new version of Python

1
conda create --name py3 python=3

If you get prompted to install a new package, type ‘y’ and hit "Enter". Display your list of environments with

Displaying new environment

1
conda info --envs

to check if py3 was successfully created, and activate py3 by using: 

Activating py3

1
source activate py3

Now check the version of Python being used with: 

Checking py3's Python version

1
python --version

Now we can switch to one of our previous environments that we first made when installing conda. Let’s switch back to testing. Using the command: 

Switching to an environment with a different Python version

1
source activate testing

Now you check which version we are using with the same used earlier.

This should show the same version that was used when you first installed conda.

Now let’s deactivate testing with the command: 

Deactivating testing

1
source deactivate testing

Managing Packages

Use the Terminal window for this section.

The following command will allow you to see the packages installed in your environment: 

Listing packages command

1
conda list

This website provides a list of packages available to install: http://docs.continuum.io/anaconda/pkg-docs.html

The package that we will install to practice is called “beautifulsoup4” – this is a package which provides a Python library designed for screen-scraping/MIT.

To search if this package is available to be installed, try to use the following command: 

Searching for a package

1
conda search beautifulsoup4

If it displays the package, then it is available to install. Use the following command to install a package: 

Install a Package

1
conda install --name test2 beautifulsoup4

You will be prompted if you want to proceed. Type ‘y’ and hit "Enter"

Now activate the environment that this package was installed in, in this case test2: 

Activating environment with the new package

1
source activate test2

Now let’s check if the package is installed by using:

Listing the packages

1
 conda list

It should be listed if it is installed.

Installing Packages

Use the Terminal window and an internet browser for this section.

Not all packages can be installed with conda install, so we can pull packages from anaconda.org. The following website has other packages that we can use and install: http://anaconda.org. For the example, we will install a package called ‘bottleneck’.

Go to this page and use the search bar in the upper left corner to search for ‘bottleneck’.

Choose the most frequently downloaded version of bottleneck. The format of the text should be something similar to ‘conda-forge/bottleneck1.2.1’. This is split into two places that you can click on. The first is ‘conda-forge’, which is the owner of the package. We can ignore this one and click on ‘bottleneck1.2.1’.  At the middle to bottom portion of the page, there should be a command listed to install this package. In this case, the command is:

1
conda install -c conda-forge bottleneck

Activate your test2 environment and enter this command to install bottleneck.

Installing a package from Anaconda.org

1
2
source activate test2
conda install -c conda-forge bottleneck

You will be prompted if you want to proceed, again type ‘y’ and hit "Enter". We can now check if the new package is installed.

1
conda list

If it is installed, it show in the Terminal window after using the previous command.

Installing a Package with pip

Pip is used to install packages that are not available from conda or anaconda.org. It is only used as a package manager and does not manage environments. If you don’t have the environment ‘test2’ (or whatever environment you want to install a package in) active, activate it now. We will install a program named ‘see’ into this environment. The command for this is: 

Installing with pip

1
pip install see

Now use the listing packages command to check if ‘see’ is installed.

Note: You can replace ‘see’ with whatever pip package that you are trying to install. This is just an example.

Installing Commercial Packages

If you wish to install a commercial package, follow the format for installing packages from conda, which was described earlier in this document in the "Managing Packages" section.

Removing Packages, Environments, or conda

To remove a package from an environment, use the following command:

Removing a package from an environment

1
conda remove --name ENVIRONMENT_NAME PACKAGE_NAME

You would replace ENIVRONMENT_NAME with the name of your environment, and also replace PACKAGE_NAME with the name of the package. For an example, we will remove the ‘astroid’ package from test2: 

Ex. Removing a package

1
conda remove --name test2 astroid

You will be prompted if you want to proceed, type ‘y’ and hit "Enter". Now use the list command to check if ‘astroid’ has been removed.

To remove an environment, use the command:

Removing an environment

1
conda remove --name ENVIRONMENT_NAME --all

Replace ENVIRONMENT_NAME with the environment you want to remove. For an example, we will create a new environment called ‘trash’. Remembering how to create an environment from before, use the command: 

Ex. Removing an Environment

1
conda create --name trash python=3.5

Check that this environment exists by using: 

Ex. Removing an Environment

1
conda info --envs

Now we run: 

Ex. Removing an Environment

1
conda remove --name trash --all

You will be prompted to proceed, type ‘y’ and hit "Enter". Now check if the environment exists by looking at all of your environments again with the same command used a moment ago.

The environment ‘trash’ should no longer be there.

If you want to remove anaconda or miniconda, you can use the following commands:

Removing anaconda or miniconda

1
2
rm -rf ~/miniconda
rm -rf ~/anaconda