Skip to content

AlphaFold installed in HPCC

The following command can be used to find all versions of AlphaFold installed on HPCC:

1
[UserName@dev-amd20-v100 ~]$ ml spider AlphaFold

To find how to load a specific AlphaFold version, where <version> is the version of AlphaFold to be load, use:

1
[UserName@dev-amd20-v100 ~]$ ml spider AlphaFold/<version>

All AlphaFold versions use the same data structure and location /mnt/research/common-data/alphafold/database as mentioned in Alphafold via Singularity.

Illegal memory address

If CUDA_ERROR_ILLEGAL_ADDRESS or an illegal memory access was encountered while running AlphaFold, this is usually due to not enough memory in GPU cards (from the python package "jax"). Please try to request the high memory GPU card A100 (79GB) to run your AlphaFold jobs.

You can also set the environment variable to see if the allocated memory is enough or not:

1
export XLA_PYTHON_CLIENT_ALLOCATOR=platform

Please see GPU memory allocation for more information.

AlphaFold example

Users can get an example of AlphaFold to run on HPCC nodes. Log into HPCC and ssh to a dev node with GPU cards, then run the following command to copy the example directory AlphaFold to the current directory:

1
[UserName@dev-amd20-v100 ~]$ getexample AlphaFold

(If the above command fails, you may need to manually load powertools with module load powertools)

After you cd to the directory, you should be able to see the files inside:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
[UserName@dev-amd20-v100 ~]$ cd AlphaFold
[UserName@dev-amd20-v100 AlphaFold]$ ls
data.fasta  README  slurm_script.sb
[UserName@dev-amd20-v100 AlphaFold]$ cat slurm_script.sb
#!/bin/bash
#SBATCH --job-name AlphaFold
#SBATCH --time=12:00:00
#SBATCH --gpus=4
#SBATCH --cpus-per-task=24
#SBATCH --mem=90GB
#SBATCH --constraint=[intel18|amr|nvf|nal|nif]

echo "This script is from ICER's AlphaFold tutorial"

export NVIDIA_VISIBLE_DEVICES="${CUDA_VISIBLE_DEVICES}"

ml -* fosscuda/2020a AlphaFold/2.0.0

alphafold --fasta_paths=$PWD/data.fasta --output_dir=$PWD --preset=casp14 --max_template_date=2020-05-14 --model_names=model_1

scontrol show job ${SLURM_JOBID}
js -j ${SLURM_JOBID}

The job script file slurm_script.sb shows how to load the AlphaFold module and run the command alphafold. Since most of the specifications and variables have been set in the alphafold command script and the module file, five options for the job are specified in the command line:

1
2
3
4
5
--fasta_paths
--output_dir
--preset
--max_template_date
--model_names

Users can look into the AlphaFold documentation or use the commands:

1
[UserName@dev-amd20-v100 AlphaFold]$ alphafold --help

to find out how to use all options after you load the AlphaFold module.