Skip to content

AlphaFold installed in HPCC

To use AlphaFold installed in HPCC, users can use the command:

1
[UserName@dev-amd20-v100 ~]$ ml spider AlphaFold

to find all versions installed in HPCC and use the command:

1
[UserName@dev-amd20-v100 ~]$ ml spider AlphaFold/<version>

to find how to load a specific AlphaFold version, where <version> is the version of AlphaFold to load. All AlphaFold versions use the same data structure and location /mnt/research/common-data/alphafold/database as mentioned in Alphafold via Singularity.

Illegal memory address

If CUDA_ERROR_ILLEGAL_ADDRESS or an illegal memory access was encountered while running AlphaFold, this is mostly due to not enough memory in GPU cards (from the python package "jax"). Please try to request the high memory GPU card A100 (79GB) to run your AlphaFold jobs. You can also set the environment variable:

1
export XLA_PYTHON_CLIENT_ALLOCATOR=platform

to see if the allocated memory is enough or not. Please see GPU memory allocation for more information.

AlphaFold example

Users can get an example of AlphaFold to run on HPCC nodes. After log into HPCC and ssh to a dev node with GPU cards, users can run the powertools command:

1
[UserName@dev-amd20-v100 ~]$ getexample AlphaFold

to copy the example directory AlphaFold in the current directory. After you cd to the directory, you should be able to see the files inside:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
[UserName@dev-amd20-v100 ~]$ cd AlphaFold
[UserName@dev-amd20-v100 AlphaFold]$ ls
data.fasta  README  slurm_script.sb
[UserName@dev-amd20-v100 AlphaFold]$ cat slurm_script.sb
#!/bin/bash
#SBATCH --job-name AlphaFold
#SBATCH --time=12:00:00
#SBATCH --gres=gpu:4
#SBATCH --cpus-per-task=24
#SBATCH --mem=90GB
#SBATCH --constraint=[intel18|amr|nvf|nal|nif]

export NVIDIA_VISIBLE_DEVICES="${CUDA_VISIBLE_DEVICES}"

ml -* fosscuda/2020a AlphaFold/2.0.0

alphafold --fasta_paths=$PWD/data.fasta --output_dir=$PWD --preset=casp14 --max_template_date=2020-05-14 --model_names=model_1

scontrol show job ${SLURM_JOBID}
js -j ${SLURM_JOBID}

The job script file slurm_script.sb shows how to load the AlphaFold module and run the command alphafold. Since most of the specifications and variables have been set in the alphafold command script and the module file, 5 options:

1
2
3
4
5
--fasta_paths
--output_dir
--preset
--max_template_date
--model_names

for the job are specified in the command line. Users can look into the AlphaFold documentation or use the commands:

1
[UserName@dev-amd20-v100 AlphaFold]$ alphafold --help

to find out how to use all options after you load the AlphaFold module.