Warning

This is as a Lab Notebook which describes how to solve a specific problem at a specific time. Please keep this in mind as you read and use the content. Please pay close attention to the date, version information and other details.

This Lab Notebook was adapted from the documentation kindly provided by the Mendoza Group Wiki available at https://sites.google.com/view/mendozagroup/home

Parallel submission script for CRYSTAL23 on the MSU HPCC (2023-03-14)

Usage: Your input file should be called .d12 and be in the same directory as the submission script. The script is submitted by running the command sbatch

Tips:

The main parameters you will be changing are the job name (note that it is specified twice in the script), run time (currently set to 30 minutes), nodes/cores (currently 2 nodes, 8 cores per node), memory per cpu, and account (general or mendoza_q). These will depend on the type of job you are running and the amount of resources you would like to allocate.

There is one #SBATCH line commented out, this is to submit to scavenger queue instead of general or mendoza_q.

There are two cp lines commented out, these might come in handy to restart calculations.

#!/bin/bash --login
#SBATCH --job-name=test
#SBATCH --account=general
#SBATCH --time=0-00:30:00
#SBATCH --nodes=2
#SBATCH --ntasks-per-node=8
#SBATCH --mem-per-cpu=3G
#SBATCH -C [intel16|intel18]


module purge
module load CRYSTAL/23
UCX_TLS=ud,sm,self


export JOB=test
export DIR=$SLURM_SUBMIT_DIR
export scratch=/mnt/gs21/scratch/$USER/crys23


rm -r $scratch/$JOB
mkdir $scratch/$JOB/
cp $DIR/$JOB.d12 $scratch/$JOB/INPUT
cd $scratch/$JOB


unset FORT_BUFFERED
export I_MPI_ADJUST_BCAST=3
export I_MPI_DEBUG=5
ulimit -s unlimited
export OMP_NUM_THREADS=1

export LD_LIBRARY_PATH=/opt/software/UCX/1.12.1-GCCcore-11.3.0/lib64:$LD_LIBRARY_PATH

mpiexec -n $SLURM_NTASKS -genv UCX_TLS ud,sm,self /opt/software/CRYSTAL/23/bin/Pcrystal > $DIR/${JOB}.out 2>&1 $DIR/${JOB}.out
cp fort.9 ${DIR}/${JOB}.f9