Each user is provided with a working directory know as scratch space. This space is intended for intensive input/output (I/O) operations i.e., heavy reading and writing of data, involving very large files and/or a very large number of small files. Research groups may also request a scratch space. Unlike the home space and research space, the scratch space is not intended for long-term storage and cannot be accessed from a gateway node, with the exception of the rsync gateway used for file transfer.
Data stored in a user's scratch space is not backed-up, and files in a scratch space with no I/O operations for 45 days will be automatically deleted to ensure the space is available to all users. The limit on storage is 50TB and the initial limit on the number of files contained in a scratch space is 1,000,000 files.
A user's scratch space is available at
/mnt/scratch/$USER, or use the bash environmental variable
$SCRATCH. Use the
quota command to check a user's current space and file quotas.
1 2 3 4 5 6 7
Using a Scratch Space
Scratch Space can sustain high data transfer rates and is a good choice for data files used in running parallel on multiple nodes with intensive I/O requirements. Jobs of this type will run much faster with data accessed from a scratch space and users should follow the procedure below for best practice:
- Configure the job script and/or the main program for scratch space I/O using the path
/mnt/scratch/$USERor the variable
- Copy input data from the home space or research space to scratch space; to maintain data integrity keep the original data files in the home space or research space**
- Schedule the job and confirm successful completion of the I/O operations
- Move the resulting output data back to either the home space or research space
- Delete that data from the scratch space
Time Limits on Scratch Space
Files in a scratch space with no I/O operations for 45 days will be automatically deleted to ensure the space is available to all users. To find files in a scratch space approaching the 45 day limit run the following command:
find $SCRATCH -type f -ctime +40
+40 argument specifies files with no I/O for more than 40 days. Users may set this argument to any number of days desired, up to the 45 day limit.
This time is measured based on the "change time" of the file, so files extracted from archives will not be deleted until the 45 day limit is reached, after they have been extracted.
Multiple Scratch Spaces
You may have multiple directories in scratch, such as one associated with your home directory (/mnt/scratch/$USER) and one or more associatedithed with research spaces (/mnt/scratch/
Importantly, the limits on your scratch space useage applies to all scratch spaces. In other words, you have a maximum of 50TB and 1,000,000 files in scratch, regardless of which scratch directory those files are in.