Skip to content

Scratch Space

Each user is provided with a working directory know as scratch space. This space is intended for intensive input/output (I/O) operations i.e., heavy reading and writing of data, involving very large files and/or a very large number of small files. Research groups may also request a scratch space. Unlike the home space and research space, the scratch space is not intended for long-term storage and cannot be accessed from a gateway node, with the exception of the rsync gateway used for file transfer.

Data stored in a user's scratch space is not backed-up, and files in a scratch space with no I/O operations for 45 days will be automatically deleted to ensure the space is available to all users. The limit on storage is 50TB and the initial limit on the number of files contained in a scratch space is 1,000,000 files.

A user's scratch space is available at /mnt/scratch/$USER, or use the bash environmental variable $SCRATCH. Use the quota command to check a user's current space and file quotas.

1
2
3
4
5
6
7
$ quota

Temporary Filesystems:
---------------------------------------------------------------------------------------------------------------------------------------

/mnt/scratch (/mnt/gs21)        Space Quota  Space Used   Space Free   Space % Used Filess Quota Files Used   Files Free   Files % Used
                                51200G       0G           51200G       0%           1048576      1            1048575      0%

Using a Scratch Space

Scratch Space can sustain high data transfer rates and is a good choice for data files used in running parallel on multiple nodes with intensive I/O requirements. Jobs of this type will run much faster with data accessed from a scratch space and users should follow the procedure below for best practice:

  1. Configure the job script and/or the main program for scratch space I/O using the path /mnt/scratch/$USER or the variable $SCRATCH
  2. Copy input data from the home space or research space to scratch space; to maintain data integrity keep the original data files in the home space or research space**
  3. Schedule the job and confirm successful completion of the I/O operations
  4. Move the resulting output data back to either the home space or research space
  5. Delete that data from the scratch space

Time Limits on Scratch Space

Files in a scratch space with no I/O operations for 45 days will be automatically deleted to ensure the space is available to all users. To find files in a scratch space approaching the 45 day limit run the following command:

find $SCRATCH -type f -ctime +40

Here the +40 argument specifies files with no I/O for more than 40 days. Users may set this argument to any number of days desired, up to the 45 day limit. This time is measured based on the "change time" of the file, so files extracted from archives will not be deleted until the 45 day limit is reached, after they have been extracted.

Multiple Scratch Spaces

You may have multiple directories in scratch, such as one associated with your home directory (/mnt/scratch/$USER) and one or more associatedithed with research spaces (/mnt/scratch/). Research scratch directory, like regular research directories are shared with other memebers of your reserearch group and intended to facilitate collaboration with groups. However, like the scratch space under your username, research scratch spaces is not backed-up, and files will be automatically deleted after 45 days.

Importantly, the limits on your scratch space useage applies to all scratch spaces. In other words, you have a maximum of 50TB and 1,000,000 files in scratch, regardless of which scratch directory those files are in.