Local File Systems
The local file systems are available on each cluster compute node and development node. These file systems are directly connected to each node and may be faster than the home, research, and scratch filesystems which must be accessed over the network.
There are two kinds of local file systems: the local hard drive (accessible from
/mnt/local) and RAMDISK space (accessible from
Read more about each type of storage below.
Please limit the use of the local spaces. It is also used for MPI by the MPI runtime to implement fast communication between MPI processes. Users are advised to clean up the space after use. Files that over 2 weeks old will be removed without notice. If the space is over 90% full, we may clean up unused files without notice.
Local Hard Drive
Each node has local hard drive storage accessible from either
The files on this space can be accessed locally on each node without going through network.
This space is a good choice for jobs using a single node or multiple nodes
where I/O is processed only on each node's local file. When network
traffic is high, using this space will likely allow your program to run
faster than running on Home, Research or Scratch space.
Please note that this local space is shared with all processes running on the same node and there is no direct I/O from other nodes. The space also has no auto backup. It should be used as temporary storage space. When the execution of programs in a job is completed, any useful files in this space should be saved back to Home or Research space.
To ensure the continued, reliable operation of each node, all user accounts will
be prevented from writing data to local storage that consumes more than 95% of the
Once a user account's usage of the
/tmp directory exceeds 95% or the total available space for that
directory, it will be locked and prevented from writing any additional data to the
/tmp directory until
files and data are removed bringing the user account back under 95% space utilization.
In addition to the space restrictions on
/tmp, local user account quotas are also in place for another shared directory, the
/var directory. This directory is rarely accessed directly by user accounts, and shouldn't be a concern for
most users of the HPCC; however, if you receive any errors that your user account has exceeeded the
quota, please contact us right away, and we'd be glad to help troubleshoot the error with you.
The environment variable
$TMPDIR references a directory that is automatically created when your job starts on the local node.
This directory is automatically deleted after the job finishes. It is accessible from both
/tmp/local/$SLURM_JOBID. Since local hard drive storage is shared amongst all jobs on a node, using
a convenient way to keep your files separated and organized.
Any files saved to
$TMPDIR should be transferred to a Home or Research space
before the job ends.
RAMDISK space is a “logical” storage space; it sits inside a
node’s RAM, not the hard drive. Linux supports a system tool that provides an
interface for users to intercept the I/O requests to
memory operations. We may think of it as a virtual disk in memory.
Due to this nature, access to this space is actually access to RAM. Since the bandwidth of the access is much higher, the I/O operations are considerably faster than the local hard drive space. However, since programs take up some of the node's memory, the usable RAM space for program execution becomes less.
This space is good for programs that do not require large memory and perform very frequent I/O on small files.