Overview of HPCC file systems
The HPCC is comprised of several different file systems:
- The Home Space
- Research Space
- Scratch Space
- Node-local storage space
Home, Research and Scratch are network file systems, meaning that each node in the cluster must go through the network switch to access these spaces. This can potentially slow down file I/O for jobs. Alternatively, there are several options for local file storage which either uses hard drives connected to each development and compute node or the node's RAM. See each respective page above for more information. We also have Guidelines for Choosing File Storage and I/O.
Data that are subject to external security requirements are considered sensitive data. These requirements may be imposed by regulations or contractual obligations. In either case, users that intend to store sensitive data on HPCC systems must work with us to ensure data security. For more information, check out our Sensitive data hosting wiki page.