Research Space

A research group's central directory, or research space, is a shared directory established by a MSU Principal Investigator (PI) for use by the members of the PI's research group.

To create a research space the PI must submit a Research Request form. The initial limit on storage is 50GB and the initial limit on the number of files contained in a research space is 1,000,000 files. A PI may request an increase in storage of up to 1TB of space at no cost by submitting a Quota Increase Request form or an increase beyond 1TB or 1 million files for an annual fee by submitting a Large Quota Increase Request form. Use the quota command to check a research group's current space and file quotas.

$ quota
Research Groups: Space    Space   Space     Space     Files     Files     Files     Files        
                 Quota    Used    Available % Used    Quota     Used      Available % Used
-----------------------------------------------------------------------------------------------
<groupname>      4096G    3733G   363G      91%       2097152   432525    1664627   21%

The group's research space is associated with an assigned group name and is located at /mnt/research/<groupname> by default. It is accessible to all users who have been added to the group by the PI. Members of the group must set their user file mode creation mask and file permissions correctly such that the other group members have the appropriate access to the shared files in the research space. See the section Using a research space for more details.

All research space files are periodically, automatically copied to an offsite disaster recovery servers, except those files that a group has opted to store in a specially requested nodr space. To request data from this system, please submit a help ticket containing the file paths and the period, i.e. the time frame, from which the files should be restored.

Using a research space

To configure a research space such that all group members have the appropriate access to the files and directories contained within, it is important to read and follow the instructions below.

1. Ensure that all directories created in a research space have the group ownership set to the <groupname> and the set-group-ID (setgid) bit enabled.

By default, the group's research space /mnt/research/<groupname> is set with the correct group ownership and setgid bit. This setting makes newly created sub-directories and files within inherit the group ownership of the parent directory rather than the primary group of the individual user. To check the group ownership and setgid bit of a directory within the research space use the ls -ld <path/to/directory> command:

$ ls -ld /mnt/research/<groupname>/<subdirectory>
drwxrws--- 9 <username> <groupowner> 8192 Jul 22 08:38 /mnt/research/<groupname>/<subdirectory>

The setgid bit is indicated by the s in place of the group's executable permissions. See the page File Permissions on HPCC for more details.

If the access permissions of a sub-directory are not set correctly, the group may encounter a Disk quota exceeded error when creating or copying files. See the section Quotas on a research space for more details. Use the following commands to ensure all directories and files in the group's research space have the proper settings:

$ find /mnt/research/<groupname>/ -not -group <groupowner> -print0 | xargs -0 chgrp <groupowner>
$ find /mnt/research/<groupname>/ -type d -print0 | xargs -0 chmod g+s

2. Do not use the mv or cp -p commands to transfer files into the group's research space directories.

Both mv <filename> and cp -p <filename> may preserve an undesired group ownership attribute even when transferred into a research space directory with ownership and permissions configured correctly. You should use cp <filename> without the -p option.

3. Use the rsync --chmod=Dg+s command to transfer files from a local machine to the HPCC research space.

For example, use the command

$ rsync -avz testdir --chmod=Dg+s <username>@rsync.hpcc.msu.edu:/mnt/research/<groupname>/

to transfer a directory testdir from your local machine to the HPCC research space /mnt/research/<groupname>. This will automatically configure the transferred directory with the setgid bit.

4. During a bash session set the user file mode creation mask to 0007 or any lower value.

Use the umask command to set the file mode creation mask, e.g.

$ umask 0007

For the duration of the session, files and directories created by the user are now readable, writable and executable for all members of the research group. Alternatively, a user may run the following Powertools command one time to add the line umask 0002 to the user's .bashrc file:

$ module load powertools
$ umask_in_bash                

5. Ensure that each group member's currently active group ID is set to the research group of interest.

Each time a group member logs in to the HPCC that user's group ID is set to the primary group ID assigned by default when a user's account was created. However, a user may be added to other research groups at the request of that group's PI. After a login, the user may then toggle between group memberships using the newgrp <groupname> command when needed. For more information, refer to the Change Primary Group page. A user may request that the primary group ID be changed from the default setting by submitting a request via a help ticket.

Quotas on a research space

The space and file quotas on an research space are calculated by matching the group ownership settings of files stored on the HPCC to that of the research space group name. Hence, a user may not create files larger than 8 MB in a specified research space with a group ownership attribute different from that of the research space. Attempting to do so will likely result in a Disk quota exceeded error even though use of the quota command indicates that the reseach space quotas have not been exceeded. To resolve the Disk quota exceeded error in the absence of actual quota violations, users should follow the instructions in the section Using a research space to ensure that:

• The directory into which the files will be transfered has the same group ownership as the research space and the set-group-ID bit

• If the file already exists, its group ownership has been changed to that of the research space

• If the file is to be created, the primary group of the user creating the file is set to that of the research space