Transferring data with Globus
What is Globus?
Globus is a free service to the MSU community for secure, reliable research data management. Globus gives users the ability to move and share data regardless of user or file location through a single web browser-based interface. Users can manage data from any device (e.g., supercomputer, tape archive, lab cluster or equipment, public cloud, or personal computer/laptop) from anywhere in the world using their existing institutional identities. Use of Globus removes data management roadblocks by providing unified access to all storage locations, making it easy to work with data while ensuring reliability and security. For more information on Globus, check out the Globus website here.
Why should I use Globus?
Globus is ideal for moving large files and data transfers between ICER’s HPCC or external research collaborators because of its truly fire-and-forget method of transferring data. After you initiate a file transfer, Globus will work on your behalf to optimize transfer performance, monitor for transfer completion and correctness, and recover from network errors, credential expiration, and collection downtime without restarting the transfer. This allows you to navigate away from File Manager, close the browser window, and even logout.
Transferring data with Globus
The HPCC has a Globus data transfer endpoint, msuhpcc. This can be used
to do large data transfers to/from your personal computer, to/from
collaborators or to/from external HPC sites. You can also use it to
share data.
With Globus, you can create a transfer (to or from) between your computer and a folder on the HPCC which you have access to. This could be a folder in your home, research or scratch spaces.
You can also create Globus "shares" to your external colleagues. They can use your created link to access the HPCC directory. **You can not create a globus share on any folder in your home directory. ** You can only share folders in scratch space or any research space you have access to.
To use Globus Online, please perform the following steps:
- If you do not have a globus account, create one at https://www.globus.org
- Log into the MSU Globus Online portal, https://globus.msu.edu and set up a free Globus Online account.
- If you wish to use Globus to transfer data to/from your local computer, install the Globus Connect Personal tool.
- On the Globus Start Transfer page,
enter
msuhpccas the end point on one side. This will pull up an authentication window. Use your MSU NetID for the username, and your MSU NetID password for the passphrase. This will set up an authenticated session that will last as long as specified in the Credential Lifetime field, up to a limit of 2 weeks. - Select and authenticate with the other endpoint for your transfer, and initiate your transfer. An example can be seen from How To Transfer Files with Globus.
- To share data in a HPCC folder accessible to you with other persons, please check How To Share Data Using Globus.
More information about using Globus can be found on Data transfer and sharing using Globus or Globus support page.
For further training on the Science DMZ and how to use Globus, please self-enroll in ICER's DMZ Globus Training D2L course.