Skip to content

Mapping HPC drives with SSHFS

Besides mapping HPCC drives with SMB, SSHFS can also enable mounting of HPCC file systems on a local computer. Different from SMB mapping, which can only map to home or research space via the MSU campus network, this method can also work on your scratch space and uses any internet network.

If users would like to use SSHFS without entering their password, they can generate an authentication key by following the direction of SSH Key-Based Authentication. After connecting to the HPCC with a terminal and executing the command ssh-keygen, two files, id_rsa and id_rsa.pub, are generated in the hidden directory ~/.ssh. Besides copying the file id_rsa.pub to the file authorized_keys, you also need to copy (transfer) the file id_rsa to your local computer. Make sure you know the path of the file in the local computer. You will need the file location when you set up SSHFS.

On Mac OSX

  1. Download and install (or upgrade) the most recent versions of the following packages: FUSE for macOS and SSHFS from https://osxfuse.github.io.

  2. Reboot (not required)

  3. Using the Terminal, create a directory (as <local_mount_point> in step 4) for each filesystem you wish to mount.  If you are creating the folder outside of your home directory, you may need to use sudo before each command (sudo = superuser do ). 

    1
    2
    3
    4
    5
    6
    7
    [MacBook-Pro:~ icer2]$ mkdir <local_mount_point>
    /* begin example home directory */
    [MacBook-Pro:~ icer2]$ mkdir /Users/icer2/hpcc_home
    /* end example home directory */
    /* begin example scratch directory in the Mac /Volumes folder where drives are mounted */
    [MacBook-Pro:~ icer2]$ mkdir /Volumes/scratch
    /* end example scratch directory */
    
  4. Mount the directory using the sshfs command. We suggest you add these additional flags to the command to make it be more "Mac-like" : -ocache=no ,  -onolocalcaches  and -o volname=hpcc_home .  For the last option, '-o volname' is the name that displays in the Finder title bar, so change it for difference file folders (e.g. use -o volname=hpcc_scratch for your scratch folder). After running the command, enter the password for logging into HPCC and the FUSE drive icon will show on the desktop of your local Mac computer.

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    [MacBook-Pro:~ icer2]$ sshfs <user_id>@rsync.hpcc.msu.edu:<remote_directory_to_mount> <local_mount_point> -ovolname=hpcc -o allow_other,defer_permissions,follow_symlinks,reconnect -ocache=no -onolocalcaches
    <net_id>@rsync.hpcc.msu.edu's password:
    /* begin example hpc's home directory, using /mnt/home/hpc/ */
    [MacBook-Pro:~ icer2]$ sshfs hpc@rsync.hpcc.msu.edu:/mnt/home/hpc/ /Users/icer2/hpcc_home -o volname=hpcc_home -o allow_other,defer_permissions,follow_symlinks,reconnect -ocache=no -onolocalcaches
    hpc@rsync.hpcc.msu.edu's password:
    /* end example home directory */
    /* begin example hpc's scratch directory with authorized key file ~/.ssh/id_rsa, using /mnt/gs18/scratch/users/hpc */
    [MacBook-Pro:~ icer2]$ mkdir /Volumes/scratch
    [MacBook-Pro:~ icer2]$ sshfs hpc@rsync.hpcc.msu.edu:/mnt/gs18/scratch/users/hpc /Volumes/scratch -o volname=hpcc_scratch -o allow_other,defer_permissions,follow_symlinks,reconnect,IdentityFile=~/.ssh/id_rsa -ocache=no -onolocalcaches
    (No password input)
    /* end example scratch directory */
    

    If <remote_directory_to_mount> is a static link, please make sure to put / at the end of the directory path:

    • For your home space, please use /mnt/home/<user_id>/ instead  of /mnt/home/<user_id>.
    • For your research space, please use /mnt/research/<group_name>/ instead of /mnt/research/<group_name>.

    As the above example (starting from line 5), home space /mnt/home/hpc/ is used on line 7 instead of /mnt/home/hpc.

  5. To unmount a filesystem, use the umount command.

    Note

    It's just the letter u before mount, NOT unmount.

    1
    2
    3
    4
        umount <local_mount_point>
        /* begin example */
        umount /Users/icer2/hpcc_home
        /* end example */
    

    You'll see these folders in the finder if you use the Go menu, but you won't see them listed in the left side with the other mounted drives.  You must use the terminal and umount command to disconnect.

References:

https://www.digitalocean.com/community/tutorials/how-to-use-sshfs-to-mount-remote-file-systems-over-ssh

On Linux Systems

Please refer to this web site:

https://tecadmin.net/install-sshfs-on-linux-and-mount-remote-filesystem/

for how to mount remote filesystem over SSH on Linux.

On Windows OS

  1. Install the latest stable release of winfsp (version 1.11.22176 as of 8/11/22) WinFsp installer
  2. Full upstream documentation: WinFsp Documentation

Note

Administrative access is required

  1. Install the latest stable release of SSHFS-Win (version 3.5.20357 as of 8/11/22) SSHFS-Win installer
  2. Full upstream documentation: SSHFS-Win Documentation

Note

Administrative access is required

  1. Mounting of Home and Scratch can be done through the command prompt or with the windows file explorer once WinFsp and SSHFS-Win are installed.

    1. Mounting Home directory with the command prompt
      • DriveLetter (H: or Z: etc.) is optional, but username is required
      • Command structure C:\>net use [DriveLetter>:] \\sshfs\<username>@rsync.hpcc.msu.edu
      • example C:\> net use F: \\sshfs\ryanjos2@rsync.hpcc.msu.edu
    2. Mounting Scratch directory with the command prompt requires a slightly different command. Note the addition of '.r' to sshfs (\\sshfs.r\) that designates to start from the root directory.
      • DriveLetter (H: or Z: etc.) is optional, but username is required
      • Command structure C:\>net use [DriveLetter:] \\sshfs.r\<username>@rsync.hpcc.msu.edu\mnt\scratch\<username>
      • example C:\>net use R: \\sshfs.r\ryanjos2@rsync.hpcc.msu.edu\mnt\scratch\ryanjos2
    3. Mounting Home or Scratch directory with Microsoft file browser:
      • Both Home and Scratch use the same steps, just with different network paths
        • Home: \\sshfs\username@rsync.hpcc.msu.edu
        • Scratch: \\sshfs.r\username@rsync.hpcc.msu.edu\mnt\scratch\username
      • In file browser on the 'This PC' page, select 'Map Network Drive' img!
      • Fill in the appropriate network path (\\sshfs\username@rsync.hpcc.msu.edu), select a drive letter and any desired options, and click Finish. img!
  2. Mounting a research space requires an additional tool, the SSHFS-Win Manager GUI. This Tool can also be used to mount Home and scratch space as well using the correct directory. These filesystems do not need any advanced settings. For home directory, leave the path blank. For Scratch space, use \mnt\scratch\hpcc_username\ for the path with the trailing slash.

    1. Install latest stable release of sshfs-win-manager (1.3.1 as of 8/11/22)
    2. open the SSHFS-Win Manager application img! If the SSHFS-Win Manager application is minimized, it can be opened using the task bar img!
    3. Create a new mount for the research space img!
      1. Basics tab
        • Name: HPCC-groupname (descriptive name for mounted drive)
        • Host: rsync.hpcc.msu.edu
        • Auth Method: password (ask on connect)
        • Path: /mnt/research/research_groupname/ (Note: ending slash)
        • Drive letter can be left as auto or you can select a specific Drive letter img!
      2. Select the Advanced tab and add the following options (connect on startup and reconnect on connection loss are optional)
        • Custom Command Line Params: On
        • idmap -> user
        • create_file_umask -> 000
        • umask -> 000
      3. sftp_server -> sg research_groupname -c /usr/libexec/openssh/sftp-server img!
    4. After Saving the new drive mapping, click the connect icon img!
    5. After successful authentication, the drive should show connected (green) and the folder is now available in the file browser img!

Other Mapping Software