Skip to content


This is as a Lab Notebook which describes how to solve a specific problem at a specific time. Please keep this in mind as you read and use the content. Please pay close attention to the date, version information and other details.

The following is a tutorial by Wendy Leuenberger (with input from Alex Wright and Erin Zylstra) in the Zipkin Quantitative Ecology Lab. Wendy wrote this tutorial for her lab group and agreed to share it here in case other groups find it useful. If you or your team have similar tutorials we would be delighted to highlight them here as an ICER Lab Notebook.

HPCC via RStudio and GitHub (2022-09-28)

Wendy Leuenberger

This document is a guide to using MSU's HPCC. I access the HPCC via the Terminal in RStudio, and I move files from my computer to the HPCC using GitHub. I used the website to set up this system. I reference pages of that guide for instructions or additional reading. This method takes a decent amount of setting up but is convenient once in place.

Alex Wright and Erin Zylstra contributed to the sections about using the HPCC.

Software and Account Requirements


You need RStudio and R installed on your computer.



Additional reading on versions:


GitHub Account

You need a GitHub account for this method. You can make one here: Here's some additional reading on making an account:

You will need to know your username, password, and the email associated with your account for later steps.


Make sure that RStudio is running GitBash when it opens the Terminal. To do so, click Tools -> Global Options -> Terminal -> New terminals open with GitBash. If it doesn't say GitBash, change the drop down menu so that it does. It GitBash is not an option, you'll need to download Git. Here's reading on how to install Git for different operating systems:

Check installation

Go to the Terminal in RStudio. It's one of the tabs next to the Console. (Or use key command Shift+Alt+M) Type which git to make sure Git is installed.

HPCC set up


You need an account for the HPCC. A PI can request one. Instructions are here:


Make sure you can log in. Go to the Terminal window in RStudio. Type ssh -XY and press enter. (Example: I type ssh -XY

Type your password and press enter. Sometimes it makes you do it twice.

If it doesn't work, contact ICER to see why.

Map the drive

This method works best if the HPCC drive is mapped to your computer. Follow the instructions here to do so. If it doesn't work or you get weird results, contact ICER.

GitHub and RStudio

Git configuration

Introduce yourself to Git using your GitHub user name and the email used for your GitHub account (may or may not be your MSU one). Use the Terminal in RStudio or the R package usethis to do so.

Using the Terminal:

git config --global 'Jane Doe'

git config --global ''

git config --global --list

Using usethis:

## install if needed (do this exactly once):

## install.packages("usethis")


use_git_config( = "Jane Doe", = "")

For further reading: If you want GitHub to use 'main' instead of 'master' for the branch names, there is code for that on this page.

Personal Access Tokens

Personal Access Tokens (PAT) are the way that GitHub authenticates your computer's identity. You need one for each computer that you work from. I use https tokens, though there are also ssh tokens. Read here and the following chapter if you want more information (There's a ton of information on this step):

Create token

usethis::create_github_token() This line of code will open GitHub and begin the process of generating a token. You will need your password. The default settings that open up for the token have been fine by me so far. I usually name the token 'MSU laptop 9/22' or something similar to designate which computer it's for. They're easy enough to create that I leave the default 30 day expiration date, but you can change it if you wish. Click Generate token. Copy the string of numbers and characters that comes up. It'll be something like ghp_LettersNumbersHere231. This is the token.

Save token

Install the gitcreds package if necessary (install.packages('gitcreds')). Run gitcreds::gitcreds_set(). Follow the prompts. If this is your first token, I think it just asks you to enter the token. Paste the token when prompted. If you already have an active token, select 2 when prompted to replace the token, and enter the new token

GitHub Repository

There are some good chapters on testing out repositories on the web guide. I'm going to skip them so that this set-up is more streamlined. We may have to revisit them if we run into any trouble. Here's the chapters in question: and

Create a repo

We're going to create a new repository for a project. This is the easiest way to set it up. Here's the detailed reading:

There are also instructions for existing projects, either with GitHub first or last

Go to your GitHub page. Click New next to your repositories, or on your profile, navigate to Repositories and then click New. Give your repo a name ('testrepo'). All other defaults are fine to start. You can add a description and a readme file if you want. The readme files are needed for the Zipkin lab and they can help keep things organized, but they're not required. If you work with proprietary data, you need to make sure the repo is set to private. I usually set my repos to private unless they're complete and ready to be shared. Once you're done with the settings, click Create repository.

Click Code on the repo page. Copy the https URL (example:

Link GitHub and RStudio

Create RStudio project based on GitHub

There are many different ways to do this. I create RStudio projects for each of my Repos. I create one using the RStudio GUI. Click File -> New Project -> Version Control -> Git. Enter the https URL for the repo. Click Browse to choose where to save the repository and R project file. I have a folder on OneDrive for GitHub projects. I also have some in other folders for research. As long as you can find it again, it's fine. Click Create Project.

Additonal approaches and information is in the same chapter as creating repos:

GitHub via RStudio terminal

I find the command line really helpful for the RStudio workflow. There's only a couple of commands that are needed. There is also a Git GUI built into RStudio if you prefer that approach.

In your R project that is linked to your GitHub repository, navigate to the Terminal (separate tab next to the Console, or Alt+Shift+M). Type git status. The output will show you tracked files (ones that have been uploaded to Git in a previous version but have changes) and untracked files (not on GitHub yet). Files uploaded to GitHub and haven't been changed don't show up. This output is a helpful summary to see what files might need attention. I usually do this first.

Before you start making any changes, type git pull. I've found that it's helpful to do that first as a habit to prevent any merge problems if someone else on a shared repo uploaded something, or if you have files on the HPCC that got changed.

Add a file to the folder on your computer that is linked to your repo. I created an R code file called AddMe.R

Go back to your Terminal in RStudio. Type git status. You can now see your new file in the Untracked files section. Type git add AddMe.R (or whatever you called your file). Type git status. Now AddMe.R is in the section of Changes to be committed. You've told GitHub that you want this file, but it's not ready yet. All changes to your repos have to be committed and then pushed to get them on the repo.

Type git commit -m "Add my first file to GitHub". Commits are kinda like a save point. These can be incremental points and updates. For example, if I get one new piece of code working, I'll commit it with a note of what part got updated. You have to commit before you can upload. Type git status and you can see that your local branch is ahead of your GitHub repo by one commit.

To upload, type git push. This will upload the file to GitHub. You can see online that it's now present. Type git status. Your local computer is now up to date with the main GitHub.

If you make a change from another source (the HPCC, or if a collaborator makes a change), then you'll want to pull those changes with git pull.

These are pretty much all the functions I do with GitHub. There is more reading online, as well as other chapters about how Git/GitHub work.

Three Terminals

All of these pieces result in being able to do everything from the Terminal in RStudio. You can open multiple terminals. Once you've navigated to the Terminal, there's a dropdown menu right under the word 'Terminal' that is probably labeled 'Terminal 1'. Use the 'New Terminal' option to open two additional Terminal windows.

Terminal 1 saves and uploads files, Terminal 3 downloads files, and Terminal 2 is where you run the files on the HPCC.

Terminal 1 is where you save and upload files to your repository. This Terminal's working directory should be your GitHub Repository. This on should already be in the correct place if you're working from an RStudio project that's linked to your GitHub. You can rename this Terminal if you want. Mine is labeled 'Laptop'.

Terminal 2 is your HPCC account. This is where you run your code. I label this one 'HPCC'. On this terminal, log into your HPCC account with the ssh -XY and your password. To do work on the HPCC, you need to navigate to a development node (ex. dev-intel18). There are multiple. It doesn't matter which one you use. I always use one with low current usage. Type ssh dev-intel18 (or whichever dev-intel you want) to enter the node.

Terminal 3 is your mapped HPCC drive. This is where you download your files from your repository. I have this one labeled 'Y Drive'. If you are off campus/not wired in, you need to be remotely connected to the BIG-IP Edge Client. You also need to be logged into your HPCC account. Find your mapped drive in your files and click on it. You may need to enter your password again. Then go to your third terminal Type cd /y (or whatever letter you picked) to set your mapped drive as the working directory for this terminal. This terminal window can be a bit slower than your personal computer.

Get GitHub Repo on the mapped drive

The repo needs to be on your mapped drive/HPCC account. You can copy and paste it into your mapped drive. Or you can add it using the git clone command. Type git clone and the https URL for that repo (ex git clone Check that it's working via the git status command.

Implementing this method and using the HPCC

Make sure that you are in your repository on all three terminals. Use the cd command to do. Just cd will put you at the base of your file structure (i.e. the c/ drive). cd .. will move you up one folder. ls shows you the contents of your current working directory. cd test-repo/ would let me enter the test-repo repository if that were a folder in my current working directory. cd GitHub/test-repo/ would enter the GitHub folder and then the test-repo Repository within the GitHub folder. cd y/ would put me on the Y drive while cd c/ would put me on the C drive. If cd y/ (or whatever letter you used for your mapped drive) doesn't work, make sure you are logged into the HPCC, the BIG-IP Edge Client if necessary (if on WiFi), and can open the Y drive in your finder window.

Push and pull

I do all of my changes from my computer itself. I regularly commit my changes and push whenever I want to move files to the HPCC. Navigate to the mapped drive terminal and pull those changes. Then switch to the HPCC and submit the jobs using the sbatch files (see Lab-Resources GitHub) or otherwise use the files on the HPCC. You then add, commit, and push any model output from the HPCC back to your computer.

Necessary files

R code

If you are using R, do not set a working directory in your code. Your working directory can be set in the .sb file or can be the directory that you submit the job from.

Make sure that you save any output from your code. You can save R objects as .RData. Here's an example of saving the Out object to the Output folder as Out.RData

save(list = Out, file = file.path('Output', 'Out.RData')))

Data and models

Any data you use in your R code must be on the HPCC. It can be in your working directory or in a subfolder. The file path must be given when uploading it if it's not in the working directory. Any .stan text files or other files that are referenced must also be available

.sb file

The sbatch file contains information needed to submit a job. Here is an example for a model run with Stan. You will need to adapt, and some programs (JAGS) require different settings. If you are running code on the development node and not submitting a job, you don't need this file.

#!/bin/bash --login

# how long? Can specify up to 7 days in hours\ #SBATCH --time=168:00:00

# how much memory?\ #SBATCH --mem=20G

# specify nodes needed.\ #SBATCH --ntasks=1

# specify CPUs (or cores per task)\ # Match this to your number of chains\ #SBATCH --cpus-per-task=3

# email me\ #SBATCH --mail-type=FAIL,END\ #SBATCH

# change to current directory or specify file path\ cd $SLURM_SUBMIT_DIR

# export R_LIBS_USER=/mnt/home/leuenbe9/R_Lib/4.0.2-X11-20200622

# add necessary modules\ module purge\ module load GCC/11.2.0\ module load OpenMPI/4.1.1\ module load R/4.1.2\ R --no-environ

# run R commandline with the Rscript command\ Rscript Code/R/TryModel_ISM.R --vanilla

squeue -l $SLURM_JOB_ID

scontrol show job $SLURM_JOB_ID

Line endings


The HPCC requires LF line breaks to run. It cannot run with the standard Windows line breaks (CRLF). You can change each file through command line, but it's often easier to tell git to change it for you. To do so, create a text file call .gitattributes. Set a default, and then state which file types need to be converted. A * indicates any text file, *.R means that any file with the extension .R will be converted to LF line endings when uploaded to GitHub. It won't change the original file on your computer. You will need this .gitattributes file in every repository.

Here's example text for the .gitattributes file:

# Set the default behavior, in case people don't have core.autocrlf set.\ * text=auto

# Declare files that should always have LF line endings once uploaded\ # Customize for which types of files you need\ *.stan text eol=lf\ *.R text eol=lf\ *.sb text eol=lf

Save this file in your repository. Then use git add, git commit, and git push to upload it to your GitHub.

Here's additional information and ways to set it globally:

I think all you need to do to set it globally is type git config -- global core.autolf. I'm testing it now. We'll see if it works.

Manually changing

From your working directory, type file *. This command will provide details on each file. You may see with CRLF line terminators on some files. If you do, type dos2unix filename.extension for each file. You can do more than one at once dos2unix filename1.extension filename2.extension.

Running code

Development node

If you want to test or troubleshoot, you can run an R on a development node. ( dev-intel16, dev-intel18 dev-amd20, accessed by ssh when first logging onto the HPCC) Purge modules and then load the ones you need:

module purge\ module load GCC/11.2.0\ module load OpenMPI/4.1.1\ module load R/4.1.2

Then you can either enter R by typing R (type q() to quit), or by running an entire file by typing Rscript FileName.R

Submit batch job

From your working directory, type sbatch This will submit the job.

To check on your job, type sq. It will show as PENDING or RUNNING. If it ran into errors or finished, it will no longer show up.

.out file

Once your job is running, a SLURM-#######.out file will be created. Don't open it until the job is done. It will contain the output from R and your code, and will have some error messages if the job fails.

Common errors

Packages not loaded

Not all R packages or R versions are available on the HPCC. If something you need isn't available, contact ICER via Teams or making a ticket. Include the R version you want to use and the package names. They're pretty fast about getting packages loaded.

Contact ICER if needed

  1. Office Hours: Monday/Thursday 1-2 PM on Teams (maybe also in person?)
  2. ICER User Manual: (see especially the sections labeled: Job Scheduling by SLURM and Job Management by SLURM)