Migrating R Packages

As part of ICER's OS upgrade, the system libraries and available versions of R have changed. This will may require R packages to be reinstalled. To do so, please follow the steps below.

Optional

In limited testing, some R packages installed on the current system appear to work on the new system without intervention. However, packages that have external dependencies will need to account for new module names, and will likely need to be reinstalled. Additionally, packages installed with an older version of R than is available in the new module system will need to be reinstalled.

Save your currently installed packages

First, start a backwards compatibility container, and start the version of R you are used to using on the HPCC. For the purposes, of this example, we will use R/4.2.1.

ssh user@hpcc.msu.edu
ssh dev-amd20
old_os
module purge
module load GCC/11.3.0 OpenMPI/4.1.4 R/4.2.1
R

Within R, save your installed packages to a file and exit.

packages <- installed.packages()[,"Package"]
save(packages, file = "Rpackages")
q()

Backup your previous library

To avoid conflicts between packages installed on different operating systems, move your old packages to a different location. Usually, your packages are stored in subdirectories of ~/R/x86_64-pc-linux-gnu-library labeled by the version of R they were installed with, but if you would like to double check, you can run R and look at the first entry of

.libPaths()

Move this library to a backup location so you can continue to use the CentOS operating system if necessary. Run this command from the command line.

mv ~/R/x86_64-pc-linux-gnu-library ~/R/x86_64-pc-linux-gnu-library.centos_backup

Reinstall packages with a version of R on the new operating system

Now, exit the container back into a development node running the new operating system. Load the version of R you would like to use. ICER highly recommends using R-bundle-CRAN/2023.12-foss-2023a, as this already contains a large number packages users often need, and will reduce the number you need to install manually.

# Exit backwards compatibility container
exit
# Load desired version of R
module purge
module load R-bundle-CRAN/2023.12-foss-2023a
R

Then reinstall the packages listed in your Rpackages file that are not already installed.

load("Rpackages")
new_packages <- setdiff(packages, installed.packages()[,"Package"])
for(p in new_packages)
install.packages(p)

If you would like to check what packages will be installed before running the install.packages() for loop, you can look at new_packages.

Your packages will now be ready to use with this new version of R.

Update old batch scripts

Assuming these packages were used in a batch script, you now need to update it for the new system. Suppose we have the following script, my_script.sb.

#/bin/bash --login

#SBATCH --time=01:00:00
#SBATCH --cpus-per-task=2
#SBATCH --mem-per-cpu=500MB

module load GCC/11.3.0 OpenMPI/4.1.4 R/4.2.1
Rscript do_some_work.R

Copy the script for the new system.

cp my_script.sb my_script_ubuntu.sb

Then, change to using the new version of R.

#/bin/bash --login

#SBATCH --time=01:00:00
#SBATCH --cpus-per-task=2
#SBATCH --mem-per-cpu=500MB

# Make sure we have the right version of R loaded
module purge
module load R-bundle-CRAN/2023.12-foss-2023a

Rscript do_some_work.R

To test, make sure you log into an Ubuntu development node, and submit the job using

sbatch my_script_ubuntu.sb

Optional: Going back to the old system

In the event that you need to rerun your R code in a CentOS backwards compatibility container, R will not find your old libraries that you have moved to ~/R/x86_64-pc-linux-gnu-library.centos_backup. You have two options:

Tell R where your packages are using an environment variable

This option works for any R code run after the R_LIBS_USER environment variable has been set.

#/bin/bash --login

#SBATCH --time=01:00:00
#SBATCH --cpus-per-task=2
#SBATCH --mem-per-cpu=500MB

module load GCC/11.3.0 OpenMPI/4.1.4 R/4.2.1
export R_LIBS_USER=~/R/x86_64-pc-linux-gnu-library.centos_backup
Rscript do_some_work.R

Note that even though the R_LIBS_USER variable is set in the script, you can also set it from the command line before starting R interactively.

Tell R where your packages are in your R code

This option requires modification to your R code, but is more flexible. Before you run any library() commands in an R script or interactively, run the R command

.libpaths("~/R/x86_64-pc-linux-gnu-library.centos_backup")