Skip to content

Singularity Introduction

Note

This tutorial is adapted from the Container Camp Tutorial produced and copyrighted by CyVerse.

Singularity allows users to run software inside of containers. Another popular container system is Docker, which is interoperable with Singularity. Singularity provides many features that make it well suited for HPC applications, and therefore, it is the container system that is installed on our HPCC.

Installation

To install Singularity on your local machine, please see the installation instructions in the documentation. Singularity can only be run natively on Linux, but if you need to run it locally on a Windows or Mac computer, you can use a virtual machine provided by the creators of Singularity. See these alternative instructions for details.

Note

As of May 2023, the version of Singularity on the MSU HPCC is currently 3.11. For any questions that this tutorial or the Advanced Topics page do not answer, please consult the official documentation for this version. All Singularity commands are built into the system such as singularity shell and singularity exec, which means you can invoke these commands directly from the command line.

Check installation

If you would like to check your installation of Singularity on your local machine or the installation on the HPCC, you can run

1
2
3
4
5
6
$ singularity pull shub://vsoch/hello-world
INFO:    Downloading shub image
59.75 MiB / 59.75 MiB      [========================================================================================] 100.00% 10.46 MiB/s 5s

$ singularity run hello-world_latest.sif
RaawwWWWWWRRRR!! Avocado!

In the above example, we used the Singularity Hub "unique resource identifier," or URI, "shub://" which tells the software to run an image from Singularity Hub. To get help, you can use the --help flag which gives a general overview of Singularity options and subcommands as follows:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
$ singularity --help

Linux container platform optimized for High Performance Computing (HPC) and
Enterprise Performance Computing (EPC)

Usage:
  singularity [global options...]

Description:
  Singularity containers provide an application virtualization layer enabling
  mobility of compute via both application and environment portability. With
  Singularity one is capable of building a root file system that runs on any
  other Linux system where Singularity is installed.

Options:
  -d, --debug     print debugging information (highest verbosity)
  -h, --help      help for singularity
  --nocolor   print without color output (default False)
  -q, --quiet     suppress normal output
  -s, --silent    only print errors
  -v, --verbose   print additional information
      --version   version for singularity
 ...

You can use the help command if you want to see the information about subcommands. For example, to see the pull command help,

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
$ singularity help pull

Pull an image from a URI

Usage:
singularity pull [pull options...] [output file] <URI>

Description:
The 'pull' command allows you to download or build a container from a given
URI. Supported URIs include:

library: Pull an image from the currently configured library
    library://user/collection/container[:tag]

docker: Pull an image from Docker Hub
    docker://user/image:tag

shub: Pull an image from Singularity Hub
    shub://user/image:tag

oras: Pull a SIF image from a supporting OCI registry
    oras://registry/namespace/image:tag

http, https: Pull an image using the http(s?) protocol
    https://library.sylabs.io/v1/imagefile/library/default/alpine:latest
...

Downloading pre-built images

We already downloaded a pre-built image "hello-world" from shub, one of the registries, using pull command. This is the easiest way to use Singularity.

You can use the pull command to download pre-built images from a number of Container Registries. Here we’ll be focusing on the Singularity Hub or Docker Hub. The following are some of container registries.

Pulling an images from Sylabs cloud library

In this example, I will pull a base Alpine container from Sylabs cloud:

1
2
3
$ singularity pull library://sylabsed/linux/alpine
INFO:    Downloading library image
 2.08 MiB / 2.08 MiB [===========================================================================================] 100.00% 4.74 MiB/s 0s

You can rename the container using the --name flag:

1
2
3
$ singularity pull --name my_alpine.sif library://sylabsed/linux/alpine
INFO:    Downloading library image
 2.08 MiB / 2.08 MiB [===========================================================================================] 100.00% 9.65 MiB/s 0s

The above example will save the image in the current directory as my_alpine.sif

Pulling an image from Docker hub

Many programs are available as Docker containers pre-built, and many of those are available on Docker Hub. For more details on using Docker containers with Singularity, see our section on Migrating from Docker to Singularity.

Here is a quick example of pulling an Alpine Docker container for use with Singularity.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
$ singularity pull docker://alpine
INFO:    Converting OCI blobs to SIF format
INFO:    Starting build...
Getting image source signatures
Copying blob df20fa9351a1 done
Copying config 0f5f445df8 done
Writing manifest to image destination
Storing signatures
2020/08/20 15:53:52  info unpack layer: sha256:df20fa9351a15782c64e6dddb2d4a6f50bf6d3688060a34c4014b0d9a752eb4c
INFO:    Creating SIF file...
INFO:    Build complete: alpine_latest.sif

Interacting with images

You can interact with images via the shell, exec, and run commands. To learn how to interact with images, let's first pull an image lolcow_latest.sif from the library.

1
singularity pull library://sylabsed/examples/lolcow

shell

The shell command allows you to spawn a new shell within your container and interact with it as if it is a virtual machine.

1
2
$ singularity shell lolcow_latest.sif
Singularity>

The change in prompt indicates that you have entered the container. Once inside of a container, you are the same user as you are on the host system.

1
2
Singularity> whoami
choiyj

You will also have access to a few directories that you can access outside the container, most notably, the directory you ran the container from and your home directory. Try running pwd and ls ~ inside the container to verify this.

Anything you write to these directories will stay around after you are done with the container. This is a significant difference from Docker, where the default is to close off the files in the container and you need to use "bind mounts" to manually connect file spaces.

To exit from a container, type exit.

1
2
Singularity> exit
$

exec

The exec command allows you to execute a custom command within a container by specifying the image file. For instance, to execute the cowsay program within the lolcow_latest.sif container:

1
2
3
4
5
6
7
8
9
$ singularity exec lolcow_latest.sif cowsay container camp rocks
______________________
< container camp rocks >
 ----------------------
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||

You can also use shell command to run the program in the container.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
$ singularity shell lolcow_latest.sif
Singularity> cowsay container camp rocks
 ______________________
< container camp rocks >
 ----------------------
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||

run

Singularity containers contain runscripts. These are predefined scripts which define the actions of a container when user runs it. The runscript can be performed with the run command, or simply by calling the container as though it were an executable.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
$ singularity run lolcow_latest.sif
 _________________________________________
/ You're ugly and your mother dresses you \
\ funny.                                  /
 -----------------------------------------
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||

Submitting a Singularity job

In general, running Singularity commands is the same as running any kind of program when you prepare your SLURM script.

In this case, you put your Singularity commands (singularity exec <image>.sif <command>) right after the sbatch directive lines you use to specify your job resources. If the program needs to use multiple threads/cores on a node, say 8, you would request 8 cores using #SBATCH --cpus-per-task=8 as you would do with any other sbatch script.

For situations where you are using MPI within the container (e.g., you would like to split your code over multiple nodes) or would like to use GPU resources, please see Using Singularity with MPI and GPUs on the Advanced Topics page.