Installing Pytorch/Pytorch Lightning Using Anaconda
This guide will walk you through installing Pytorch and/or Pytorch Lighting using conda. It assumes you have already installed either Anaconda or Miniconda. See the guide on using conda for more.
Setup - Checking Python
If you installed conda on your own and not following our using conda guide, the HPCC may be trying to use the system python installation instead of your own. To test if this is the case, run
1 | |
/opt/software, you will need to run module unload Python. If the output starts with the path to your Anaconda or Miniconda installation, you don't need to do anything else.
Note
If you are affected by the above issue, you will have to run module unload Python every time you
wish to use your own python installation. You may wish to add the module unload Python command to your
$HOME/.bashrc file.
Installing Pytorch
Since Pytorch works best when using a GPU, it needs to be installed on a development node with a GPU. We recommend using dev-amd20-v100 for the latest hardware. Run
1 | |
Note
You will be restricted to running Pytorch on nodes with v100 GPUs. See the page on cluster resources and SLURM job specifications for more.
You will also need the CUDA compiler, so load this using our module system:
1 | |
It's best practice to use Conda environments to organize your Python packages. Create a new conda environment with the name pytorch run
1 | |
To switch to this new environment, run
1 | |
Now that you are on a GPU development node, have loaded the CUDA module, and activated your new environment, you can install Pytorch with the following command:
1 | |
Installing Pytorch Lightning
It's best to install Pytorch following the instructions above before installing Pytorch Lightning, or GPU-support may not function correctly.
After Pytorch has been installed, Pytorch Lightning can be installed to the same pytorch environment using
1 | |