Warning
This is as a Lab Notebook which describes how to solve a specific problem at a specific time. Please keep this in mind as you read and use the content. Please pay close attention to the date, version information and other details.
Lab Notebook --- Ignoring Local Python Libraries (2023-07-13)
Python will automatically populate the internal "sys.path" variable with the location of package libraries. One such location is the .local/lib folder of your home directory on HPCC, provided that the python folder there corresponds to python version being used (i.e. python3.6 for running any Python 3.6.x version). This can create conflicts with containerized software (such as Singularity containters) and (potentially) other python environments like conda by causing python to try and load a module from .local/lib instead of the intended source.
In principle, this should not cause a problem if .local/lib is low enough on the load order in sys.path, but in practice this is not always the case. Since there is not benefit to having these additional libaries in sys.path from running something Singularity, the issue can be avoided by excluding the .local/lib path from sys.path.
To have python ignore the .local/lib folder when populating sys.path, you can set the following environment variable:
1 | |
Example Useage: Running AlphaFold from a Singularity container
HPCC has versions of AlphaFold available as Singularity containers (such as /opt/software/alphafold/2.0.0/alphafold.sif). These containers run Python 3.8.x and if you have a .local/lib/python38 folder, you may face an error like the following:
Setting PYTHONNOUSERSITE as above should resolve this issue