Skip to content

Warning

This is as a Lab Notebook which describes how to solve a specific problem at a specific time. Please keep this in mind as you read and use the content. Please pay close attention to the date, version information and other details.

LM Studio

LM Studio is a graphical user interface tool designed for developers and researchers to explore pre-trained LLMs. This tool typically requires quite a bit a ICER resources including multple CPUs, a GPU and a lot of memory. The following is instrucions for getting LM Studio Downloaded and running on the MSU HPCC.

Installation and Running LLM Studio on the HPCC.

This tutorial will show you how to download/install and run LLM studio ont he HPCC.

  1. Connect to the HPCC via an OnDemand using your MSU Login/password.
  2. Start an "Interactive Desktop" via the "Interactive Apps" Tab at the top of the page:
    • Ask for 4 hours
    • Ask for 10 cores
    • Ask for 300GB of Memory
    • Select checkbox "I would like to receive an email when the session starts" (this is helpful because it can take a while to start)
    • Select checkbox "Advanced Options" then ask for 1 GPU.
  3. When ready, Click the button to "Launch Interactive Desktop"
  4. From the "Menu" in the upper left select "Internet" and then open the "Firefox" app inside the online desktop.
  5. Navigate to http://lmstudio.ai and select "Download LM Studio for Linux" option.
  6. From the "Menu" in the upper left select "System Tools" then select "MATE Terminal"
  7. Load the CUDA module by typing module load CUDA in the terminal (this may not be required but it dosn't hurt anything).
  8. Inside the Terminal command line change to the "Downloads" directory by typing cd Downloads
  9. View that the downloaded file is present by typing ls
  10. Change permissions on the download file so it can be run by typing chmod 755 ./LM_Studio-0.2.25.AppImage (Note version number may be different from when these instructions were made).
  11. Run the downloaded file by typing ./LM_Studio-0.2.25.AppImage

Optimizing LM Studio to take advantage of the CPUs and GPUs

Once LM Studio is up and running inside the OnDemand Desktop you will be able to download lM models and try them out. However, the default settings do not take full advantage of the resources that we requested in step 2 above. Click on the small "gear" icon in the upper right of the LM Studio interface to open up settings and make the following changes (Notice that there is a virtical scroll bar and some of the following sections my be outside your current view).

  1. Find the "Inference Parameters" and set "CPU Threads" n_threads to 10 to match the number of requested cores.
  2. Find the "GPU Acceleration" section. Make sure "GPU Offload" checkbox is set and slide the bar to "Max".

NOTE: We are planning to wrap LM Studio in it's own dedicated OnDemand App but this should get most people started.