DRAFT Feb 2019
These instructions are for using Ansys on the current HPCC environment that uses Slurm scheduler. They may be helpful for those who are transitioning from our previous Moab/Torque/PBS scheduler.
We are facing a temporary issue with Ansys licensing due to changes made over the many versions HPCC has installed. If you have license issues when Using Ansys, here is the work-around:
2. SSH Connect to any development node, remembering to add -X options.
3. Start this program on any dev node. it launches a GUI
- On the left side of the window are three buttons. Click the button "set License Preferences for User \<username>". A new window will open
5. select Release 19.2 in that new window and click OK
6. another window will open with tabs across the top and two options in the bottom that are the same for each tab. On the bottom, click the option for "Use a seperate license for each application". It doesn't matter which Tab you've slected (Solver/PrePost/etc). That setting should be the same for all tabs.
7. click OK, which closes that window.
8. In the original Ansys license utility, click File, and then "exit" to close it. This modifies the config file in your home directory.
9. Close any current sessions in which you running Ansys and start it again on any method (dev node, in 'salloc' interactive job etc). You should now be able to use the features you needed before.
Guidelines for scheduling parallel (MPI) jobs
Since moving to Slurm, batch scripts that had worked under Torque are no longer working. Here are some guidelines for requesting resources.
Use --ntasks instead of nodes
-–nodes= will request that number of unique computers,
but what most users want is the number of tasks across. Then in
addition, use number of tasks instead of number of nodes for -t
Don't forget to request memory
Request memory per task, and since the default is to have 1 cpu per
task, you can request memory using
Create a temporary file for node list
Inside the job, Fluent requires a file of a particular format ,and the slurm node file doesn't work. This seems to work
1 2 3 4 5 6
Example fluent Job script (using Intel compiler). Increase tasks and memory as needed
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
After you have logged into a development nodes with an X11 terminal (or use the webrdp gateway as described above), You may run ANSYS tools in parallel and interactively as follows.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
This solver uses a different hosts file format for the par-dist parameter. The following uses an example Definition file provided by Ansys 19.2.
The batch script will adapted the par-dist file depending on how you specify tasks and tasks-per-node (the example below does not specify tasks per node). Code is taken from https://secure.cci.rpi.edu/wiki/index.php?title=CFX.
CFX5 Solver Example sbatch
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32