Buy-In Accounts with SLURM
If you want to have priority access to our clusters, you can purchase buy-in nodes. Users who run on buy-in nodes receive priority access to their nodes within 4 hours and are exempt from the 1 million CPU hour per year limit.
Types of Partitions
There are three types of partitions configured in SLURM.
|Buyin Partition (names vary)||A partition is created for each buy-in account. Each buy-in partition includes all non-buy-in nodes, allowing buy-in jobs to span buy-in and non-buy-in nodes. These jobs get equal consideration for scheduling on non-buy-in nodes as jobs in general-long. When jobs submitted to these partitions request a wall time of four hours or less, they are also submitted to the general-short, enabling them to use other available buy-in nodes and ensuring they are scheduled as fast as possible.|
|general-short||This partition includes all nodes--buy-in and non-buy-in--and runs jobs that request a wall time of four hours or less. Jobs in this partition are considered for scheduling after jobs in buy-in partitions and the general-long partition. To prevent these jobs from being continuously bumped by general-long/buy-in jobs, they are also submitted to general-long. Jobs with a requested wall time of four hours or less are automatically submitted to this partition.|
|general-long||This partition includes non-buy-in nodes and allows jobs to run for up to seven days. Jobs with a requested wall time of four hours or less are also submitted to the general-short partition.|
|general-long-bigmem||This partition includes non-buy-in nodes with more memory and CPU cores than most nodes and allows jobs to run for up to seven days. Jobs requesting more than 256GB or 40 CPUs per node are automatically submitted to this partition. This partition ensures large jobs get priority access to large nodes over jobs that can run elsewhere. Jobs with a requested wall time of four hours or less are also submitted to the general-short partition.|
|general-long-gpu||This partition contains non-buy-in nodes with GPUs and allows jobs to run for up to seven days. Jobs requesting GPUs are automatically submitted to the partition. This partition ensures jobs requesting GPUs get priority access to nodes with GPUs over jobs not requesting GPUs. Jobs with a requested wall time of four hours or less are also submitted to the general-short partition.|
In most cases, you will not need to specify a partition when submitting a job. Given a specified (or default) account, SLURM's job submit plugin will assemble the partition list automatically.
When submitting a job without specifying an account, your default
account is used. You can check your default account using the
buyin_status power tool.
1 2 3 4 5 6
fordste5 has a default account of
fordste5 submits a
job without specifying an account, it will be submitted to the
partition. If the job requested four hours or less of wall time, it will
be submitted to the
general-short partition as well. Jobs submitted by
users with a default account of
general will be queued in the
|Default Account||Wall Time<=4 Hours||Wall Time<=4 Hours and >=256GB or 40CPU||Wall Time<=4 Hours and GPUs Reqested||Wall Time>4 Hours||Wall Time>4 Hours and >=256GB or 40CPU||Wall Time>4 Hours and GPUs|
<buyin> is the name of buyin account.
Using Non-Default Accounts
You can see what accounts you have access to using the
1 2 3 4 5 6
This output shows that
fordste5 is a member of the
and classres buy-in accounts. Both
to partitions with the same name.
fordste5 has a default account of
classres, if they want to
submit jobs to
test1, they will have to explicitly specify the
account at submission. This is done by using either using the
srun/sbatch/salloc, or by adding the
#SBATCH -A test1
directive to their batch script.
Checking the Status of Buy-Ins
The status of buy-ins can be viewed with the
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
This tool will list all jobs that are queued for a buy-in account, all nodes associated with the buy-in account–including node status–and a list of jobs running on each node. Because jobs with a wall time of four hours or less can run on any buy-in node, it is possible to see jobs running on buy-in nodes from other accounts.
Buy-in partitions have the same name as a given buy-in account
Managing Buy-In Account Membership
SLURM allows for the configuration of account coordinators. Account coordinators can add and remove users to accounts that they coordinate. Buy-in account owners are configured as coordinators of their buy-in accounts and can request that other users also be added as coordinators.
If you had a buy-in account configured in the old Moab scheduler, account coordinators for your SLURM buy-in account were copied from the managers listed on the Moab account.
View what users have access to your buy-in account:
Add a user to your buy-in account:
Remove a user from your buy-in account: