Cluster resources
HPCC maintains several clusters. They are named according to the year of installation. Each cluster has very similar hardware with specific processors but has some variety in configuration, such as different coprocessors, memory capacity, or number of CPUs. However, with many different kinds of configurations, HPCC uses a single-queue system managed by SLURM, a resource management software. Jobs submitted to SLURM job queue can run on any possible nodes, unless there are specifications on cluster constraints. Users only have to specify resource requirements and our scheduler can assign your job to an appropriate cluster.
All clusters currently run Ubuntu 22.04 and use the SLURM resource manager. They are connected to each other with file systems through Infiniband. Your home, research, and scratch is available and identical on all nodes.
The following table lists nodes that are currently available to run jobs; jobs can be submitted from any of our development nodes. Please note that you may need to scroll the table to see all columns.
| Cluster Type | Node Count | Processors (Clock Speed) | Cores | Memory | Disk Size | GPUs (Number) | Node Name |
|---|---|---|---|---|---|---|---|
| amd22 | 48 | AMD EPYC 7763 Processor (2.445 GHz) | 128 | 493 GB | 412 GB | acm-[000-047] | |
| 18 | AMD EPYC 7763 Processor (2.445 GHz) | 128 | 996 GB | 412 GB | acm-[050-067] | ||
| 6 | AMD EPYC 7763 Processor (2.445 GHz) | 128 | 2005 GB | 412 GB | acm-[048,049,068-071] | ||
| amd21 | 11 | AMD EPYC 7713 Processor (2.0 GHz) | 128 | 512 GB | 1.92 TB | A100 SXM with 81920 MB memory(4) | nal-[000-010] |
| intel21 | 6 | Intel Xeon 8358 (2.6 GHz) | 64 | 256 GB | 1.92 TB | A100 PCIe with 40960 MB memory(4) | nif-[000-005] |
| amd20 | 207 | AMD EPYC 7H12 Processor (2.595 GHz) | 128 | 493 GB | 412 GB | amr-[000-101], amr-[136-238], amr-[252-253] | |
| 45 | AMD EPYC 7H12 Processor (2.595 GHz) | 128 | 996 GB | 412 GB | amr-[104-135], amr-[239-251] | ||
| 2 | AMD EPYC 7H12 Processor (2.595 GHz) | 128 | 2005 GB | 412 GB | amr-[102-103] | ||
| amd20-v100 | 20 | Intel(R) Xeon(R) Platinum 8260 CPU (2.40 GHz) | 48 | 178 GB | 412 GB | v100s with 32768 MB memory(4) | nvf-[000-016], nvf-[018-020] |
| 1 | Intel(R) Xeon(R) Platinum 8260 CPU (2.40 GHz) | 48 | 565 GB | 412 GB | v100s with 32768 MB memory(4) | nvf-017 | |
| intel18 | 112 | Intel(R) Xeon(R) Gold 6148 CPU (2.40 GHz) | 40 | 83 GB | 413 GB | skl-[000-011,013-112] | |
| 19 | Intel(R) Xeon(R) Gold 6148 CPU (2.40 GHz) | 40 | 178 GB | 413 GB | skl-[113-131] | ||
| 28 | Intel(R) Xeon(R) Gold 6148 CPU (2.40 GHz) | 40 | 367 GB | 413 GB | skl-[132-139,148-167] | ||
| 8 | Intel(R) Xeon(R) Gold 6148 CPU (2.40 GHz) | 40 | 745 GB | 413 GB | skl-[140-147] | ||
| 1 | Intel(R) Xeon(R) Gold 6148 CPU (2.40 GHz) | 40 | 996 GB | 413 GB | skl-012 | ||
| intel18-v100 | 8 | Intel(R) Xeon(R) Gold 6148 CPU (2.40 GHz) | 40 | 367 GB | 413 GB | v100 with 32768 MB memory (8) | nvl-[000-007] |
| intel16 | 26 | Intel(R) Xeon(R) CPU E5-2680 v4 (2.40 GHz) | 28 | 492 GB | 190 GB | lac-[250-253,256-261,302-317] | |
| 39 | Intel(R) Xeon(R) CPU E5-2680 v4 (2.40 GHz) | 28 | 240 GB | 190 GB | lac-[224-225,228-248, 278-285,294-301] | ||
| 313 | Intel(R) Xeon(R) CPU E5-2680 v4 (2.40 GHz) | 28 | 115 GB | 190 GB | lac-[000-023,032-191,200-223, 254,255,276,277,318-341, 350-369,372,372-445] | ||
| intel16-k80 | 48 | Intel(R) Xeon(R) CPU E5-2680 v4 (2.40 GHz) | 28 | 240 GB | 190 GB | k80 with 12206 MB memory (8) | lac-[024-031,080-087,136-143, 192-199,286-293,342-349] |
| intel16-xl | 2 | Intel(R) Xeon(R) CPU E7-8867 v3 (2.50 GHz) | 64 | 2.93 TB | 860 GB | vim-[000,001] | |
| 1 | Intel(R) Xeon(R) CPU E7-8867 v4 (2.40 GHz) | 144 | 5.86 TB | 3.7 TB | vim-002 | ||
| intel14 | 18 | Intel(R) Xeon(R) CPU E5-2670 v2 (2.50 GHz) | 20 | 240 GB | 416 GB | csm (11 nodes) & css (10 nodes) | |
| 8 | Intel(R) Xeon(R) CPU E5-2670 v2 (2.50 GHz) | 20 | 115 GB | 416 GB | css-[002-003,020,023,032-035] | ||
| 65 | Intel(R) Xeon(R) CPU E5-2670 v2 (2.50 GHz) | 20 | 52 GB | 416 GB | css nodes | ||
| intel14-k20 | 37 | Intel(R) Xeon(R) CPU E5-2670 v2 (2.50 GHz) | 20 | 115 GB | 416 GB | k20 with 4743 MB memory (2) | csn-[001-039] |
| intel14-phi | 8 | Intel(R) Xeon(R) CPU E5-2670 v2 (2.50 GHz) | 20 | 115 GB | 416 GB | Phi card (2) | csp-[006,016-020,025-026] |
| intel14-xl | 1 | Four Intel Xeon CPU E7-8857 v2 (3.00 GHz) | 48 | 969 GB | 1.8 TB | qml-003 | |
| 1 | Four Intel Xeon CPU E7-8857 v2 (3.00 GHz) | 48 | 1.45 TB | 897 GB | qml-[002] | ||
| 1 | Four Intel Xeon CPU E7-8857 v2 (3.00 GHz) | 48 | 2.93TB | 1.1 TB | qml-000 |