Skip to content

Run elbencho I/O benchmarking

elbencho is an open-source, distributed I/O benchmarking tool designed to measure the performance of filesystems, object storage, and block devices. It is installed on the HPCC for system wide access for users. This page is a brief how-to for using elbencho to benchmark the performance of various filesystems on the HPCC.

Preparation

Load the module

module purge  
module load elbencho/3.0-23-GCC-12.2.0

Locate the filesystem for testing

Users need to decide which of the filesystems (home, research, or scratch space) will be used for the benchmarking. Make sure that you have enough quota on the selected filesystem.

Execution

Usage: 

elbencho [OPTIONS] PATH [MORE_PATHS]

For more details run 

elbencho -h

to get help, or run

elbencho --help-all
to get help for a complete list of options.

Examples

These two examples benchmark writing and reading files to your scratch space. The directory $SCRATCH/Elbencho_test needs to exist before starting (e.g., by running mkdir $SCRATCH/Elbencho_test).

To test a different filesystem, change the paths accordingly. Make sure that the spaces have sufficient quota before testing.

Write 4 large files

To test file writes, use the following command (options are explained below):

input
elbencho -w -b 4M -t 16 --direct -s 20g $SCRATCH/Elbencho_test/file{1..4}
output
OPERATION   RESULT TYPE         FIRST DONE   LAST DONE
=========== ================    ==========   =========
WRITE       Elapsed time     :    6m6.809s   6m25.007s
            IOPS             :          53          53
            Throughput MiB/s :         213         212
            Total MiB        :       78416       81920
---

Input options explanation:

-w
test write
-b 4M
the number of bytes to write in a single operation is 4M.
-t 16
the number of I/O worker threads is 16. (Default: 1)
--direct
use direct IO, no content caching.
-s 20g
file size is 20G.

Output explanation:

  • OPERATION: Type of operation (WRITE/READ). This example is WRITE type.

  • RESULT TYPE:

    • Elapsed time: Wall time of the test process
    • IOPS: Input/Output Operations Per Second
    • Throughput: Total data moved per second
    • Total MiB: Total data moved during the process

  • FIRST DONE: The aggregate performance until the fastest (first) thread finishes its work.

  • LAST DONE: The aggregate performance until the slowest (last) thread finishes its work.

elbencho displays two main aggregate result columns, which are crucial for understanding performance stability.

Random read from the given 4 files for max 20 seconds

To test random file reads, use the following command (options are explained below). Note that the files from the previous example need to exist:

input
elbencho -r -b 4k -t 16 --iodepth 16 --direct --rand --timelimit 20 $SCRATCH/Elbencho_test/file{1..4}
output
OPERATION   RESULT TYPE         FIRST DONE   LAST DONE
=========== ================    ==========   =========
READ        Elapsed time     :     20.317s     20.541s
            IOPS             :        7463        7381
            Throughput MiB/s :          29          28
            Total MiB        :         592         592
---

Input options explanation:

-r
test read.
-b 4k
the number of bytes to write in a single operation is 4k.
-t 16
the number of I/O worker threads is 16.
--iodepth 16
Depth of I/O queue per thread for asynchronous I/O is set as 16.
--direct
use direct IO, no content caching.
--rand
Read at random offsets.
--timelimit 20
20 seconds time limit for each benchmark phase.

Additional examples can be found online.