So you have purchased a new VPS (whether it is with Binary Lane or another provider), logged in with SSH and are now staring at your root shell. For many of us, the first question that comes to mind is How fast is my server?; followed quickly by How do I measure its performance?
In this article I will look at some specific methods of measuring the disk performance of your VPS.
What not to do
Chances are, you have seen this test before; perhaps even used it yourself. It is the obligatory dd test - here is one popular variety:
dd if=/dev/zero of=test bs=64k count=16k conv=fdatasync
This test is popular because dd is pre-installed on almost all Linux servers. While it is a simple way of seeing if something is broken (for example, if you see 10MB/sec than your server is overloaded) it has a number of problems:
- This is a single-threaded, sequential-write test. If you are running the typical web+database server on your VPS, the number is meaningless because typical services do not do long-running sequential writes.
- The amount of data written (1GB) is small; and hence can be strongly influenced by caching on the host server, or the host's RAID controller. (The conv=fdatasync only applies to the VPS, not the host).
- It executes for a very short period of time; just a few seconds on faster I/O subsystems. This isn't enough to get a consistant result.
- There's no read performance testing at all.
Measuring random IOPS with FIO
- Random reads, random writes, or a combination of both. Databases in particular will pull data from all over your disk - known as random access.
- 4 kilobyte blocks. Again, databases and many other programs will read very small chunks of data - 4 kilobytes is a good working estimate.
- Multiple threads. If your website has multiple visitors, your website will serve them all at the same time. We want our benchmark to simulate this behaviour of multiple things accessing the disk at once.
cd /root yum install -y make gcc libaio-devel || ( apt-get update && apt-get install -y make gcc libaio-dev </dev/null ) wget https://github.com/Crowd9/Benchmark/raw/master/fio-2.0.9.tar.gz ; tar xf fio* cd fio* make
With FIO compiled, we can now run some tests.
Random read/write performance
./fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=test --bs=4k --iodepth=64 --size=4G --readwrite=randrw --rwmixread=75
This will create a 4 GB file, and perform 4KB reads and writes using a 75%/25% (ie 3 reads are performed for every 1 write) split within the file, with 64 operations running at a time. The 3:1 ratio is a rough approximation of your typical database.
FIO will now tick along printing a summary as it goes, that looks like this:
Jobs: 1 (f=1): [m] [65.4% done] [161.5M/55417K /s] [41.4K/13.9K iops] [eta 00m:09s]
And eventually a full result output like this, with the numbers we want highlighted in yellow:
fio-2.0.9
Starting 1 process
Jobs: 1 (f=1): [m] [100.0% done] [164.4M/56926K /s] [42.8K/14.3K iops] [eta 00m:00s]
test: (groupid=0, jobs=1): err= 0: pid=32210: Wed Mar 13 16:16:04 2019
read : io=3073.4MB, bw=191118KB/s, iops=47779 , runt= 16467msec
write: io=1022.7MB, bw=63592KB/s, iops=15897 , runt= 16467msec
cpu : usr=21.29%, sys=69.69%, ctx=8450, majf=0, minf=4
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued : total=r=786784/w=261792/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=3073.4MB, aggrb=191117KB/s, minb=191117KB/s, maxb=191117KB/s, mint=16467msec, maxt=16467msec
WRITE: io=1022.7MB, aggrb=63591KB/s, minb=63591KB/s, maxb=63591KB/s, mint=16467msec, maxt=16467msec
Disk stats (read/write):
vda: ios=784951/261218, merge=0/23, ticks=241180/83160, in_queue=313312, util=92.35%
This tests shows that Binary Lane's SSD on this host performs 34596 read operations per second and 11527 write operations per second under a 75% read load, which is typical of SATA SSD.
Random read performance
To measure random reads, use slightly altered FIO command:
./fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=test --bs=4k --iodepth=64 --size=4G --readwrite=randread
Again, we pull the iops from the result:
fio-2.0.9
Starting 1 process
Jobs: 1 (f=1): [r] [100.0% done] [336.5M/0K /s] [86.2K/0 iops] [eta 00m:00s]
test: (groupid=0, jobs=1): err= 0: pid=32172: Wed Mar 13 16:11:11 2019
read : io=4096.0MB, bw=342980KB/s, iops=85745 , runt= 12229msec
cpu : usr=21.43%, sys=76.05%, ctx=5387, majf=0, minf=68
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued : total=r=1048576/w=0/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=4096.0MB, aggrb=342980KB/s, minb=342980KB/s, maxb=342980KB/s, mint=12229msec, maxt=12229msec
Disk stats (read/write):
vda: ios=1043901/20, merge=0/11, ticks=166560/4, in_queue=159628, util=96.92%
This tests shows Binary Lane's storage scoring 85745 read operations per second, a typical result for SATA SSD. This is higher than our mixed read/write result because writes are not being performed, freeing up disk resources to execute the reads being tested.
Random write performance
Again, just modify the FIO command slightly so we perform randwrite instead of randread:
./fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=test --bs=4k --iodepth=64 --size=4G --readwrite=randwrite
Again, we pull the iops from the result:
fio-2.0.9
Starting 1 process
Jobs: 1 (f=1): [w] [100.0% done] [0K/195.3M /s] [0 /49.1K iops] [eta 00m:00s]
test: (groupid=0, jobs=1): err= 0: pid=32203: Wed Mar 13 16:14:28 2019
write: io=4096.0MB, bw=200790KB/s, iops=50197 , runt= 20889msec
cpu : usr=19.10%, sys=62.43%, ctx=8113, majf=0, minf=5
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued : total=r=0/w=1048576/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
WRITE: io=4096.0MB, aggrb=200790KB/s, minb=200790KB/s, maxb=200790KB/s, mint=20889msec, maxt=20889msec
Disk stats (read/write):
vda: ios=0/1040658, merge=0/25, ticks=0/487300, in_queue=452284, util=89.27%
This tests shows Binary Lane's SSD scoring 50197 write operations per second.
Measuring latency with IOPing
cd /root yum install -y make gcc libaio-devel || ( apt-get update && apt-get install -y make gcc libaio-dev </dev/null ) wget https://ioping.googlecode.com/files/ioping-0.6.tar.gz ; tar xf ioping* cd ioping* make
We can now run ioping to get an idea of the per-request latency:
./ioping -c 10 .
4096 bytes from . (ext3 /dev/vda1): request=1 time=0.2 ms
4096 bytes from . (ext3 /dev/vda1): request=2 time=0.3 ms
4096 bytes from . (ext3 /dev/vda1): request=3 time=0.3 ms
4096 bytes from . (ext3 /dev/vda1): request=4 time=0.3 ms
4096 bytes from . (ext3 /dev/vda1): request=5 time=0.3 ms
4096 bytes from . (ext3 /dev/vda1): request=6 time=0.3 ms
4096 bytes from . (ext3 /dev/vda1): request=7 time=0.3 ms
4096 bytes from . (ext3 /dev/vda1): request=8 time=0.2 ms
4096 bytes from . (ext3 /dev/vda1): request=9 time=0.4 ms
4096 bytes from . (ext3 /dev/vda1): request=10 time=0.3 ms
--- . (ext3 /dev/vda1) ioping statistics ---
10 requests completed in 9006.8 ms, 3505 iops, 13.7 mb/s
min/avg/max/mdev = 0.2/0.3/0.4/0.1 ms
Here you can see the average was 0.3ms. On a healthy system, you should see relatively low variation and an average below 1.0 ms.
Final Thoughts
I hope this article has added a few more tools to your toolbelt and if nothing else, helped to dispell the idea that dd should be used for benchmarking.