Using the noop I/O scheduler

Over the past few days I've been tweaking the configuration on my HP Mini 1000 to see how to squeeze a little more performance out of this device. Today I enabled the NOOP I/O scheduler on the SSD as this improves performance a little.

Since the SSD is just a flash memory based device, it has minimal seek time and zero rotational delay, so it's sensible to disable the more complex default cfq I/O scheduler and just use the simpler NOOP scheduler. The NOOP scheduler just puts all I/O requests into an unordered First-In First Out (FIFO) queue, and can also do request merging. The rationale behind this is that I/O performance optimisation is performed by the block device such as the logic in the SSD controller.

Traditional hard disks have the overhead of head movement seek times and also rotational delay and it makes sense for these devices to try and group I/O requests together if they are physically close on the disk. However, with SSDs this does not make sense as these delays don't apply - the NOOP I/O scheduler is a better choice as it removes the grouping of I/O requests and hopefully reduces the variability in timing of I/O request completion.

To enable the NOOP I/O scheduler on SDD device /dev/sda, I used (as root):

echo noop > /sys/block/sda/queue/scheduler

..and of course this should be placed in the appropriate boot scripts.

Bonnie++, a useful benchmarking tool

When it comes to figuring out how a file system behaves on a HDD or SSD I generally first turn to Bonnie++ as a straight forward and easy to use disk benchmarking tool.

Bonnie performs a series of tests, covering character and block read/write I/O, database style I/O operations as well as tests for multiple file creation/deletion and some random access I/O operations.

To install bonnie for Ubuntu, simply do:

sudo apt-get install bonnie

To get a fair I/O benchmark (without the memory cache interfering with results) Bonnie generates test files that are at least twice the size of the memory of your machine. This can be a problem if you have servers with a lot of memory as the generated files can be huge and testing can therefore take quite a while. I generally boot a system and specify ~1GB of memory using the mem=1024M kernel parameter just so that Bonnie only tests with a 2GB file to speed my testing up.

My rule of thumb is to run Bonnie at least 3 times and take an average of the results. If you see a large variation in your results double check that there isn't some service running that is interfering with the benchmarking.

Bonnie references:

http://www.coker.com.au/bonnie++
http://linux.com/archive/feature/139742

Which I/O scheduler is best for SSD?

Linux provides several I/O schedulers that allow one to tune performance for your use patterns. Traditionally disk scheduling algorithms have been designed for rotational drives where rotational latency and drive head movement latency need to be taken into consideration, hence the need for complex I/O schedulers. However a SSD does not suffer from the HDD latency issues - does this mean the default I/O scheduler choice needs to be re-thought?

I decided to benchmark a SanDisk pSSD to see the behaviour of the different I/O schedulers using the following tests:

a) untar kernel source
b) sync
c) copy kernel source tree
d) sync
e) copy tar file
f) sync
g) rm -rf kernel source tree
h) rm -rf copy of source tree
i) sync

For each of the I/O schedulers I ran the above sequence of tests 3 times and took an average of the total run time. My results are as follows:

cfq: 3m59s,
noop: 3m25s,
anticipatory: 3m51s,
deadline: 3m42s

So it appears the default cfq (Completely Fair Queuing) scheduler is least optimal and the noop scheduler behaves best. According to the noop wikipedia entry the noop scheduler is the best choice for solid state drives, so maybe my test results can be trusted after all :-)

SSDs do not require multiple I/O requests to be re-ordered since they do not suffer from traditional random-access latencies, hence the noop scheduler is optimal.

RAID0 + SSD = speed!

I've been messing around with Ubuntu Jaunty Desktop installed on two SanDisk 32GB pSSD devices today with a RAID0 configuration and I'm really impressed with the performance. (Thanks to SanDisk for providing me with some early samples to test!)

RAID0 stripes data across the two SSDs which should theoretically double I/O rates. It also allows me to effectively create a larger disk volume by treating both drives as one larger combined device. The downside is that if one SSD fails, I have zero redundancy (unlike RAID1) and hence can lose my data.

To avoid controller saturation, I configured the SSDs so that each SSD used a different SATA controller; I believe this helps, but I've not done any serious benchmarking to verify this configuration assumption.

I used the SoftwareRAID wiki page as a quick start guide to get my head around how to configure my system; I created a small /boot partition and then a 30GB and ~2GB RAID0 volumes for / (using ext4) and swap respectively on each drive and then installed the 64 bit Jaunty Desktop using the alternative installer ISO image.

Performance just rocked! I benchmarked block I/O performance using bonnie and got block write rates of ~73MB/s and reads of ~114MB/s.

A quick test with dd writing 8GB of zeros in 4K blocks on my machine hit ~102MB/s write rate - quite astounding. I managed to get a sustained read rate of ~119MB/s read rate in 4K blocks reading a 16GB file - that's just awesome!

Boot time was impressive too. On my dual core 2.0GHz desktop I booted in ~12-14 seconds after grub stage without any optimization tweaks.

If I get some more spare time I will try and figure out how to get the partitions aligned to 128K block boundary to see how much more performance I can squeeze out of these SSDs. Watch this space!