Tuning Linux I/O Scheduler for SSDs

Linux gives you the option to select the I/O scheduler. The scheduler can be changed without rebooting too! You may be asking at this point, why would I ever want to change the I/O scheduler? Changing the scheduler makes sense when the overhead of optimizing the I/O (re-ordering I/O requests) is unnecessary and expensive. This setting should be fine-tuned per storage device. The best setting for an SSD will not be a good setting for an HDD.
The current I/O scheduler can be viewed by typing the following command:

mrice@host:~$ cat /sys/block/sda/queue/scheduler
noop anticipatory deadline [cfq]

The current I/O scheduler (in brackets) for /dev/sda on this machine is CFQ, Completely Fair Queuing. This setting became the default in kernel 2.6.18 and it works well for HDDs. However, SSDs don't have rotating platters or magnetic heads. The I/O optimization algorithms in CFQ don't apply to SSDs. For an SSD,  the NOOP I/O scheduler can reduce I/O latency and increase throughput as well as eliminate the CPU time spent re-ordering I/O requests. This scheduler typically works well on SANs, SSDs, Virtual Machines, and even fancy Fusion I/O cards.  At this point you're probably thinking "Ok, I'm sold! How do I change the scheduler already?"  You can use the echo command as shown below:

mrice@host:~$ echo noop | sudo tee /sys/block/sda/queue/scheduler

To see the change, just cat the scheduler again.

mrice@host:~$ cat /sys/block/sda/queue/scheduler
[noop] anticipatory deadline cfq 

Notice that noop is now selected in brackets. This change is only temporary and will reset back to the default scheduler, CFQ in this case, when the machine reboots. You need to edit the grub configuration in order to keep the setting permanently. However, this will change the I/O scheduler for all block devices. The problem is that NOOP is not a good choice for HDD. I'd only permanently change the setting if the machine only has SSDs.
On Grub 2

Edit: /etc/default/grub
Add "elevator=noop" to the GRUB_CMDLINE_LINUX_DEFAULT line.
sudo update-grub

At this point, you've changed the I/O scheduler to NOOP. How do you know if it made a difference? You could run a benchmark against it and compare the numbers (just remember to flush the file-system cache). The other way is to take a look at the output from iostat. I/O requests spend less time in the queue with the NOOP I/O scheduler. This can be seen with in the "await" field from iostat. Here's an example of a larger write operation with NOOP.

iostat -x 1 /dev/sda

Device:	sda
rrqm/s	0.00
wrqm/s	143819.00
r/s	6.00
w/s	2881.00
rkB/s	24.00
wkB/s	586800.00
avgrq-sz	406.53
avgqu-sz	0.94
await	0.33
r_await	3.33
w_await	0.32
svctm	0.11
%util	31.00