The Linux kernel I/O schedulers attempt to balance the need to get the best possible I/O performance while also trying to ensure the I/O requests are "fairly" shared among the I/O consumers. There are several I/O schedulers in Linux, each try to solve the I/O scheduling issues using different mechanisms/heuristics and each has their own set of strengths and weaknesses.
For traditional spinning media it makes sense to try and order I/O operations so that they are close together to reduce read/write head movement and hence decrease latency. However, this reordering means that some I/O requests may get delayed, and the usual solution is to schedule these delayed requests after a specific time. Faster non-volatile memory devices can generally handle random I/O requests very easily and hence do not require reordering.
Balancing the fairness is also an interesting issue. A greedy I/O consumer should not block other I/O consumers and there are various heuristics used to determine the fair sharing of I/O. Generally, the more complex and "fairer" the solution the more compute is required, so selecting a very fair I/O scheduler with a fast I/O device and a slow CPU may not necessarily perform as well as a simpler I/O scheduler.
Finally, the types of I/O patterns on the I/O devices influence the I/O scheduler choice, for example, mixed random read/writes vs mainly sequential reads and occasional random writes.
Because of the mix of requirements, there is no such thing as a perfect all round I/O scheduler. The defaults being used are chosen to be a good best choice for the general user, however, this may not match everyone's needs. To clarify the choices, the Ubuntu Kernel Team has provided a Wiki page describing the choices and how to select and tune the various I/O schedulers. Caveat emptor applies, these are just guidelines and should be used as a starting point to finding the best I/O scheduler for your particular need.
