Thursday, 28 January 2010

Using the FIEMAP ioctl() to get file extents

The fiemap ioctl() allows user space programmes to glean the extent information from a file's inode in a fairly direct and fast way.

Mark Fasheh (Suse) has written up a detailed description of this ioctl() in a LWN article, and this is really worth reading.

I've put some quick-and-hacky sample code in my repository to dump out the file extents of given files. This code works in two phases.  Step 1, one allocates a fiemap structure and queries the number of extents a file has.   Step 2, one re-allocates the fiemap structure with enough space for the extent data to be filled in by the ioctl() and re-issues the ioctl() query to get the requested extent data.   Data such as the physical and logical block location and also the size of the extent and extent flags is returned for each extent.

Code such as ureadahead uses fiemap; ureadahead in fact re-probes with the fiemap ioctl() if it detects that the number of extents has increased - just in case the file is growing during the probe.

[ Update ]

It appears that cp has recently been modified to use FIEMAP to detect where holes are in a file to make copying more efficient. However, some users have reported seeing cp corruption issues - the root cause was that FIEMAP was not dealing with regions of a file that have been written to that don't have blocks yet assigned.  This is alarming, as the delayed allocation mechanism is used by modern filesystems.  This is described in more detail in a recent LWN article.

1 comment:

  1. Hi Colin,

    I have been google searching for fiemap IOCTL on linux. I stumbled on this page. Yeah! your code really works, and I can see the extents for ext3 mounted filesystems on Suse 11. Noticed that earlier versions of Suse does not support this ioctl.

    However, looks like it does not support NTFS mounted filesystem yet, wondering if you could give some help/pointers on where to get a similar functionality for NTFS extents reader on a linux box?

    -Grace Chen