Making FIEMAP and delayed allocation play well together

Posted Feb 24, 2011 21:20 UTC (Thu) by dougg (guest, #1894)
Parent article: Making FIEMAP and delayed allocation play well together

Been looking at the FIEMAP ioctl description in the kernel documentation. That should frighten away any user space folks thinking of using it. For a start it could define what an "extent" is.
Anyway I have a different angle. Will FIEMAP work when a file is opened O_DIRECT? What about when the file is a partition or a disk (with or without O_DIRECT)? When a SCSI disk is opened O_DIRECT the FIEMAP ioctl could map through to the SCSI GET LBA STATUS command. Most likely I'm just dreaming.

Making FIEMAP and delayed allocation play well together

Posted Feb 24, 2011 21:26 UTC (Thu) by corbet (editor, #1) [Link] (3 responses)

An extent is a group of blocks in a file laid out contiguously on disk by the filesystem. It's a filesystem concept, which is what is needed to answer your questions. O_DIRECT shouldn't change anything. If your file descriptor is for a partition or a block device, there's no filesystem, so FIEMAP will make no sense. And FIEMAP cannot possibly map to a low-level SCSI operation, since there is no filesystem knowledge at that level.

Making FIEMAP and delayed allocation play well together

Posted Feb 24, 2011 21:57 UTC (Thu) by razb (guest, #43424) [Link] (1 responses)

I have hacked raid5+xfs several years ago. from time to time I had this delayed allocation which was a huge headache. Question , can we prevent delayed allocation ?

Making FIEMAP and delayed allocation play well together

Posted Feb 25, 2011 16:32 UTC (Fri) by nix (subscriber, #2304) [Link]

Delayed allocation is a really useful technique to (among other things) keep fragmentation down and increase the size of contiguous writes hitting the disk to something closer to the umpty-megabytes-at-once which the disk would actually prefer. It's better fixed than ripped out, I'd say.

Making FIEMAP and delayed allocation play well together

Posted Feb 24, 2011 22:19 UTC (Thu) by dougg (guest, #1894) [Link]

cp seems to be using FIEMAP on the src file to detect sparseness (i.e. holes) so it doesn't waste time reading potentially a lot of zeros. Unix already has good support for generating a sparse dst (unless dst is being overwritten). Next consider 'cp /dev/sda /dev/sdb' (don't try that at home) with unmapped (aka trimmed) blocks on /dev/sda . The SCSI GET LBA STATUS command on /dev/sda would play the same role as FIEMAP.