Allocating uninitialized file blocks

Posted Apr 19, 2012 9:48 UTC (Thu) by epa (subscriber, #39769)
Parent article: Allocating uninitialized file blocks

Maybe the filesystem needs a low-priority scrubber process that zeroes out unallocated blocks, and marks them as zeroed. They can then be allocated quickly later.

If enough already-zeroed blocks aren't available (or they aren't contiguous enough) then you still have to go to the work of initializing blocks when fallocate() is called. But if those blocks are never used and the file is then shrunk or deleted, the zeroed blocks are still available for next time so the work isn't entirely wasted.

Allocating uninitialized file blocks

Posted Apr 19, 2012 14:56 UTC (Thu) by vonbrand (subscriber, #4458) [Link] (9 responses)

The problem is that zeroing blocks is work, and besides SSDs (and RAIDs) shouldn't be written except when really needed.

Why not just split the range fallocated and just rewrite the affected blocks, not the whole range?

Allocating uninitialized file blocks

Posted Apr 19, 2012 17:01 UTC (Thu) by epa (subscriber, #39769) [Link]

Yeah, it's work, my point is that since most systems have idle periods and busy periods, it makes sense to do some of this work during idle periods so that it doesn't have to be done when the system (and the disk) is busy.

Allocating uninitialized file blocks

Posted Apr 19, 2012 18:40 UTC (Thu) by slashdot (guest, #22014) [Link]

If it's an SSD or a "smart" RAID array, use TRIM instead (if it's not supported, throw the stupid thing away and get a decent one).

Allocating uninitialized file blocks

Posted Apr 20, 2012 7:32 UTC (Fri) by Cyberax (✭ supporter ✭, #52523) [Link] (6 responses)

Modern SSDs have TRIM which 'zeroes' the TRIM-ed blocks automatically.

Allocating uninitialized file blocks

Posted Apr 20, 2012 19:31 UTC (Fri) by drag (guest, #31333) [Link] (5 responses)

But then you are depending on your hardware to have correct functionality.
Which is a very very unsafe assumption.

Anyways, I believe TRIM just tells them that the blocks are no longer being used. It's not a order to zero out the blocks. Without trim the SSD doesn't know if the file system still expects them to be used or not.

Allocating uninitialized file blocks

Posted Apr 20, 2012 20:17 UTC (Fri) by dlang (guest, #313) [Link]

trim says that the blocks are not being used. If there is an attempt to read from blocks that are not being used, the SSD returns all 0's (it doesn't actually read the block, because after the trim, that block doesn't actually exist on the flash media anymore)

If you were to take apart the drive and bypass the controller to read the flash chips directly, you would have a chance at recovering the data.

Allocating uninitialized file blocks

Posted Apr 20, 2012 21:33 UTC (Fri) by Cyberax (✭ supporter ✭, #52523) [Link] (3 responses)

>But then you are depending on your hardware to have correct functionality.

Its presence is indicated by the 'Deterministic read data after TRIM' capability (you can check it using "hdparm -I"). So it's not like you need to blindly trust your SDD.

>Anyways, I believe TRIM just tells them that the blocks are no longer being used. It's not a order to zero out the blocks.

With deterministic zeros one can also use it as a way to quickly erase blocks.

Besides, overwriting something on SDD in most cases would NOT actually overwrite it in the real hardware flash due to load balancing and remapping.

Allocating uninitialized file blocks

Posted Apr 20, 2012 21:39 UTC (Fri) by dlang (guest, #313) [Link] (2 responses)

unfortunately trim is not a fast command for many drives

Allocating uninitialized file blocks

Posted Apr 20, 2012 21:45 UTC (Fri) by Cyberax (✭ supporter ✭, #52523) [Link] (1 responses)

That depends. TRIM-ing 100Gb of data on my Intel SSD-based RAID takes little more than 2-3 seconds, it's way faster than writing zeros directly and more flash-friendly.

However, TRIM command can't be queued. So it probably makes no sense to use it for large allocations and/or to keep a pool of recently-trimmed pages for immediate small allocations.

Allocating uninitialized file blocks

Posted Apr 20, 2012 21:51 UTC (Fri) by dlang (guest, #313) [Link]

the fact that it can't be queued is a major performance hit.

The SSD does keep a pool of unused pages for new allocations. What trim does is it lets the SSD know that you no longer care about the data on that block, and so it can add the block back to that pool.

If the SSD runs out of this pool, writing slows drastically as it must first erase a block before it can write anything. If you are expecting to do a LOT of writing to a SSD, you may want to make sure that you partition it to something less than the advertised size so that that extra space will remain in the pool (this works as long as that extra space has never been written to, or is explicitly relased via a TRIM command)