LWN: Comments on "Allocating uninitialized file blocks"

Allocating uninitialized file blocks

butlerm — Tue, 01 May 2012 05:50:18 +0000

It sounds like EXT4 and any other modern filesystems with this problem need a more robust form of unwritten extent tracking similar to that used by XFS.

Allocating uninitialized file blocks

szaka — Mon, 23 Apr 2012 01:21:10 +0000

The flag (with a better name) could be helpful for filesystems which can't fully support uninitialized allocated blocks efficiently. We are supporting several such interoperable filesystems (NTFS, exFAT, FAT) where changing the specification is not possible.

There is real user need despite explaining potential security consequences. Typical usage scenarios are using a large file as a container for an application which tracks free/used blocks itself. Windows supports this feature by SetFileValidData() if extra privilege is granted.

The performance gain can be huge on embedded using low-end storage and SoC. In one of our cases it took 5 days vs 12 minutes to fully setup a large file for use.

Allocating uninitialized file blocks

dlang — Fri, 20 Apr 2012 21:51:49 +0000

the fact that it can't be queued is a major performance hit.

The SSD does keep a pool of unused pages for new allocations. What trim does is it lets the SSD know that you no longer care about the data on that block, and so it can add the block back to that pool.

If the SSD runs out of this pool, writing slows drastically as it must first erase a block before it can write anything. If you are expecting to do a LOT of writing to a SSD, you may want to make sure that you partition it to something less than the advertised size so that that extra space will remain in the pool (this works as long as that extra space has never been written to, or is explicitly relased via a TRIM command)

Allocating uninitialized file blocks

Cyberax — Fri, 20 Apr 2012 21:45:49 +0000

That depends. TRIM-ing 100Gb of data on my Intel SSD-based RAID takes little more than 2-3 seconds, it's way faster than writing zeros directly and more flash-friendly.

However, TRIM command can't be queued. So it probably makes no sense to use it for large allocations and/or to keep a pool of recently-trimmed pages for immediate small allocations.

Allocating uninitialized file blocks

dlang — Fri, 20 Apr 2012 21:39:31 +0000

unfortunately trim is not a fast command for many drives

Allocating uninitialized file blocks

Cyberax — Fri, 20 Apr 2012 21:33:19 +0000

>But then you are depending on your hardware to have correct functionality.

Its presence is indicated by the 'Deterministic read data after TRIM' capability (you can check it using "hdparm -I"). So it's not like you need to blindly trust your SDD.

>Anyways, I believe TRIM just tells them that the blocks are no longer being used. It's not a order to zero out the blocks.

With deterministic zeros one can also use it as a way to quickly erase blocks.

Besides, overwriting something on SDD in most cases would NOT actually overwrite it in the real hardware flash due to load balancing and remapping.

Allocating uninitialized file blocks

dlang — Fri, 20 Apr 2012 20:17:42 +0000

trim says that the blocks are not being used. If there is an attempt to read from blocks that are not being used, the SSD returns all 0's (it doesn't actually read the block, because after the trim, that block doesn't actually exist on the flash media anymore)

If you were to take apart the drive and bypass the controller to read the flash chips directly, you would have a chance at recovering the data.

Allocating uninitialized file blocks

drag — Fri, 20 Apr 2012 19:31:26 +0000

But then you are depending on your hardware to have correct functionality.
Which is a very very unsafe assumption.

Anyways, I believe TRIM just tells them that the blocks are no longer being used. It's not a order to zero out the blocks. Without trim the SSD doesn't know if the file system still expects them to be used or not.

Allocating uninitialized file blocks

Cyberax — Fri, 20 Apr 2012 07:32:08 +0000

Modern SSDs have TRIM which 'zeroes' the TRIM-ed blocks automatically.

Allocating uninitialized file blocks

jzbiciak — Fri, 20 Apr 2012 05:41:07 +0000

Well, this new fallocate() feature is explicit. It's not like we're suddenly changing the semantics of holes in files. It's really about policy vs. mechanism. We need to ask if this is a useful mechanism, and if so, can user space use it safely if it adopts appropriate polices?

The definition of "appropriate policy" depends entirely on the usage scenario and security requirements of the system and application. A DVR disk that has nothing but video files on it won't leak anything interesting, so this fallocate() mode may be perfectly suited to it, for example, assuming a bittorent-style scattered download.

All that said, this new mode does need to prove its usefulness. If the performance issue is unique to ext4, then it's probably better to just fix ext4.

Allocating uninitialized file blocks

Fowl — Thu, 19 Apr 2012 21:22:55 +0000

That would be the SetFileValidData function, which "is useful in very limited scenarios. "

To use it requires the SE_MANAGE_VOLUME_NAME privilege (ie. raw disk access).

"Applications should call SetFileValidData only on files that restrict access to those entities that have SE_MANAGE_VOLUME_NAME access. The application must ensure that the unwritten ranges of the file are never exposed, or security issues can result as follows."

[Something about giving just enough rope to hang yourself / trusting programmers to know what they're doing.]

Allocating uninitialized file blocks

intgr — Thu, 19 Apr 2012 20:51:26 +0000

> that used the equivilant option in Windows

Are you saying that there's an equivalent to the new proposed FALLOC_FL_NO_HIDE_STALE flag too? I'd be surprised. Can you link to the discussion?

(Maybe you're just confused. I'm sure there is a fallocate() equivalent feature in Windows -- but fallocate() doesn't leak any foreign file data, without this new proposed flag)

Allocating uninitialized file blocks

nybble41 — Thu, 19 Apr 2012 19:28:25 +0000

The problem is that while a root process or one with raw I/O capabilities can see those blocks itself, it wouldn't usually write them out to a file which other users could read. However, if a root process allocates space for a file readable by non-root processes, and that space remains uninitialized, the other processes will have access to the former contents of those blocks.

Allocating uninitialized file blocks

slashdot — Thu, 19 Apr 2012 18:40:38 +0000

If it's an SSD or a "smart" RAID array, use TRIM instead (if it's not supported, throw the stupid thing away and get a decent one).

Allocating uninitialized file blocks

epa — Thu, 19 Apr 2012 17:01:40 +0000

Yeah, it's work, my point is that since most systems have idle periods and busy periods, it makes sense to do some of this work during idle periods so that it doesn't have to be done when the system (and the disk) is busy.

Allocating uninitialized file blocks

joey — Thu, 19 Apr 2012 15:44:24 +0000

I was just reading yesterday about a torrent program that used the equivilant option in Windows. So every partially downloaded torrent file exposes other "deleted" data. Ugh.

Allocating uninitialized file blocks

jzbiciak — Thu, 19 Apr 2012 15:35:15 +0000

How is this at all different than just opening up the device node associated with the partition? If you're root, what's stopping you?

As Ted Ts'o suggested, if a process has the capability to do raw I/O, why not let it see raw disk blocks occasionally? You've already given it permission to do low level I/O in spite of a filesystem, so what's the harm in letting it stale blocks within a filesystem?

Allocating uninitialized file blocks

sandeen — Thu, 19 Apr 2012 15:21:30 +0000

As the thread wore on, numbers were posted which show XFS has no serious performance degradation under the same workload. At this point, blowing a security hole in ext4 and promoting the flag to the VFS level seems incredibly premature.

I'd look into just what is making ext4 slow here but so far I can't reproduce a slowdown anything like what the patch submitter has seen...

-Eric

Allocating uninitialized file blocks

vonbrand — Thu, 19 Apr 2012 14:56:37 +0000

The problem is that zeroing blocks is work, and besides SSDs (and RAIDs) shouldn't be written except when really needed.

Why not just split the range fallocated and just rewrite the affected blocks, not the whole range?

Allocating uninitialized file blocks

epa — Thu, 19 Apr 2012 09:48:11 +0000

Maybe the filesystem needs a low-priority scrubber process that zeroes out unallocated blocks, and marks them as zeroed. They can then be allocated quickly later.

If enough already-zeroed blocks aren't available (or they aren't contiguous enough) then you still have to go to the work of initializing blocks when fallocate() is called. But if those blocks are never used and the file is then shrunk or deleted, the zeroed blocks are still available for next time so the work isn't entirely wasted.

Allocating uninitialized file blocks

slashdot — Thu, 19 Apr 2012 07:15:14 +0000

This is something that should never be done unless the benefit is truly huge it is mathematically provable that there's no other way to achieve it.

Even if an application has root privileges, a bug in it (even just not zeroing out padding in a structure) may result in an accidental leak of data from a totally unrelated file to a remote client.

The only sensible variant is zeroing blocks from deleted files in the background and only using those scrubbed blocks.