DAX on BTT

Posted May 5, 2016 20:38 UTC (Thu) by phro (subscriber, #29295)
Parent article: DAX on BTT

Actually, what I said was that sector tearing doesn't usually happen on SSDs due to the nature of the FTL. Traditional storage, however, never guaranteed sector atomicity, but it usually does provide it. When you switch over to a block driver on top of pmem, it's possible that there will be increased risk of tripping over torn sectors (since it's just an interrupted memcpy). I don't have any numbers to back that up, presently. If you're wondering whether applications do rely on atomic sector updates, wonder no more! There is research that shows that some applications do, in fact, expect it [1].

The real issue is that DAX and the BTT are incompatible. If you want to use DAX, you have to give up sector atomicity. If applications truly depend on that, then they can't run unmodified on a pmem device mounted with -o dax. That means that you would have to separate out your pmem mount points into those that will support legacy applications and those that will only support DAX. By combining the two, you get the best of both worlds.

I got the overwhelming impression that the room was not convinced that applications should rely on atomic sector updates. Such applications are broken and should be fixed. Thus, there is little impetus to support the mixed DAX+BTT mode that was proposed.

[1] http://research.cs.wisc.edu/wind/Publications/alice-osdi1...

DAX on BTT

Posted May 5, 2016 21:11 UTC (Thu) by stellarhopper (subscriber, #84666) [Link] (1 responses)

Are you suggesting that we should indeed pursue DAX+BTT? One of the cons we also realized was that even if we do support this hybrid model, it will preclude DAX mappings of larger than a page (i.e. 2MB and 1GB) mappings, and the lost performance there is probably not worth the minor gains in convenience from the hybrid mode.

DAX on BTT

Posted May 6, 2016 14:48 UTC (Fri) by phro (subscriber, #29295) [Link]

I think breaking basic assumptions of applications is bad, yes. So, pursuing DAX+BTT is interesting in that respsect. I forgot about the limitation you described, however. That does put a rather serious monkeywrench in the works, but I think that it could be worked around. The question is whether anyone is willing to put the work in, and at this stage it's not clear whether it would be worth the effort.

DAX on BTT

Posted May 6, 2016 0:00 UTC (Fri) by neilbrown (subscriber, #359) [Link] (3 responses)

> Traditional storage, however, never guaranteed sector atomicity

citation needed.

My model of traditional storage includes a ECC for each block. So the options for a read after an aborted write are:
- old data
- new data
- read error (ECC reports an uncorrectable error)

How can you get a torn sector?

DAX on BTT

Posted May 6, 2016 0:15 UTC (Fri) by andresfreund (subscriber, #69562) [Link] (1 responses)

I'd argue that a read error is an atomicity problem.

DAX on BTT

Posted May 8, 2016 13:39 UTC (Sun) by robbe (guest, #16131) [Link]

Is there any basic device that guarantees no errors, ever? (Sure, you can build stacks of redundancy that make them less and less probable.)

FWIW, ECC does not guarantee detection of errors. I don’t know what the distance of the code used for your disk is (are these values universal?), so I can’t tell what the probability of an undetected error is.

DAX on BTT

Posted May 6, 2016 14:50 UTC (Fri) by phro (subscriber, #29295) [Link]

>> Traditional storage, however, never guaranteed sector atomicity

> citation needed.

I suppose "never" is a strong word. What I meant to say was that the SCSI and ATA standards did not say anything about power-fail write atomicity of a single sector. Because they did not standardize it, you cannot rely on it.