extreemly informative and interesting, but... [LWN.net]

extreemly informative and interesting, but...

Posted Mar 29, 2009 5:37 UTC (Sun) by tstover (subscriber, #56283)
Parent article: Filesystems? *I'll* give ya fileysytems!

Every time I read threads like this, I always ask the same question: so how the heck do databases work? If you can't at least guarantee in-order write properties, what good is a log/journal?

(Log in to post comments)

extreemly informative and interesting, but...

Posted Mar 29, 2009 20:52 UTC (Sun) by cortana (subscriber, #24596) [Link]

Check out PostgreSQL's manual:

When the operating system sends a write request to the disk hardware, there is little it can do to make sure the data has arrived at a truly non-volatile storage area. Rather, it is the administrator's responsibility to be sure that all storage components ensure data integrity. Avoid disk controllers that have non-battery-backed write caches. At the drive level, disable write-back caching if the drive cannot guarantee the data will be written before shutdown.

extreemly informative and interesting, but...

Posted Mar 29, 2009 21:16 UTC (Sun) by tstover (subscriber, #56283) [Link]

I get volatility part, what needs some big time disambiguation is all the talk of writes arriving on disk out of order. Of course writes to different files might be interwoven unpredictably, but what I want to know is - can call sequences like this arrive on platter out of order?

lseek(fd, 0, SEEK_CUR);
write(fd, "a", 1);
lseek(fd, 0, SEEK_CUR);
write(fd, "b", 1);
lseek(fd, 0, SEEK_CUR);
write(fd, "c", 1);

or with mmap()ed io:

x = 'a';
msync(&x, 1, MS_SYNC);
x = 'b';
msync(&x, 1, MS_SYNC);
x = 'c';
msync(&x, 1, MS_SYNC);

I would love to find this out as I have never seen a straight answer. Think about it though, if you could get data like this out of order on platter then userspace journaling (database transactions) would be a complete fantasy. Unless there is another file interface I don't know of.

extreemly informative and interesting, but...

Posted Apr 5, 2009 10:31 UTC (Sun) by dlang (✭ supporter ✭, #313) [Link]

the answer is yes

unless you do fsync or add a barrier (not currently possible from userspace)

this is the big issue with write barriers in ext3. without such barriers the writes to the journal can end up out of order. since the journal is laied out as a sequential block on disk, it's relativly unlikely to happen, but it is possible.