LWN: Comments on "Featherstitch: Killing fsync() softly"

Featherstitch: Killing fsync() softly

dlang — 2009-10-26T08:23:27+00:00

if this with a ext2 filesystem layout is comparable to ext3 (which uses basicly the same layout) is there any chance of someone submitting this to the kernel?

if this avoids the ext3 fsync bug it would be a big win for many real-world systems, in spite of the added cpu and memory useage

Featherstitch: Killing fsync() softly

vaurora — 2009-10-11T01:53:24+00:00

This might be a difference in the use of the word "implemented" - I believe (and do correct me if I'm wrong) that soft updates were designed and written once, for FreeBSD, and then the code was inherited to the other BSDs. But I can see this being interpreted as "exists and works in a production operating system" in which case I would have to list all FreeBSD descended operating systems.

Anyway, please do let me know if soft updates were re-implemented independently at any point. The only instance I know of is the Featherstitch-based re-implementation described in this article.

Featherstitch: Killing fsync() softly

erh — 2009-10-10T17:46:34+00:00

as for the rest of the article: sounds like a neat idea.

Featherstitch: Killing fsync() softly

erh — 2009-10-10T17:37:34+00:00

Wow,it's really hard to keep reading an article when the very first sentence in it is plainly wrong. Soft updates have been implemented in NetBSD, DragonflyBSD and OpenBSD, not just FreeBSD.

Patchset API

nix — 2009-10-09T22:33:23+00:00

Quite so, this is really silly :) I'd recommend against using prefixes
like 'batch_*()' in available-everywhere interfaces, though, 'cos it's the
sort of prefix that batch-processing systems are *certain* to be using
already.

Featherstitch: Killing fsync() softly

cesarb — 2009-10-08T16:00:33+00:00

For some reason, this description reminded me of darcs and its "theory of patches".

Patchset API

sfink — 2009-10-07T22:18:34+00:00

Ooh! A bikeshed with peeling paint!

How about batch_*()? Or wb_*() for writebatch? (I'm ignoring the "patch" word, since I find it far more confusing than enlightening.)

Will it always be write-specific? Would it ever be useful to say something like "read blocks A-D, but I'll need either both A and B or both C and D before I can make progress"? Something to do with RAID parity blocks, or quasi-realtime processing using memory that needs to be pinned but only while the process is scheduled or starts a pass, or maybe (ooh, here's a confusing one) when you want to read data from in-progress writes that are themselves being controlled by a dependency DAG? Or heck, maybe you're trying to read off of multiple devices and you only have enough power to spin up a subset at a time.

I guess I'm just making up bullshit.

Mmm. Bullshit and bikesheds. They go so well together, like chocolate and peanut butter. If you'll excuse the juxtaposition.

Featherstitch: Killing fsync() softly

TRS-80 — 2009-10-04T15:06:05+00:00

ZFS also has a transactional engine (DMU) underneath, although it's not exposed to user space.

Featherstitch: Killing fsync() softly

vaurora — 2009-10-02T18:42:31+00:00

Yes, you are right! I will get LWN to fix that for me.

Featherstitch: Killing fsync() softly

Cyberax — 2009-10-02T11:02:03+00:00

Might this lead to full filesystem transactions someday?

NTFS in Vista supports some transactional operations (the ones which are easy to implement) and it really helps sometimes.

One can run a distributed transaction with database and filesystem as XA datasources. For example, I use it in a document management system - XA transactions guarantee that document data and metadata in the database are always in sync.

Patchset API

nix — 2009-10-01T15:09:00+00:00

i.e. time to write a trivial wrapper library? Maybe, but let's get the names right first: pg_*() is pretty awful: it doesn't say 'disk writes' to me at all, it says 'PostgreSQL'!

writegroup_*() might be better but is a bit long. wg_*() is shorter but somewhat opaque...

Patchset API

epa — 2009-10-01T10:48:10+00:00

The pg_* calls look like a great way for apps to specify exactly what they want from the filesystem. I wonder to what extent it's possible to provide them on plain vanilla filesystems without Featherstitch. For example, on minixfs almost every pg_* call would just do sync() or fsync(); on ext3 it would be either fsync() or a no-op depending on whether data=ordered already provides the necessary guarantees; and so on. Then apps could start using the rich interface now, even if the kernel is at the moment a bit stupid and does things slower than it could. Over time, the kernel can be optimized to provide the same guarantees but faster.

Featherstitch: Killing fsync() softly

njs — 2009-10-01T06:05:48+00:00

Pedantic erratum: I believe you're missing a call to pg_engage(rename_pg) in the second part of your example.

Thanks for a great article. It leaves me wondering -- what are the chances we application programmers will ever get a chance to play with these toys for real? Care to speculate on the viability of a patchgroup-style API showing up in real world systems over the next, I dunno, 5 years?