What are the semantics when you open a device node (either block, e.g. disk, or char, e.g. tape)? Does the kernel ever use page cache for device files? Does O_DIRECT do anything? What about O_SYNC? Does fsync always generate a barrier?
Also, what about the different disk abstraction layers (LVM, dm-crypt, MD RAID, DRBD, etc) -- what's involved in passing an fsync() all the way down the stack?