> causes files to disappear at random points in time.
Having files disappear spontaneously makes sense to me. A 'file' is a natural unit of caching. There is a clear distinction between the 'file' and the 'name', so that you can unlink a cache file even while it is in use, and the process using it will not lose out.
Having arbitrary blocks in the middle of a file disappear spontaneously it not something that I am so comfortable with. There is no 'natural unit' (so John had to invent 'ranges' and worry about semantics for merging etc) and there is no 'object/name' distinction so you have to think carefully about races between access and discard.
I would really like it if the whole 'volatile data' thing could be done with files. Files get marked as 'volatile' and the filesystem can unlink them as desired. One problem is that open/mmap/close is a whole lot slower than any single systemcall, and definitely slower than a simple memory access that might (but usually doesn't) cause SIGBUS.
Maybe an madvise style interface that works for ranges in anonymous memory, and some sort of per-file interface for filesystems when a shared cache is required.