Temporary files: RAM or disk? [LWN.net]

Temporary files: RAM or disk?

Posted Jun 4, 2012 7:27 UTC (Mon) by neilbrown (subscriber, #359)
In reply to: Temporary files: RAM or disk? by Serge
Parent article: Temporary files: RAM or disk?

> If file creation/deletion (metadata change) triggers disk access you'll see all the lines different.

This doesn't agree with my understanding of ext3 journalling, so maybe I expressed it poorly.

If you put a 5 second sleep in that loop, I expect you would see changes. I do - once I found a suitably quiet ext3 filesystem to test on.

The metadata blocks do go into the next transaction, but transactions can live in memory for up to 5 seconds before they are flushed.

(Log in to post comments)

Temporary files: RAM or disk?

Posted Jun 4, 2012 10:17 UTC (Mon) by Serge (guest, #84957) [Link]

> If you put a 5 second sleep in that loop, I expect you would see changes.

The exact number of seconds depends on /proc/sys/vm/dirty_*_centisecs value and /proc/sys/vm/laptop_mode...

Anyway, are you talking about file content or file name being written to disk in 5 seconds? Or both?

We can check whether content of deleted file is written to disk, run:
for i in `seq 100`; do dd if=/dev/zero of=f bs=1M count=10; rm -f f; done
then check /proc/diskstats or `iostat -k`. If you see writes increased in 1GB, your filesystem writes data even for deleted files. My ext3 does not.

> I do - once I found a suitably quiet ext3 filesystem to test on.

Try /boot. :) Or just insert some USB flash stick and create ext3 there.

Temporary files: RAM or disk?

Posted Jun 4, 2012 11:28 UTC (Mon) by neilbrown (subscriber, #359) [Link]

No. The "5 seconds" that I was taking about is not a /proc/sys/vm/dirty* number. It is ext3 (and presumably ext4) specific.
It defaults to 5 seconds (JBD_DEFAULT_MAX_COMMIT_AGE) and can be changed by the "commit=nn" mount option.

That many seconds after a journal transaction has been opened, it is closed and flushed - if it hadn't been closed already.

It is the metadata that is written to the journal - inodes, free-block bitmaps, directory names etc.
The file contents are handled differently for different settings of "data=".
ordered: data that relates to the metadata in flushed before the metadata is written to the journal
writeback: data is written according to /proc/sys/vm/dirty* rules
journal: data is written to the journal with the metadata.

I'm not sure what the default is today. If you create then delete a file, the data will not go to disk, except possibly for "data=journal". But the metadata will.

Temporary files: RAM or disk?

Posted Jun 4, 2012 15:17 UTC (Mon) by Serge (guest, #84957) [Link]

> If you create then delete a file, the data will not go to disk, except possibly for "data=journal". But the metadata will.

That's harder to test. Maybe compare amount of writes generated by something like:
for i in `seq 10`; do touch $i; rm -f $i; done
with amount of writes generated by:
for i in `seq 1000`; do touch $i; rm -f $i; done
Every creation/deletion is written to disk if the latter line generates about 100 times more writes. On my ext3 I see sub-equal number of writes...

But, anyway, looks like it's not a problem for /tmp then, meaning that ext2 would not be (noticeably) better than ext3 in /tmp use cases.