User: Password:
|
|
Subscribe / Log in / New account

Temporary files: RAM or disk?

Temporary files: RAM or disk?

Posted Jun 4, 2012 7:10 UTC (Mon) by Serge (guest, #84957)
In reply to: Temporary files: RAM or disk? by neilbrown
Parent article: Temporary files: RAM or disk?

> So with ext3 journalling [...] changed will be written to the journal on the next journal commit

Probably. But it won't trigger disk access. You can check that:
for i in `seq 5`; do echo 123 > f; rm -f f; grep sda1 /proc/diskstats; done
(replace "sda1" with the disk you write to)

If file creation/deletion (metadata change) triggers disk access you'll see all the lines different. But if lines are same, then there was no disk access.

Cache still works for journaled filesystems. Linux kernel is written by smart people, yeah.

PS: I've seen reiserfs to trigger "read" in such test. You can see description of diskstats numbers in:
http://www.kernel.org/doc/Documentation/iostats.txt


(Log in to post comments)

Temporary files: RAM or disk?

Posted Jun 4, 2012 7:27 UTC (Mon) by neilbrown (subscriber, #359) [Link]

> If file creation/deletion (metadata change) triggers disk access you'll see all the lines different.

This doesn't agree with my understanding of ext3 journalling, so maybe I expressed it poorly.

If you put a 5 second sleep in that loop, I expect you would see changes. I do - once I found a suitably quiet ext3 filesystem to test on.

The metadata blocks do go into the next transaction, but transactions can live in memory for up to 5 seconds before they are flushed.

Temporary files: RAM or disk?

Posted Jun 4, 2012 10:17 UTC (Mon) by Serge (guest, #84957) [Link]

> If you put a 5 second sleep in that loop, I expect you would see changes.

The exact number of seconds depends on /proc/sys/vm/dirty_*_centisecs value and /proc/sys/vm/laptop_mode...

Anyway, are you talking about file content or file name being written to disk in 5 seconds? Or both?

We can check whether content of deleted file is written to disk, run:
for i in `seq 100`; do dd if=/dev/zero of=f bs=1M count=10; rm -f f; done
then check /proc/diskstats or `iostat -k`. If you see writes increased in 1GB, your filesystem writes data even for deleted files. My ext3 does not.

> I do - once I found a suitably quiet ext3 filesystem to test on.

Try /boot. :) Or just insert some USB flash stick and create ext3 there.

Temporary files: RAM or disk?

Posted Jun 4, 2012 11:28 UTC (Mon) by neilbrown (subscriber, #359) [Link]

No. The "5 seconds" that I was taking about is not a /proc/sys/vm/dirty* number. It is ext3 (and presumably ext4) specific.
It defaults to 5 seconds (JBD_DEFAULT_MAX_COMMIT_AGE) and can be changed by the "commit=nn" mount option.

That many seconds after a journal transaction has been opened, it is closed and flushed - if it hadn't been closed already.

It is the metadata that is written to the journal - inodes, free-block bitmaps, directory names etc.
The file contents are handled differently for different settings of "data=".
ordered: data that relates to the metadata in flushed before the metadata is written to the journal
writeback: data is written according to /proc/sys/vm/dirty* rules
journal: data is written to the journal with the metadata.

I'm not sure what the default is today. If you create then delete a file, the data will not go to disk, except possibly for "data=journal". But the metadata will.

Temporary files: RAM or disk?

Posted Jun 4, 2012 15:17 UTC (Mon) by Serge (guest, #84957) [Link]

> If you create then delete a file, the data will not go to disk, except possibly for "data=journal". But the metadata will.

That's harder to test. Maybe compare amount of writes generated by something like:
for i in `seq 10`; do touch $i; rm -f $i; done
with amount of writes generated by:
for i in `seq 1000`; do touch $i; rm -f $i; done
Every creation/deletion is written to disk if the latter line generates about 100 times more writes. On my ext3 I see sub-equal number of writes...

But, anyway, looks like it's not a problem for /tmp then, meaning that ext2 would not be (noticeably) better than ext3 in /tmp use cases.

Temporary files: RAM or disk?

Posted Jun 4, 2012 14:13 UTC (Mon) by hummassa (subscriber, #307) [Link]

In my work machine, on ext4, all lines are different.


Copyright © 2017, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds