Once upon atime
A recent discussion on write throttling turned to atime after Ingo Molnar pointed out that atime was probably a bigger performance problem than just about everything else. He went on to say:
He also claimed that it was "perhaps the most stupid Unix design idea
of all times
".
Such discussion leads quickly to the question of what should be done about this old situation. One step that any Linux user can take now is to mount filesystems with the noatime option, which turns off the tracking of access times. For filesystem-intensive tasks, the performance reward can be immediately apparent. Unfortunately, turning off atime unconditionally will occasionally break software. Some mail tools will compare modification and access times to determine whether there is unread mail or not. The tmpwatch utility and some backup tools also use atime and can misbehave if atime is not correct. For this reason, distributors tend not to make noatime the default on installed systems.
Another approach was added in 2.6.20: the relatime mount option. If this flag is set, access times are only updated if they are (before the update) earlier than the modification time. This change allows utilities to see if the current version of a file has been read, but still cuts down significantly on atime updates. This option is not heavily used, perhaps because few people have heard of it and many distributions lack a version of mount which is new enough to know about it. Using relatime can still confuse tools which want to ask questions like "has this file been accessed in the last week?"
To fix that problem, Linus suggested a tweak to how relatime works: update it if the current value is more than a certain time in the past - one day, for example. Ingo responded with a patch implementing that behavior and adding a couple of new boot options: relatime_interval, which specifies the update interval in seconds, and default_relatime, which turns on the relatime option in all filesystems by default.
Something resembling this version of the patch might go into 2.6.24. It
was suggested that, whenever a file's inode is to be written to disk
anyway, the kernel might as well update atime as well. Alan Cox objected
that this change might make the overall behavior less predictable, which
might not be desirable. No new version of the patch with this feature has
been posted, so chances are it will not be in the version which gets merged
- if and when that happens.
Index entries for this article | |
---|---|
Kernel | Filesystems/Access-time tracking |
Posted Aug 9, 2007 9:09 UTC (Thu)
by addw (guest, #1771)
[Link]
Why not defer the inode updates, then you might get one set of disk writes either when the file system is unmounted or when it becomes idle or at scheduled intervals (eg every 1/2 hour). In practice many systems do not open large numbers of files, but repeatedly access a subset time and again. The usual exception to this is the backup program.
From my point of view: if the system dies before it writes the atime out then not a lot is lost; maybe a spurious ''you have mail'' when I login again.
Memory usage is the primary cost as the in-memory inode needs to be kept longer than it normally would be. This could be ameliorated by having a new strucut (device, inode#, atime) to store this info if we trash the in-memory inode.
Posted Aug 9, 2007 12:26 UTC (Thu)
by rfunk (subscriber, #4054)
[Link] (2 responses)
Posted Jun 2, 2012 22:52 UTC (Sat)
by mirabilos (subscriber, #84359)
[Link] (1 responses)
Posted Jun 3, 2012 18:19 UTC (Sun)
by mathstuf (subscriber, #69389)
[Link]
Posted Aug 9, 2007 14:52 UTC (Thu)
by davecb (subscriber, #1574)
[Link]
Solaris has a "defer atime" mount option that does the atime update
only when other disk I/O is scheduled, which gives a fairly fine-grained
update, and makes the behavior less prefictable, but closer
to what happens when atime is turned on. This approach has
been in place for a number of years (five or six, at least) and
doesn't confuse programs or unsuspecting users.
--dave
Posted Aug 9, 2007 16:12 UTC (Thu)
by elanthis (guest, #6227)
[Link] (5 responses)
The thread this discussion was in notes that the kernel devs like Linus explicitly not only noatime, but also nodiratime. I ASSume that noatime only affects files and that directories need nodiratime.
Posted Aug 9, 2007 16:17 UTC (Thu)
by corbet (editor, #1)
[Link] (4 responses)
Posted Aug 9, 2007 20:17 UTC (Thu)
by tarvin (guest, #4412)
[Link] (3 responses)
Are you sure about noatime implying nodiratime? mount(8) states: while mount(2) puts it this way: I wonder how to interpret mount(2)'s (all types of) files for noatime: Is a directory considered a file in this context?
Posted Aug 9, 2007 20:39 UTC (Thu)
by corbet (editor, #1)
[Link] (2 responses)
So if NOATIME is set, the NODIRATIME flag is never even checked.
Posted Aug 10, 2007 6:41 UTC (Fri)
by tarvin (guest, #4412)
[Link] (1 responses)
- But your definite statement is comforting, thanks. I'll keep using just "noatime".
Posted Aug 10, 2007 14:26 UTC (Fri)
by jzbiciak (guest, #5246)
[Link]
Posted Aug 10, 2007 12:39 UTC (Fri)
by liljencrantz (guest, #28458)
[Link]
The additional memory requirements should be relatively small, but the extra time needed to perform atime lookup might not be.
It would also mean that after a crash atimes will be wrong.
Posted Aug 11, 2007 1:12 UTC (Sat)
by xkahn (subscriber, #1575)
[Link]
http://msdn2.microsoft.com/en-us/library/ms724290.aspx
Posted Aug 17, 2007 15:19 UTC (Fri)
by endecotp (guest, #36428)
[Link]
Part of the reason that atime causes disk traffic is that if a file is read many times (each read satisfied from cache) an atime update will be generated for each time the file is read. So you get multiple disk writes to same the inodes over and over.Defer updates
I'm pretty sure the noatime incompatibility with mail programs shows up
only when using a single-file mailbox format (usually mbox). Yet another reason to
use maildir instead.
noatime and mail programs
noatime and mail programs
noatime and mail programs
The article notes: It was suggested that, whenever a file's inode is to be written to disk anyway, the kernel might as well update atime as well. Alan Cox objected that this change might make the overall behavior less predictable, which might not be desirable.
Once upon atime
Everyone knows about noatime, but it seems almost nobody uses nodiratime.nodiratime
noatime implies nodiratime - it's a superset. You can use nodiratime by itself to turn off the updates on directories only. As you note, it does not seem to be widely used.
nodiratime
Does noatime imply nodiratime?
noatime
Do not update inode access times on this file system[...]
nodiratime
Do not update directory inode access times on this filesystem.
MS_NOATIME
Do not update access times for (all types of) files on this file system.
MS_NODIRATIME
Do not update access times for directories on this file system.
Yep, I'm sure. When in doubt, use the source. From touch_atime() in fs/inode.c:
Does noatime imply nodiratime?
void touch_atime(struct vfsmount *mnt, struct dentry *dentry)
{
/* ... */
if (inode->i_flags & S_NOATIME)
return;
if (IS_NOATIME(inode))
return;
if ((inode->i_sb->s_flags & MS_NODIRATIME) && S_ISDIR(inode->i_mode))
return;
OK. I've been using noatime systematically for years. And then I read about a prominent kernel developer like Molnar using both noatime and nodiratime, making me worried: Have I been missing out on I/O-performance for years?Does noatime imply nodiratime?
Actually, Andrew Morton actually corrected Ingo on this point:
Does noatime imply nodiratime?
From: Andrew Morton
To: Ingo Molnar
Subject: Re: [PATCH 00/23] per device dirty throttling -v8
Date: Sun, 5 Aug 2007 00:29:34 -0700
On Sun, 5 Aug 2007 09:21:41 +0200 Ingo Molnar [email blocked] wrote:
> even on a noatime,nodiratime filesystem
noatime is a superset of nodiratime, btw.
I trust Andrew on this point. :-)
One other possibility is to keep atime updates in memory and only write them out to disk if the page was to be flushed to disk anyway. That would mean storing atimes in memory in a separate structure outside of the regular page.Once upon atime
Interestingly, Microsoft seems to have looked at this issue in the past. From MSDN: Once upon atime
Not all file systems can record creation and last access time and not
all file systems record them in the same manner. For example, on NT
FAT, create time has a resolution of 10 milliseconds, write time has a
resolution of 2 seconds, and access time has a resolution of 1 day
(really, the access date). On NTFS, access time has a resolution of 1
hour.
The only program that I run that benefits from atime is Debian's "popcon", which records which packages are installed and which of those has been used recently. This would work well with the "update atime when > 1 day" approach suggested by Linus.Popularity contest