Throwing one away [LWN.net]

Throwing one away

Posted Sep 27, 2012 19:09 UTC (Thu) by cras (guest, #7000)
In reply to: Throwing one away by nix
Parent article: Throwing one away

rename()s during readdir() can cause the renamed files to be returned twice or never. I'm not entirely sure if other non-renamed files can be returned twice. Deleting files (or at least rename()ing to another directory) can cause some files to be skipped with some filesystems (unfortunately I didn't keep more details). Dovecot's Maildir syncing code has some comments related to this. https://lkml.org/lkml/2004/10/24/230 also.

(Log in to post comments)

Throwing one away

Posted Oct 5, 2012 18:56 UTC (Fri) by foom (subscriber, #14868) [Link]

Ooh, according to that lkml thread, it *is* possible to get a consistent state of a directory, on linux. You just need to allocate a buffer big enough for the whole directory to fit in. There doesn't appear to be a way to figure out how big that needs to be ahead-of-time, but at least you could iterate, doubling the buffer each time:

>> 3. the application could be offered an interface for atomic directory
>> reads that requires the application to provide sufficient memory in a
>> single contiguous buffer (making it thread-safe in the same go).
>
>Actually, you can do this today, if you use the underlying
>sys_getdents64 system call. But the application would have to
>allocate potentially a very large amount of userspace memory.