User: Password:
Subscribe / Log in / New account



Posted Jan 28, 2010 17:18 UTC (Thu) by iabervon (subscriber, #722)
Parent article: fincore()

It seems to me like, once you're using both fadvise() and a non-portable syscall, you could inform the kernel in more detail about your actual usage pattern, and it could store the info and decide what to do. The system should be able to make good choices if you could tell the kernel, in order, the pages you intend to use (up until the kernel tells you it doesn't want to store any more hints for you), and have the kernel only prefetch up to where it can fit in memory, prefer to drop the ones which are further out (and likely newest rather than oldest), and remember what you've requested that isn't in memory such that when you drop your interest in the pages you've passed, it can prefetch new ones then.

It doesn't make sense to have a userspace heuristic for figuring out kernel limits when you need kernel support to implement it, particularly if the information you're getting only helps if you are right about the kernel's heuristics. Maybe the kernel will stop evicting pages that have been requested but not used when asked to prefetch more pages, and heuristics based on checking whether pages are in core and an assumption as to the kernel's use of the hints will give entirely wrong results.

(Log in to post comments)


Posted Jan 29, 2010 17:38 UTC (Fri) by giraffedata (subscriber, #1954) [Link]

I agree. First of all, fadvise() does not request prefetch. It advises the kernel that you are going to access a certain part of the file soon. It is up to the prefetcher to decide how to exploit that information.

Only the prefetcher, in the kernel, can properly decide how much memory to allocate for prefetching this particular file. Memory is a resource shared between processes, and coordinating resource usage between processes is fundamentally the kernel's responsibility. The application should just look out for itself.

Copyright © 2017, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds