| |||||||||||||
Large pages, large blocks, and large problemsLarge pages, large blocks, and large problemsPosted Sep 19, 2007 20:44 UTC (Wed) by vblum (guest, #1151)Parent article: Large pages, large blocks, and large problems
How do applications with large allocated arrays for numerics (say) get around the page size / memory fragmentation issue? Wouldn't one expect them to experience some slowdown when the system's memory gets heavily fragmented, if the basic arrays are significantly larger than the cache size? just wondering ...
(Log in to post comments)
Large pages, large blocks, and large problems Posted Sep 19, 2007 22:06 UTC (Wed) by i3839 (guest, #31386) [Link] Only the kernel accesses memory by physical addresses, userspace processes use virtual memory, so "fragmented memory" can look like one contiguous chunk to them.
Large pages, large blocks, and large problems Posted Sep 20, 2007 9:18 UTC (Thu) by vblum (guest, #1151) [Link] Yes, masking fragmentation at a low level will make things look contiguous, but in reality someone somewhere has to do the work to bring the fragments together. Is the adverse impact on performance (wall-clock time for large operations) really negligible? (thanks to clever cache use?)
Large pages, large blocks, and large problems Posted Sep 20, 2007 12:55 UTC (Thu) by i3839 (guest, #31386) [Link] Not really.
There is always a mapping made between the virtual addresses and the physical ones, so what's different is which physical pages are chosen. Choosing contiguous physical pages is harder and takes more time, but it doesn't give any advantages. Bringing the fragments together is done by mapping them to a contiguous virtual memory range.
Mapping to contiguous physical pages might be faster if the hardware supports (really) variable sized pages, but as far as I know none does.
Fragmentation doesn't slow things down, it only makes it harder to allocate chunks bigger than one pagesize.
The demand for bigger pagesizes comes from people who think it will reduce overhead for their workload, because everything that's done per page can be done less often. The main thing preventing this from working seamlessly is fragmentation.
|
|||||||||||||