ZONE_DEVICE and the future of struct page

Posted Mar 23, 2017 22:25 UTC (Thu) by djbw (subscriber, #78104)
In reply to: ZONE_DEVICE and the future of struct page by roc
Parent article: ZONE_DEVICE and the future of struct page

In fact, it's not a waste. It's fundamental to many kernel paths. The DAX enabling without pages loses get_user_pages() support which disables not only DMA / direct-I/O, but also fundamental operations like fork and ptrace. We're already paying this 1.5% overhead for main memory, and my argument is that we should simply pay that overhead for persistent memory as well. It's not enough to convert some paths to use pfn_t and with new kmap() primitives, because that leaves us an ongoing maintenance burden of dual code paths as developers add new struct page usages. Unless we create a plan to get rid of struct page everywhere we should not special case persistent memory... especially when we have a mechanism to pay the overhead cost from pmem itself.

Once we mandate struct page for DAX this appears to open up several clean up opportunities like re-using more of the the core page cache implementation and unifying device-DAX / filesystem-DAX.

ZONE_DEVICE and the future of struct page

Posted Mar 23, 2017 23:39 UTC (Thu) by Cyberax (✭ supporter ✭, #52523) [Link] (1 responses)

There are problems with that. Persistent memory is fast and durable, but not as fast and durable as the regular volatile RAM. So you have to put your page structures in the main RAM and this adds up quickly - a 2Tb persistent array will require around 32Gb or page structures.

ZONE_DEVICE struct page != ZONE_NORMAL struct page in terms of write rate

Posted Mar 24, 2017 0:15 UTC (Fri) by djbw (subscriber, #78104) [Link]

I'm not convinced that's going to be a problem in practice. Consider that the bulk of what makes struct page a frequently accessed data structure is when it is used by the core mm for general purpose page allocations. The ZONE_DEVICE mechanism never releases these pages for that high frequency usage. Another mitigation is that struct page writes are buffered by the cpu cache, which further reduces the write rate to media.