LWN.net Logo

Looking forward to 2.7

The bulk of the development effort on the kernel is currently aimed at stabilizing things for the 2.6 release. Chances are that things will stay that way for the better part of a year - remember that a fair amount of stabilization work has to happen after 2.6.0 is released. Even so, we're starting to see hints (and even code) showing where some things might go in 2.7.

A number of people maintain their own special-purpose kernel trees. Most of them are aimed at adding features to the 2.4 or 2.5 kernels; many serve as staging areas for patches which, it is hoped, will be merged into the mainline soon. Those of you who find 2.5.x to be overly stable and boring, though, may want to have a look at William Lee Irwin's -wli patch series, which is full of stuff that no rational person would consider putting into 2.5 at this point. Some of the work to be found there includes:

  • Single-page kernel stacks and interrupt stacks. This work, discussed here last December, increases the number of processes a system can support by reducing the per-process memory usage for stacks.

  • Object-based reverse mapping (covered in February). This technique cuts down on virtual memory management overhead in most cases. In 2.5.73-wli-1, object-based reverse mapping for anonymous objects (i.e. user-space memory) was added as well.

  • High-memory page mid-level directories. The PMD is the middle tier on systems which use three-level page table schemes - such as x86 systems with massive amounts of memory. The "highpmd" patch moves these page directories into high memory, thus reducing the amount of low memory required by each process on the system. Low memory (the memory, usually below 1GB, which is directly addressable by the kernel) tends to be scarce on truly huge systems, so any change which shifts data structures to high memory can be helpful.

As a result of these (and numerous other) patches, William claims a five-fold increase in the number of processes which can be supported by a massive system. This work certainly improves scalability, and may well make it into the mainline - but not in 2.5. (The -wli patches do not currently include his page clustering work, which is even more bleeding-edge. Page clustering, too, may well become a 2.7 feature.)

More in the realm of vaporware currently is Daniel Phillips's 2.7 agenda. Daniel has been the source of numerous interesting ideas in the past (though somewhat fewer completed implementations). Among other things, the shared page table patch (which could also be a 2.7 candidate) was originally written by Daniel. Looking forward to 2.7, Daniel has a few topics of interest:

  • Memory defragmentation. Once a Linux system has been running for a bit, it can get hard for kernel code to allocate blocks of two or more physically contiguous pages. In most cases, kernel hackers don't even try. Daniel suggests the creation of a defragmentation daemon which would move pages around in an attempt to create larger contiguous blocks of free memory. Additions made to the kernel in 2.5 (such as the reverse-mapping VM) will help in this regard, since pages cannot be moved unless the kernel knows where all the pointers to the page are.

  • Variable-size pages. This idea includes page clustering to create large pages along with "sub-pages" which are smaller than the physical page size. Daniel claims to have a prototype implementation which makes the kernel smaller and faster, and which simplifies a number of things.

  • A physical block cache. This would be a separate address space which tracks physical blocks on a given volume. There are various performance benefits which would come from such a structure.

It is far too soon to say with any kind of certainty where the 2.7 development series will go. Linus explicitly resists creating any sort of explicit plan, preferring to see what sorts of developments prove interesting enough to actually get implemented and used. Still, one can read from these early hints that the developers expect to remain interested in virtual memory topics for a while yet.


(Log in to post comments)

Looking forward to 2.7

Posted Jun 26, 2003 3:05 UTC (Thu) by cpeterso (guest, #305) [Link]

I'm looking forward to the compressed caching patch, which creates a "swap space" of compressed pages between uncompressed pages and the swap space on disk. As memory gets faster and cheaper, using CPU time to compress pages in memory can sometimes be faster than writing them to disk.

There is such a patch for Linux, but it does seem to get much testing.

http://linuxcompressed.sourceforge.net/

Looking forward to 2.7

Posted Jun 26, 2003 13:39 UTC (Thu) by zooko (subscriber, #2589) [Link]

I was wondering if the variable-size pages would be useful for the memory-compression stuff, since the compressed page could be stored in a smaller page frame ?

memory defragmentation

Posted Jun 26, 2003 3:14 UTC (Thu) by cpeterso (guest, #305) [Link]

Daniel's plans for memory defragmentation require some fancy footwork to relocate physical pages, like needing to know what kernel data structures are pointing to which pages. Once they've gone that far, how long until someone hacks the Linux kernel to support garbage collection?? :-)

memory defragmentation

Posted Jun 26, 2003 17:19 UTC (Thu) by elanthis (guest, #6227) [Link]

Kernel supported garbage collection would be most excellent. A few modifications to ELF to support GC hints/etc. in the executable format, and GC could become very commonplace. (Like it should have years ago.)

Imagine natively compiled Java or C# apps that make use of kernel-accelerated GC... or C/C++ extensions that offer the same. Heck, languages like Python/Ruby/etc. could drop their own home-brewed GCs and use the kernel one (when running in Linux) to increase their own performance as well.

Just remember, tho, compiler support is _required_ for a good GC implementation to work as efficiently as possible. Things like the Boehm-Weiss collector, while certainly cool "hacks", don't perform nearly as well as they could, since they do so much work ( a good deal of which is guess work ) to try to figure things out that the compiler could've provided for it.

memory defragmentation

Posted Jun 26, 2003 19:50 UTC (Thu) by daniel (subscriber, #3181) [Link]

<i>Daniel's plans for memory defragmentation require some fancy footwork to relocate physical pages, like needing to know what kernel data structures are pointing to which pages. Once they've gone that far, how long until someone hacks the Linux kernel to support garbage collection??</i>

The kernel already has garbage collection, for example, dentries and cold cache pages. But that's probably not what you meant. If you mean, we should use algorithms that don't keep track explicitly of what references what, and clean up with a classic garbage collector, I'd argue against it. Explicit reference counting is generally a more efficient, predictable way to handle dynamic objects, and so is more suitable for kernel use. Sure, it may be more work to write and debug code that works that way, but that is just the way life is in kernel land.

Regards,

Daniel

Looking forward to 2.7

Posted Jul 3, 2003 18:44 UTC (Thu) by mmarq (guest, #2332) [Link]

So 2.7 is going to be a real "memory management party",... good...
IMHO 2 points of most importance:

1) Theres gonna be a way to get "MOST" relevant specific hardware drivers out of core Kernel, not only because it would stop the binary only modules grind of teeth, but because as i often state...how can otherwise the driver model in kernel evolve to permit the industry support near to that of Windows without collapsing the Linux kernel under is own weight !... is not that hard, isnt it ?, to imagine a kernel that offers similar hardware support than Windows but witch sources are near 1 Gig, and took 4, 5 or 6 years to get to the stable series ?
Perhaps DKMS could do it( http://www.lerhaupt.com/linux.html#dkms ) or perhaps a split driver model ala I2O,... or both...

2) What about improving real-time capabilities, explicitly required in OSDL for the telecomunications industry, but also much helpfull for the new CE Linux forum and the intire embeded industry...
Perahps a "somehow" parcial merge with RTlinux "IDEAS" (totaly GPL i belive), especialy in how it manages RT processes and threads, would not be such a bad idea...
IMHO could even help in the above as to make relevant firmware and drivers (if not all) RT processes and or threads
Interesting stuff here http://www.thinkingnerds.com/projects/minirtl/minirtl.html.

Copyright © 2003, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds