|
|
Subscribe / Log in / New account

SLOB nears the end of the road

By Jonathan Corbet
December 23, 2022
The kernel project tries hard to avoid duplicating functionality within its code base; whenever possible, a single subsystem is made to serve all use cases. There is one notable exception to this rule, though: there are three object-level memory allocators ("slab allocators") in the kernel. The desire to reduce the count has been growing stronger over the years, and some steps have been taken in 6.2 to eliminate the least-loved allocator — SLOB — in the relatively near future.

The job of a slab allocator is to provide the kernel with (usually) small chunks of memory in an efficient way. The heavily used kmalloc() function is implemented by the slab allocator, but there is also a lower-level API specialized for the management of numerous objects of the same size. It is quite common for a kernel subsystem to need to allocate instances of a given structure, for example; all of those instances are normally the same size and can be managed in a slab.

The kernel's oldest slab allocator is typically just called SLAB (though the name is not an acronym); it has been there in one form or another since nearly the beginning. SLAB is intended to be a general-purpose allocator suitable for most workloads, and serves that purpose reasonably well. Even so, developers have occasionally wished for a different approach to object allocation; that led to the SLOB allocator, which was added in 2006 for the 2.6.16 kernel release; its purpose is to support the smallest of systems where SLAB's memory use was seen as being too high:

SLOB is a traditional K&R/UNIX allocator with a SLAB emulation layer, similar to the original Linux kmalloc allocator that SLAB replaced. It's significantly smaller code and is more memory efficient. But like all similar allocators, it scales poorly and suffers from fragmentation more than SLAB, so it's only appropriate for small systems.

The third allocator, SLUB, followed for 2.6.22 in 2007. SLUB, which was intended to eventually replace SLAB, was a reaction to the perceived complexity and scalability problems in SLAB; it had a strong focus on scalability and performance. There have been attempts to add other allocators — SLQB in 2008 and SLEB in 2010, for example — but the appetite for more allocators had mostly faded by then; Linus Torvalds made it clear in 2010 that he would not accept any more of them. So, in 2022, the set of mainline slab allocators remains SLAB, SLOB, and SLUB.

Since then, there has been occasional talk of removing at least one of the existing slab allocators, but no real movement in that direction — until Vlastimil Babka took up the issue this year. In November, he proposed deprecating SLOB as a step toward its eventual removal:

The unsurprising reasons include code maintenance burden, other features compatible with only a subset of allocators (or more effort spent on the features), blocking API improvements (more on that below), and my inability to pronounce SLAB and SLUB in a properly distinguishable way, without resorting to spelling out the letters.

The API improvement mentioned there is adding the ability to use kfree() on objects obtained from the lower-level slab functions (specifically kmem_cache_alloc()); the SLOB allocator's version of kfree() can only handle objects allocated with kmalloc(), which complicates code elsewhere in the kernel. This cost is maybe justified if it brings other benefits, but Babka suggested that, in fact, nobody is actually using SLOB. The small devices it was aimed at (less than 32MB of RAM) don't exist in large numbers anymore, and even distributions for small devices (such as OpenWrt) are not using it.

In the ensuing discussion, Paul Cercueil described his attempts to use SLOB, concluding that it simply does not work properly. Torvalds also recalled hearing about SLOB problems in recent times; Aaro Koskinen responded, though, that SLOB might just be exposing driver bugs that are hidden by the other allocators. In the end, though, SLOB had no real defenders; nobody is advocating for keeping it in the kernel.

Still, one cannot just remove an allocator from the kernel without trying to meet the needs of any existing users. So Babka posted a patch set that tried to minimize the impact on any SLOB users that may remain. It introduces a new SLUB_TINY configuration option that removes many of the scalability and debugging features in an attempt to make SLUB as small as possible. SLUB will never be as small as SLOB but, hopefully, it can be made small enough for what passes as a "small system" in the 2020s. The SLOB configuration option was then renamed to SLOB_DEPRECATED, and it gained a warning that SLOB would be removed "in a few cycles". This patch series was merged for 6.2.

Unless a compelling case for the retention of SLOB comes up over the course of the next year, the removal of that allocator seems nearly certain. Then Linux will be down to just two allocators, both of which claim to handle the general case.

Babka has made it clear that he sees two as still being too many; the removal of SLAB is sure to come up once the SLOB removal is complete. Getting rid of SLAB will not be quite so easy, though, since SLAB still has active users. Notably, Google is said to be using SLAB internally. It seems that SLUB is still worse for some use cases; as long as that situation persists, removing SLAB will be hard. This problem is complicated by a lack of general understanding of just where and why SLUB falls down.

Getting a handle on that situation may take some time, so the kernel is likely to continue to have two slab allocators for some time yet. But even the removal of SLOB will make life easier in a number of ways. It can seem like code added to the kernel is there forever, but it is, sometimes, possible to get rid of old subsystems with enough patience and effort.

Index entries for this article
KernelMemory management/Slab allocators


to post comments

Small systems

Posted Dec 24, 2022 11:39 UTC (Sat) by epa (subscriber, #39769) [Link] (17 responses)

The slab allocator, as the article says, has been there almost since the beginning. In those days 32 megabytes of RAM was a lot. Linux commonly ran on PCs with eight megs or less. What has changed to make the slab allocator no longer suitable for these small systems? And can it be partially undone by adding new tuning knobs to the allocator?

Small systems

Posted Dec 24, 2022 13:54 UTC (Sat) by tamiko (subscriber, #115350) [Link] (7 responses)

The last time I checked a stock distribution kernel required around 50 MBytes of RAM to boot. For today's standards where we measure the amount of RAM of mobile phones in GBytes this is tiny.

I think that this "high" memory requirement is mostly due to speed optimizations in the kernel (and allocator). So the mentioned SLUB_TINY option might just do the trick in removing/reducing most of these default allocations.

I remember vaguely that someone gave a talk not too long ago presenting a modified Linux kernel that required less than a MByte to boot.

Small systems

Posted Dec 24, 2022 15:06 UTC (Sat) by Wol (subscriber, #4433) [Link] (3 responses)

Last I heard, the minimum requirement for the *stock* kernel to boot (admittedly, config options pared down to a minimum) was 6MB.

And it was pointed out that your typical distro installer needed more than that ...

It's all the stuff on top of the kernel that is responsible for demanding all that ram.

Cheers,
Wol

Small systems

Posted Dec 24, 2022 15:51 UTC (Sat) by atnot (subscriber, #124910) [Link] (1 responses)

For reference, OpenWRT, the most common linux distribution for constrained devices, has all but dropped support for 32/4 (MB of Flash/RAM) devices: https://openwrt.org/supported_devices/432_warning, with 64/8 being declared passable and 128/16 useful. So that appears to be the practical limit.

There's still devices that fall below this range of course, but I personally think that even if it could be done, it doesn't makes much sense to support them in linux today in the first place. There's a real cost to every order of magnitude of memory you support, and they usually run some custom RTOS out of the box anyway, which they are probably much better served by.

Small systems

Posted Dec 26, 2022 14:58 UTC (Mon) by fman (subscriber, #121579) [Link]

Just to clarify: That is is 4 MB of *flash* (persistent storage) and 32 MB of RAM

Small systems

Posted Dec 24, 2022 17:55 UTC (Sat) by rhack (guest, #90448) [Link]

And there is crazy people still around :).
https://github.com/marmolak/gray486linux

4 MB seems to be just guess, but it's possible to boot linux kernel (6.1)
via netboot with 8 MB RAM.

Small systems

Posted Dec 24, 2022 18:46 UTC (Sat) by willy (subscriber, #9762) [Link] (1 responses)

Small systems

Posted Dec 24, 2022 20:27 UTC (Sat) by tamiko (subscriber, #115350) [Link]

Yes, that was the article (not talk) that I remembered!

Small systems

Posted Dec 29, 2022 3:49 UTC (Thu) by disconnect3d (guest, #160000) [Link]

Azure Sphere from Microsoft is a Linux that runs on a board with 4MiB of RAM: https://events19.linuxfoundation.org/wp-content/uploads/2...

Small systems

Posted Dec 24, 2022 23:04 UTC (Sat) by developer122 (guest, #152928) [Link] (8 responses)

It's not the same allocator anymore. As the article notes:

> SLOB is a traditional K&R/UNIX allocator with a SLAB emulation layer, similar to the original Linux kmalloc allocator that SLAB replaced.

SLAB wasn't around in those days.

Small systems

Posted Dec 24, 2022 23:31 UTC (Sat) by heatd (subscriber, #160156) [Link] (7 responses)

Yes it was, SLAB replaced the original kmalloc() around 2.1.something (and the original slab per Bonwick94 replaced the original, more traditional SunOS malloc).

Small systems

Posted Dec 24, 2022 23:50 UTC (Sat) by developer122 (guest, #152928) [Link] (6 responses)

By then people had more ram.

Small systems

Posted Dec 26, 2022 1:43 UTC (Mon) by khim (subscriber, #9252) [Link] (5 responses)

Linux 2.2 was released in 1999. Windows 2000 system requirements of 32MB were considered crazy heavy and that's why Windows 98 survived for quite some time after that

It's possible that Linux 2.2 wasn't supporting 4MB well, but it definitely worked on that amount of RAM. And 16MB was usable with GUI (even if slow).

Small systems

Posted Dec 31, 2022 2:52 UTC (Sat) by patrakov (subscriber, #97174) [Link] (4 responses)

Just to correct - Windows 2000 officially required 64 MB of RAM during installation.

Small systems

Posted Dec 31, 2022 12:47 UTC (Sat) by khim (subscriber, #9252) [Link] (3 responses)

Microsoft Knowledge Base is not official enough?

64MB is something called “recommended minimum” while 32MB if “the minimum supported”. And yes, it was possible to use it with 32MB… but that was too much, that's why PC makers forced Microsoft to release Windows ME which also, officially, needed 32MB, but it was actually possible to use it with 32MB.

Small systems

Posted Dec 31, 2022 14:00 UTC (Sat) by zdzichu (subscriber, #17118) [Link] (1 responses)

You are comparing two lines of Windows system. Win2000 was a continuation of NT4, while 95, 98, ME were consumer line of Windows. They were separate, concurrent lines, which eventually converged into Windows XP.

Small systems

Posted Dec 31, 2022 15:03 UTC (Sat) by khim (subscriber, #9252) [Link]

True, but irrelevant. We are discussing system requirements here. Formally Windows 2000 and Windows ME had the same 32MB minimum memory requirement, but in practice Windows 2000 with 32MB was barely usable while Windows ME was adequate.

Small systems

Posted Jan 2, 2023 14:12 UTC (Mon) by patrakov (subscriber, #97174) [Link]

Indeed, I messed up. If one boots the installer CD with insufficient RAM, it will clearly display an error screen, with 32 MB as the stated requirement.


Copyright © 2022, Eklektix, Inc.
This article may be redistributed under the terms of the Creative Commons CC BY-SA 4.0 license
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds