How many groups do you need?

This problem came to Rusty Russell's attention; it seems there is a client down under with a need for almost 200 simultaneous groups. The client is running Samba-based servers and having to deal with NT's hierarchical group structure. So Rusty sent out a patch making the maximum number of groups adjustable via a sysctl file. If a process expands to more than 32 groups, the code will dynamicly allocate an array to store them all.

Rusty's patch asked for comments, and got them in the form of a rather exasperated note from Tim Hockin. Tim has been posting a very similar patch for quite some time; LWN included a version of this patch back in October, 2002. Despite repeated posts, the patch had never been picked up and merged.

In response to Tim's note, Rusty set about the task of merging the two patches. It turns out that Tim's needs were a little different, however; his customers need, for some reason, to have processes be members of up to 10,000 groups. That changes things a bit. So, among other things, the combined patch will use vmalloc() to allocate the array of group IDs should an attempt with kmalloc() fail. Since vmalloc() can create large virtual arrays in kernel space through the use of page tables, it works better when large, virtually contiguous chunks of memory are required.

That approach didn't fly with Linus, however; he asked: "Have you been looking at glibc sources lately, or why do you believe that we should encourage insane usage?" These comments have led Tim to take a turn reworking the patch. In his version, the kernel maintains a list of individual pages full of group IDs when the need arises. The lists are shared and reference counted (to help avoid unnecessary copying); they are also sorted so that searches are quick. Given that Linus is "definitely happier about this one," there is a distinct possibility that it may yet find its way into the 2.6 kernel. Sometimes, getting your patch in is just a matter of waiting for somebody else to reimplement it.