| |||||||||||||
Process containersProcess containersPosted May 31, 2007 18:20 UTC (Thu) by utoddl (subscriber, #1232)Parent article: Process containers
This is a reimplementation of groups, but with more features attached than simply "you may or may not access this local file or directory". It looks like an extention of what OpenAFS's PAGs (process authentication groups) give you -- and what has kept their camel out of the kernel tent for years.
(Log in to post comments)
Process containers Posted May 31, 2007 21:51 UTC (Thu) by IkeTo (subscriber, #2122) [Link] I have some difficulties understanding your comment. I've looked at OpenAFS for a tiny bit of time, my impression is exactly what you say: it is a filesystem, and PAG is a system for you to tell the filesystem who you are. How is this is anything to do with process container, which seems to be mainly a tool for system administrators or service startup scripts to limit the amount (rather than identities) of system resources like CPU and network bandwidth (rather than files) that the process can use, based on the "echo" commands executed by administrator manually or via scripts (rather than via the user creation and login procedure)?
Process containers Posted Jun 1, 2007 14:02 UTC (Fri) by utoddl (subscriber, #1232) [Link] Fair enough. Let's see if I can connect the dots.
Ignore for the moment the implementation of either groups or process containers, and just look at the semantics. A given process can be in multiple groups; child processes inherit groups from their parents; special circumstances can alter which groups are added or dropped from a process' group list. Likewise for processes in containers. If you were to replace the labels in the diagram from the article with numbers, you could implement the processes "in-container-x" property with the existing group mechanism.
Process group lists have always been a light-weight set of properties that processes carry around and pass on through fork(). The fact that (almost) nothing except file systems uses them not withstanding, it seems somebody finally noticed that the semantics of passing around properties in this way is useful for other things like processor affinity, throttling, and other things the article mentions.
AFS (and later OpenAFS) piggy-backed process authentication group membership on the group mechanism. The AFS kernel module would add a group (actually a pair of group numbers) to a processes group list to create a new PAG. Child processes would inherit these just like any other groups through fork(), but no file system -- including AFS -- used these group numbers to check file access. Instead, AFS would use these numbers to associate a process with a specific PAG, which is just a set of processes which share a cached token. The token *is* used for access control, but membership in a PAG is just a property like any other group membership. The semantics for group membership and inheritance just happens to be exactly what you want for an authenticated file system like AFS.
Besides that, though, these semantics happen to be exactly what you want for processor affinity, bandwidth throttling, CPU limits, etc. But rather than piggy-backing these capabilities onto the existing group mechanism as AFS did, they've invented another parallel mechanism for passing process properties around. Group membership and process container "in-ness" are just properties after all.
To be fair, the time tested group mechanism has its limits. Group lists are rather short (or thay were last time I ran into that issue). They also aren't explicitly hierarchical like process containers (though what that buys us wasn't immediately obvious to me upon reading the article). It wouldn't surprise me if the old UNIX groups weren't eventually reimplemented as containers. Then you could eventually have hierarchical UNIX groups!
The point of my "camel in the tent" comment was that the way AFS piggy-backed the process properties it was interested in on top of groups was met with skepticism and sometimes out-right contempt by some kernel developers. The reasons include NIH (Not Invented Here -- AFS predates linux by a fair few years), the kernel module itself is maintained out-of-tree (it builds for several OSes other than Linux and not just on the current versions, so it contains a lot of "cruft", at least in the eyes of the kernel hard-core), and it's hobbled by being under the IPL license (basically IBM's GPL with a "we can take it proprietary later if we want" clause). AFS on recent kernels has switched to using keyrings -- yet another special purpose property propagation mechanism -- to implement PAGs, but the other factors still keep AFS/OpenAFS on the outside looking in.
The kernel goes through this periodic process where some new functionality is added, then somebody points out that this new thing and this other old thing have similar operations, then some common code is developed that they can both use or one gets folded into the other. We've seen it over and over, and I wouldn't be surprised to see it happen with groups and process properties.
Process containers Posted Jun 1, 2007 14:33 UTC (Fri) by IkeTo (subscriber, #2122) [Link] > Process group lists have always been a light-weight set of properties that> processes carry around and pass on through fork().
Can you clarify a little bit? AFAIK, there are two concepts of "groups" in the current kernel. One is called the "process group", as is set by setpgid(). Each process belong to one group (rather than many). That group is used for signal deliveries, allowing users to send signals to all processes of a group, either by explicit "kill" command/system call, or by using a special terminal character. The other is the "supplementary group IDs", as is set by setgroups(). Each process has a small number of those. It is used by system administrators to control the files or other resources that each user can access. The numeric values are meaningful not only to the kernel, but to the admin as well. They assign each user a list of such group IDs in /etc/group, and the login procedure will assign the login shell (or X session) process to use that list. There is also the session ID, but that doesn't seem like being what you mean.
So by "process group" do you mean one of these existing concepts, or is there yet another group concept in the process carried by the process that either is hidden in the kernel or that I forgot?
Process containers Posted Jun 1, 2007 15:06 UTC (Fri) by utoddl (subscriber, #1232) [Link] I was talking about supplementary group IDs as set by setgroups().In the particular AFS context, when the older libafs kernel module loaded, it would swipe the setgroups entry in the sys_call_table (?sp) so it could handle the necessary details of associating an AFS PAG, token, and process. It was an admitted hack, but one that has worked in various forms for over a decade in a half dozen major flavors of UNIX. Other methods were invented for Linux when the kernel police make the sys_call_table read-only. BTW, this was/is another reason to dislike what AFS does with the supplementary group list. It's rather disconcerting to do "id -a" and see groups with no associated names, but that's common if your shell is in a PAG. Behold: $ id -a uid=12428(utoddl) gid=12428(utoddl) \ groups=10(wheel),1511(atnid),12428(utoddl),1094942735
Process containers Posted Jun 1, 2007 16:40 UTC (Fri) by IkeTo (subscriber, #2122) [Link] Thanks. I understand your posts now. But I don't think I like the idea. At the very least, I don't think it reasonable to arbitrarily allocate user ID space to something completely unrelated to users this way. And of course it provides a horrible interface to users.
Process containers Posted Jan 25, 2008 17:35 UTC (Fri) by rijrunner (guest, #49442) [Link] Well, my read of this is a bit different. This looks to me like a side-effect of the virtualization changes to the kernel and how Oracle works. Basically, virtualization requires carving out a set of system resources (memory, cpu, disk, network, etc, etc) and assigning them to a virtual machine to manage. The key is that the kernel has to be able to define parameters that can be isolated and restricted in their size and scope. What the container concept seems to be - which I could be misunderstanding based on only a cursory reading - is extending that ability to isolate resources to processes running within the base OS. ie, if you are putting hooks into the kernel to be able to define and limit system resources for virtual machines, why not extend it to processes and resources at the OS level?
|
|||||||||||||