LWN: Comments on "The managed resource API"

The managed resource API

rwmj — Fri, 02 Mar 2007 09:07:16 +0000

There was a concurrent Caml Light, as shown in the link you posted.

My problem is not that the kernel devs have decided on and rejected garbage collection, but that they're implementing a really bad form of garbage collection without any rational investigation into the field.

Rich.

The managed resource API

joib — Thu, 01 Mar 2007 22:10:10 +0000

Then again, the Ocaml runtime is single-threaded, and the usual response to requests to make it multithreaded is that it would make the GC massively slower. Or to be precise, multiple threads are supported but there is a global lock so that only one thread may be executing ocaml code at a time. See e.g. http://caml.inria.fr/pub/ml-archives/caml-list/2002/11/64...

I mean, GC would certainly be nice but while I don't have any GC implementation experience I'd think the effort to write a concurrent GC scalable up to 2048-way NUMA machines (or whatever is the biggest single image system Linux runs on at the moment) is far from trivial.

The managed resource API

rwmj — Thu, 01 Mar 2007 21:55:46 +0000

There is a lot of details about the guts here, written by one of the aforementioned experts:

http://cristal.inria.fr/~doligez/caml-guts/

I'm afraid that I don't know of any readable introduction to the OCaml internals. However if you are
really interested then I'd recommend you read this chapter from the manual:

http://caml.inria.fr/pub/docs/manual-ocaml/manual032.html

It took me about a year of on-again off-again reading to understand exactly how clever the runtime
representation of types is which the above chapter describes.

For a general introduction to OCaml, http://www.ocaml-tutorial.org/

Rich.

The managed resource API

massimiliano — Thu, 01 Mar 2007 21:29:39 +0000

I'm interested!

Any link to a "readable" paper with a comparison of their approaches, or anyway some more detailed information?

The managed resource API

rwmj — Thu, 01 Mar 2007 19:57:10 +0000

That was a bit of a dig at Java. There are a number of architectural problems with the Java runtime which is why it's taken an enormous effort to get something which is still (IME) pretty sluggish except under very special conditions (long running server-side processes).

Compare to the OCaml GC. OCaml is maintained by a small team of developers and regularly performs close to and often faster than equivalent C programs. They made a lot of very smart choices with runtime representations of values, and it helps that they have a couple of world-leading GC academics writing the code.

Rich.

The managed resource API

quintesse — Thu, 01 Mar 2007 18:40:17 +0000

Java has a bad GC?? Do you have any kind of basis for that statement because I was under the impression that it was doing a pretty good job. Sun engineers are working constantly to improve Java's GC, often not just by incremental changes but also by including completely new algorithms. So I'm sure they would be delighted if someone could point out to them that there are far superior GCs out there!

The managed resource API

rwmj — Thu, 01 Mar 2007 08:44:13 +0000

A garbage collector is a 'Fire and forget' memory allocation strategy. What is described here is a memory allocator that groups together allocations so that you can tell the system "when this piece of memory here is recycled, then these pieces over here are no longer needed either".

That shows a poor understanding of garbage collectors. In fact a GC has precisely this information - that groups of memory are related and that when one piece of memory is no longer reachable, that causes other memory allocations to be unreachable too. This is encoded implicitly in the relationship of pointers between memory areas. All that is happening here is that the kernel devs have made this relationship explicit, inefficiently.

Rich.

The managed resource API

rwmj — Thu, 01 Mar 2007 08:40:28 +0000

The problem is that reference counting is the heaviest garbage collector there is. Real, well-implemented GCs[1] are much better than reference counting.

Rich.

[1] And they are rare. Java and Emacs have really bad GCs, but because everyone is familiar with them, they assume that GC itself is bad.

The managed resource API

nix — Mon, 26 Feb 2007 16:53:51 +0000

There already is at least one (in net/unix/Space.c, IIRC). If you don't garbage-collect fds in transit down AF_UNIX sockets, you open a trival DoS hole. (But that's a very special-purpose GC so may not count.)

The managed resource API

liljencrantz — Sun, 25 Feb 2007 12:13:26 +0000

I disagree. A garbage collector is a 'Fire and forget' memory allocation strategy. What is described here is a memory allocator that groups together allocations so that you can tell the system "when this piece of memory here is recycled, then these pieces over here are no longer needed either". That is sometimes exteremely useful, since it turns out that allocation time is often a time when you have pretty detailed knowledge of when something will die.

The managed resource API

k8to — Sat, 24 Feb 2007 10:11:41 +0000

If you consider a reference counter to be a type of garbage collector, it seems they are slowly doing it. I don't realy expect a heavier garbage collector to appear anytime soon, however.

The managed resource API

flewellyn — Fri, 23 Feb 2007 23:28:45 +0000

So we've come incrementally closer to having a garbage collector in the kernel, I see.

Why they don't just bite the bullet and DO it, I have no idea.

The managed resource API

nlucas — Thu, 22 Feb 2007 22:54:58 +0000

I have not seen the final patch, but it should only affect initialization and cleanup phases, so it would at most make module loading/unloading slower.

I guess that if one doesn't load all kernel modules right at once (and most distros with udev don't) the effect will not be measurable.

The managed resource API

jospoortvliet — Thu, 22 Feb 2007 19:47:03 +0000

I wonder if this has any performance impact?

The managed resource API

nix — Thu, 22 Feb 2007 15:19:46 +0000

It seems odd that an ease-of-use interface like the pcim_*() interface provides only partial coverage of the interfaces it's supposed to replace. Users are supposed to remember to use e.g. pci_disable_device(), but if they do the orthogonal thing and use pci_enable_device() as well, they've just introduced a bug.

Surely the sensible thing to do is to have an inline wrapper that introduces pcim_disable_device() that just thunks to pci_disable_device() (and so on for other non-covered functions in this set)? This wouldn't introduce any extra overhead, and would increase consistency.