File descriptor handling changes in 2.6.27

Yes, changing the API/ABI is not a good idea. Extending it is better. But adding flags to
every system call creating file descriptors doesn't really solve the problem. You still have
to have control over all the code that opens file descriptors. And since these new calls are
presumably not standard, any library intended to be portable will have the problems.

I absolutely agree, that close on exec should have been the default to begin with. But
changing it would break standards and compatibility. Instead I think the easiest fix would be
to introduce a new rctl that sets the default value of the close on exec flag.

Any program that does not rely on the default value of the close on exec flags can call this
prctl to secure itself against these race conditions. It does not need changes to any
libraries opening file descriptors and neither to any libraries starting child processes.

Probably it would be a good idea to make dup2 create file descriptors without close on exec if
it replace an existing file descriptor that did not have it, as well as whenever it creates a
new file descriptor with a number less than three. Otherwise too much code would require
changes.

Of course the value of this prctl would have to be reset on any successful execve call.

> if (fork() > 0) {
> 	closefrom(3);
> 	execve(...);
> }

Doing this would mean that you always control the fork()/execve(). But what if the
fork()/execve() is inside a plug-in you don't control (i.e. you can't fix its source)? With
O_CLOEXEC flag, you still get your fd closed. Without it, you don't. No?

So, if I've opened fd 16 and I'm about to run a sub-process which will use this new descriptor
(perhaps I thought to pass '16' as an argument to exec), then I should...

• Use dup to copy fd 3 somewhere else, and ensure all of my code can cope with this, perhaps
by entirely replacing file descriptors and anything that uses them (FILE * etc.) with my own
private indirection...

• Close the now unused fd 3 and replace it with a copy of fd 16 using dup2, then close that
too, incurring all the above problems again

• Call closefrom(4)

Are you sure this is simpler than fixing the design by adding close-on-exec as a potential
property of all descriptors from birth ?

Your approach clearly solves only a fraction of the problem (it doesn't consider fork + exec
by sub-routines you didn't write, e.g. in libraries, which is the most serious problem CLOEXEC
fixes), yet it incurs most of the same costs as the fix that's already been chosen and pushed
into 2.6.27.

That's not to say that closefrom() isn't an interesting API, and one which might be welcome in
Linux, but just that it doesn't actually appear to be a simpler solution, just an incomplete
one.

Surely if you wanted maximum portability you'd just use the POSIX standard
sysconf(_SC_OPEN_MAX) rather than a non-standard OS specific API, and iterate closing all FDs
& ignoring EBADF for those which aren't open. Obviously not as efficient as closefrom(), but
achieves the same end goal without having to #ifdef for OS specific APIs.

        int open_max = sysconf (_SC_OPEN_MAX);
        for (int i = 3; i < open_max; i++)
            close(i);

In any case, neither this, nor closefrom() really address the use case Uli was targetting. The
core issue is you can't guarrentee that everywhere in your program will take care to close()
FDs between fork & exec. Or the more complex scenario where you /want/ to let certain FDs
propagate to the child - the logic of which FDs to propagate to be kept at the place where the
FD is created - likely a totally different piece of code to that which does the fork/exec. Use
of O_CLOSEXEC gives the maximum flexiblity and control over FD handling

> Another simpler approach, but not as efficient, is to iterate over /proc/$$/fd and close all
of the open file descriptors listed there.

I believe that this is a standard approach, and indeed one that udrepper advocates in other
cases.  (Arguably a single-syscall approach would be cleaner, to avoid the dependency on
having /proc mounted at a well-known location, but it's hard for me to imagine that efficiency
is really an issue here -- it's not like reading /proc will hit the disk, so the overhead is
just a few extra syscall entries.)  Certainly this is useful functionality to have.

But you seem to arguing that -- since we have this other useful functionality -- close-on-exec
becomes a useless feature that would be better to ignore than to fix, while I would tend to
think that working close-on-exec and efficient closefrom are both valuable.  It would be
easier to evaluate your argument if you addressed this point directly.  The nominal benefit of
close-on-exec is that it allows locality of control -- the code that creates the fd is (often)
the code that is best prepared to know whether it should be kept local to the process or not.
If you don't have close-on-exec, then working out *which* descriptors should remain open and
which should remain closed requires long-distance coupling between the fork/exec code and all
code which creates file descriptors.  Do you disagree?