Yes, changing the API/ABI is not a good idea. Extending it is better. But adding flags to
every system call creating file descriptors doesn't really solve the problem. You still have
to have control over all the code that opens file descriptors. And since these new calls are
presumably not standard, any library intended to be portable will have the problems.
I absolutely agree, that close on exec should have been the default to begin with. But
changing it would break standards and compatibility. Instead I think the easiest fix would be
to introduce a new rctl that sets the default value of the close on exec flag.
Any program that does not rely on the default value of the close on exec flags can call this
prctl to secure itself against these race conditions. It does not need changes to any
libraries opening file descriptors and neither to any libraries starting child processes.
Probably it would be a good idea to make dup2 create file descriptors without close on exec if
it replace an existing file descriptor that did not have it, as well as whenever it creates a
new file descriptor with a number less than three. Otherwise too much code would require
Of course the value of this prctl would have to be reset on any successful execve call.