| |||||||||||||
O_NOSTDO_NOSTDPosted Aug 27, 2009 0:23 UTC (Thu) by pr1268 (subscriber, #24648)Parent article: In brief
O_NOSTD seems like an odd hack... Any longtime Unix hacker would assume that file descriptors 0, 1, and 2 refer to stdin, stdout, and stderr, respectively—I'm surprised that these aren't in the SUS or POSIX standards... How naïve am I. :-\
(Log in to post comments)
O_NOSTD Posted Aug 27, 2009 1:50 UTC (Thu) by corbet (editor, #1) [Link] That's the point: those file descriptors are defined in the standards. That's why application programmers expect them to be set up correctly. But nothing forces that to happen, and other standard-ordained behavior says that new files are always assigned the lowest available file descriptor.
O_NOSTD Posted Aug 27, 2009 2:13 UTC (Thu) by BenHutchings (subscriber, #37955) [Link]
SUS also says that exec() of a setuid executable may assign file descriptors 0, 1, and 2 to unspecified files if they are not already assigned.
O_NOSTD is utterly pointless as it would need to be added all over the place, including libraries outside the application programmer's control.
O_NOSTD Posted Aug 27, 2009 9:47 UTC (Thu) by epa (subscriber, #39769) [Link]
I think if the kernel developers don't have the guts to just turn on O_NOSTD
for all open() calls by default, then maybe the C library will. (With an O_ALLOW_STD flag for those cases like the shell where you really do want to fiddle with the standard file descriptors.)
O_NOSTD Posted Aug 28, 2009 0:53 UTC (Fri) by willy (subscriber, #9762) [Link]
This was already fixed years ago after Chuck Lever came up with the problem originally. Try running a setuid program with a standard file descriptor closed. You'll find that libcrt0 opens them again.
O_NOSTD Posted Aug 27, 2009 2:54 UTC (Thu) by foom (subscriber, #14868) [Link] Indeed. It, O_NOSTD, like O_CLOEXEC before it, are almost entirely pointless extensions that basically only gnu coreutils and glibc can ever use. And when I think of all the functions that needed extra arguments added just to be able to pass the O_CLOEXEC flag...ugh.IMO, Instead of O_CLOEXEC, glibc should have added a function to close all but a specified list of fds. And apps or libraries could call this themselves after fork, before exec. Obviously you can write this yourself already by iterating over the files in /proc/self/fds, but I just want to write:
int keep_fds[] = {0,1,2,-1}; close_everything_but(keep_fds);
and be done with it! The list of fds to keep across a given call to exec is *always* going to be small, and known to the program calling exec. And then, it's also trivial to write the compatibility library for non-glibc platforms... Sigh. And now this O_NOSTD...so instead of just calling a single function at program startup, I have to change every open call in every library that I use to use O_NOSTD? Yeah, right. If they're that concerned about number-of-syscalls-per-program-start, why not just make a single syscall for "make sure I have fds 0,1, and 2 open to a fd, otherwise open /dev/null there". Yay, now I only have a single syscall on program startup. That'd make much more sense than this proposal...
O_NOSTD Posted Aug 27, 2009 11:34 UTC (Thu) by hppnq (guest, #14462) [Link] And now this O_NOSTD...so instead of just calling a single function at program startup, I have to change every open call in every library that I use to use O_NOSTD? Yeah, right. There's nothing that forces you to use O_NOSTD. If you want to manually make sure that your fd's are numbered properly -- which has to be a bit of a kludge considering the nature of the problem of having regular fd's with a special meaning -- then it should still work as before.
Then all we'd need is CAP_DO_WHAT_I_MEAN and O_NOSH*T. There's no fun in that.
O_NOSTD Posted Aug 27, 2009 16:31 UTC (Thu) by daney (subscriber, #24551) [Link]
For a multi-threaded program, O_CLOEXEC is required if you want race free operation. Hardly pointless.
The O_NOSTD problem, on the other hand, can be worked around entirely in user space.
O_NOSTD Posted Aug 27, 2009 18:09 UTC (Thu) by foom (subscriber, #14868) [Link]
> For a multi-threaded program, O_CLOEXEC is required if you want race free operation.
> Hardly pointless.
No, it's not required. After a fork, your program is no longer multi-threaded, and you can at your
Third-party libraries Posted Aug 27, 2009 20:24 UTC (Thu) by quotemstr (subscriber, #45331) [Link] The problem is then that a third-party library might not be notified between a call to fork() and exec(). That's what pthread_atfork is for, but not everyone links against libpthread.Even with pthread_atfork, however, you can race. Consider:
[library code]
static int fd = -1;
void mylib_atfork() {
close(fd);
}
void mylib_dosomething() {
fd = open(...);
do_something_with_fd(fd);
}
If the fork happens between the return from
The real userspace solution would be for programs to just close unknown file descriptors between Now, on the other hand, this O_NOSTD business is pure junk that will uselessly take up a valuable flag bit for all eternity.
Third-party libraries Posted Aug 27, 2009 21:33 UTC (Thu) by foom (subscriber, #14868) [Link] The real userspace solution would be for programs to just close unknown file descriptors between fork and exec. But they don't, so O_CLOEXEC is a decent facility for defensive library programing. Yes, this is what I've been saying -- see previous comment regarding "close_everything_but". The bug is in the code that calls fork/exec, not the code that opens a file descriptor! Comments like this one just show how insane this whole thing is. The *bug* there is that libuuid doesn't close fds before execing a long-lived daemon! It should not be the responsibility of everyone to open all their fds with O_NOEXEC.
Third-party libraries Posted Aug 27, 2009 21:51 UTC (Thu) by giraffedata (subscriber, #1954) [Link] I don't see it. Never mind pthread_atfork() -- after all, the issue is exec, not fork. The alternative to O_CLOEXEC for a library that opens files under the covers would seem to be a prepare_for_exec() function exported by the library. The user makes sure he calls that before any exec(). As for the multithreaded program, it already has to serialize access to these file-descriptor-using functions anyway (you wouldn't want two threads opening that file at the same time), so it might as well synchronize prepare_for_exec() with those. This serialization could be done either in the library (i.e. the library is thread-safe), or outside. Oh, and here comes pthread_atfork(): it can make sure the serialization mechanism survives a fork that may precede the exec.
Third-party libraries Posted Aug 27, 2009 22:54 UTC (Thu) by Los__D (guest, #15263) [Link]
Besides "prepare_for_exec()" being a horrible interface, do you really think that programmers that doesn't care to close unknown fds, would care to call it?
Third-party libraries Posted Aug 28, 2009 1:44 UTC (Fri) by giraffedata (subscriber, #1954) [Link] Besides "prepare_for_exec()" being a horrible interface, do you really think that programmers that doesn't care to close unknown fds, would care to call it? That's beside the point. I was responding to a claim that there is no way to write a correct multithread program involving a third party library that opens files without O_CLOEXEC. And that that distinguishes O_CLOEXEC from the proposed O_NOSTD. If on the other hand you just want to argue that O_CLOEXEC is convenient, with or without threads, then you're putting it in the same class as O_NOSTD.
Third-party libraries Posted Aug 28, 2009 5:39 UTC (Fri) by Los__D (guest, #15263) [Link]
There is no other way for the library author to make sure that his library opened fds are safe.
It is hardly "convenience", but good library programming style, to make sure that internal data stays internal.
I would go so far as argue, that the naïve library user that doesn't close the fds, isn't entirely an idiot for expecting not to be responsible for library data.
Third-party libraries Posted Aug 27, 2009 23:10 UTC (Thu) by quotemstr (subscriber, #45331) [Link] Or you just do this (error checks omitted):
#include <pthread.h>
static int fd = -1;
static pthread_mutex_t fd_lock = PTHREAD_MUTEX_INITIALIZER;
static void mylib_beforefork() {
pthread_mutex_lock(&fd_lock);
}
static void mylib_afterfork() {
pthread_mutex_unlock(&fd_lock);
}
void mylib_dosomething() {
pthread_mutex_lock(&fd_lock);
if(fd == -1) {
pthread_atfork(mylib_beforefork,
mylib_afterfork,
mylib_afterfork);
fd = open(...);
fcntl(fd, F_SETFD, O_CLOEXEC);
}
pthread_mutex_unlock(&fd_lock);
do_something(fd);
}
Third-party libraries Posted Sep 8, 2009 9:06 UTC (Tue) by jlokier (guest, #52227) [Link]
Doesn't work if the thread uses vfork() :-)
O_NOSTD Posted Aug 27, 2009 20:11 UTC (Thu) by kjp (subscriber, #39639) [Link]
Finally a voice of reason.
O_NOSTD is a joke. WAAAAAAAAAAAH I don't want to call some syscalls in gnu coreutils cp. So yeah, lets add more flags to open and freaking *socket* while we're at it?
WAAAAAAAAAAAAAAH I don't want to close fds in my pre exec code so I need CLOEXEC added to Everything. Jeez, even a per process flag to say no inherit by default would be better than that crap.
God, Linux is turning into Windows crapware. Please Linus...axe this crap.
O_NOSTD Posted Sep 8, 2009 8:56 UTC (Tue) by jlokier (guest, #52227) [Link]
I proposed that per-process cloexec-by-default flag some years ago, and it was (rightly) shot down for breaking third party libraries that internally create descriptors, spawn child processes and _expect_ those processes to inherit those descriptors.
Personally I'd rather break those libraries than have descriptor leaks, security holes and more complicated APIs. After all what we're doing now pretty much requires all libraries to be changed anyway, only this way, it's silent breakage in the meantime.
Old and well-known Posted Aug 27, 2009 14:18 UTC (Thu) by dwheeler (guest, #1216) [Link] The issue this patch is trying to address is noted in "Secure Programming for Linux and Unix HOWTO", section 5.3 (File Descriptors): "[do] not assume that standard input (stdin), standard output (stdout), and standard error (stderr) refer to a terminal or are even open." I don't know if this is the best way to go about it, but I applaud the idea of trying to make it easier to write correct software.
|
|||||||||||||