Flags as a system call API design pattern

The problem with open() and O_TMPFILE was that the kernel treated unknown flags as optional, but an application using O_TMPFILE would want it as *mandatory*. Such an application would have no way to tell whether O_TMPFILE actually was honoured, because the open() would succeed, regardless of whether kernel recognised that flag (e.g. if you run the application on an older kernel). You can't test and retry, because open generally didn't treat unknown flags as a failure.

Generally, to be able to introduce new mandatory flags, while allowing optional flags, you need either to distinguish between mandatory flags and optional in the API in some way, or you need some other way to allow the application to feature-test at runtime (but what if it forgets to do this, and then gets run on an old kernel?).

Otherwise, you need to rely on being able to find an API useage-specific hack that happens to work, as was done for open/O_TMPFILE, by also setting some other unrelated flags that *would* together cause an error on older kernels. ;) These kind of API-specific hacks might not always be available.

> The recipient rejects any nonzero reserved bits in the first half and ignores any reserved bits in the second half.

While I appreciate the elegance of this approach, it does have a major flaw: since the recipient ignores anything it doesn't recognize in the second half, senders are free to put whatever random data they want there. Later, when new optional flags are defined, these applications break.

This has happened multiple times in the Linux userspace APIs, and since breaking previously-working user applications isn't allowed no matter how they abuse the APIs, you effectively can't redefine any bit you've previously ignored. If you don't require a specific value for unused bits, you won't be able to use them in any later versions. Better to just reject unrecognized bits and leave userspace to implement a fallback when the syscall fails.

> Imagine a web server written by someone who knows only Firefox and tested only with Firefox. The server detects at run time that the browser is "Iceweasel." The author never heard of Iceweasel. Should the program send the Firefox-oriented data to Iceweasel and assume it is smart enough to emulate Firefox, or tell the user it doesn't know how to drive Iceweasel and avoid a possible disaster?

To answer that you would need a protocol specification. Doing this properly requires senders and receivers to work from the same spec. If you're just inferring one possible spec from the way you've seen Firefox behave then you can make up whatever arbitrary rules you want, so long as Firefox passes them, though it's safest to bail out early rather than continue after seeing something unexpected.

Normally, of course, you'd write your web server to the HTTP specification, not a particular browser, and a browser reporting itself as "Iceweasel" is still acting within the spec and thus not giving you any reason to error out.