LWN: Comments on "Choosing between portability and innovation"

Choosing between portability and innovation

scientes — Tue, 20 Mar 2012 20:07:34 +0000

> Linux is about features, while BSD emphasizes design.

While I can't exactly agree with this flame, I will note that this very thing your are describing with kqueue/epoll also happened with /dev/cryto which was on OpenBSD and there even existed a Linux patch, and then Linux did their own thing with AF_ALG. (now, they did do some benchmarking...)

Choosing between portability and innovation

gvy — Sun, 24 Jul 2011 17:29:24 +0000

> OSS came before ALSA and was technologically superior to ALSA.
I'd ask for some evidence.

And regarding
> FreeBSD (at least) also has had software mixing for well over a decade
> now (vchans) and Linux is still struggling with this functionality
-- I've actually forgot since when alsa dmix just works, maybe 5 years or so? (esound would do either back in 1998 IIRC)

> pulls this stuff every release, which is either biannually or annually!
ORLY

Centralized vs Decentralized

JasperWallace — Sat, 02 Apr 2011 02:45:38 +0000

> And they still seem to think CVS is a good idea for version control, though they do try to offer Subversion as an alternative for the radical young kids.

The problem the BSD's have is they have the entire history for kernel+userland+X in one repo and the history. If they switched to a dvcs then every developer would have to have have a copy of the history which would take massive amounts of disk space.

Also converting 15 years+ of history is not straight forward.

NetBSD does have a git repo that shadows the main cvs modules:

http://mail-index.netbsd.org/current-users/2009/10/13/msg...

Choosing between portability and innovation

daenzer — Thu, 24 Mar 2011 12:24:35 +0000

Not sure GEM is a really good example, as the people behind it seem to be firmly in the 'who cares about portability, only Linux matters' camp. In fact, it was with GEM that the DRM code stopped being developed shared between Linux and the BSDs.

This is all the more interesting as one of those people was previously a major FreeBSD graphics driver developer. Something rather dramatic seems to have happened there at some point, still not sure what that was.

Choosing between portability and innovation

daenzer — Thu, 24 Mar 2011 12:13:12 +0000

Nice article, just some minor corrections for the part about X.org and KMS.

Konstantin Belousov shouldn't need to implement any new DRI drivers for BSDs. Once the KMS and GEM APIs are available in the kernel, the only userspace changes required (if any) should be in libdrm(_intel).

AFAIK TTM stands for Translation Table Manager.

Gallium3D isn't tightly coupled to the Linux kernel at all. In fact, it's an explicit goal of it to cleanly separate parts which are specific to an OS (and windowing system etc.) environment from parts which are specific to hardware (and each in turn from parts which are specific to APIs).

Choosing between portability and innovation

jcm — Sun, 20 Mar 2011 18:01:31 +0000

The solution (on the POSIX front) is to get newer Linux APIs standardized before forcing everyone to switch to them. It is totally unacceptable to say "we're number one, you do it like this". I'm not singling anyone out, but it's just plain wrong to have that attitude. Not only does it hurt portability, but it also makes it increasingly difficult to document how Operating Systems like Linux are supposed to work. Ever wonder why (even given the demise of the publishing industry) there are so few books coming out these days for developers to read? It's because this train is moving so fast that only those riding it have a hope of making the next stop.

Choosing between portability and innovation

jcm — Sun, 20 Mar 2011 17:52:28 +0000

Awesome story. Well said. What is needed, in my personal opinion, is a giant push-back against some of the trends in the Linux area, and back toward standardization across platforms. In the process, a lot of "progress" is going to have to be undone, but that's a good thing.

Re: second class citizens

gvy — Sat, 19 Mar 2011 18:50:17 +0000

NB: writing articles on "treating non-Linux operating systems as second-class citizens" doesn't count! :)

Would be actually interesting to know why move from Linux to BSD (I suppose that the author isn't prone to "linux sux freebsd is tru unix" meme), *and* to moan subsequently on it lagging behind.

When we prepared our yearly FLOSS conference six years ago, some of local *bsd folks moaned that there were too few *bsd reports, and that Tux-branded packets for papers which were printed and generally available then are unacceptable (reminds me of "political correctness"). They were advised to, well, bring in reports, and take care of funding and printing custom packets. They organized a "registration" flashmob tagged "OSDN is not Linux" instead...

So my suggestion stays the same: either help or don't pop up.

Choosing between portability and innovation

viro — Fri, 11 Mar 2011 14:04:40 +0000

Dear duhveloper. Do not use the worst misdesigns shoved down our throats by Scamantec and its ilk. Yes, I mean fanotify(). No, the fact that it's got in does *NOT* mean it's good to use or should be enabled. Sigh... Lusers - can't live with them, can't dispose of the bodies without violating hazmat regs...

Choosing between portability and innovation

nix — Fri, 11 Mar 2011 12:40:34 +0000

Yeah, great, you can do all the flaming horror of sound mixing in kernel space. Thanks but no thanks.

Choosing between portability and innovation

nix — Fri, 11 Mar 2011 12:36:41 +0000

Virtually of the things you whine about in this email haven't had any inkling of change in years, and to be honest it doesn't matter a damn who 'did it first', it matters which works better. And in many of these cases, BSD loses massively precisely because they don't reinvent.

- /dev filesystem: two implementations (you often have to write one and throw it away to learn how *not* to do it). One upstream, architecture pretty much unchanged since day 1 (though this all depends on what you mean by a '/dev filesystem setup': one of the lessons of devfs was to try to stick to the traditional /dev layout to avoid breaking things unnecessarily). But perhaps you're referring to the shift to a single upstreamed udev database? That's *optional* and several major distros (Debian for one) don't use it.

- RC init systems: again, ambiguous. sysvinit was *the* init system for many decades, but is terribly limited, so much so that most of its functionality is almost unused. Init scripts, well, we started with BSD's 'giant shell script' approach. The difference is, BSD stuck with it even though it sucks massively in systems with package managers. Linux moved on, and even now the init script format defined right after that is still accepted by the latest bleeding-edge init replacements.

- non-root X: BSD has revoke(). Linux doesn't. Not a problem of reinvention nor NIH, just 'this is a really hard problem'. Most of the BSDs still don't have non-root X because they are NIHing with respect to each other.

- Packet filtering: the last packet filter redesign was something like eight years ago. I know of iptables packet filters that've been working unchanged for all that time. Huge code churn this is not.

- USB stacks: how many? well, er, one (which didn't work very well until hotplugging worked properly: the same hotplugging you damn elsewhere). Yeah, we have support for things like USB 3.0. That's not a 'new stack', that's changes to an existing stack to support new hardware.

Sheesh.

Choosing between portability and innovation

gcooper — Fri, 11 Mar 2011 12:04:16 +0000

I stand corrected on that point; thanks for the clarification.

Choosing between portability and innovation

patrick_g — Fri, 11 Mar 2011 09:31:37 +0000

>>> OSS came before ALSA and was technologically superior to ALSA. It was in Linux for a while, and has been adopted in *BSD for a long period of time, but was abandoned for licensing disagreements (BSD licensing)

Rubbish ! The Linux community abandoned OSS because the main dev (Hannu Savolainen) made his support for newer sound devices and improvements proprietary.

Choosing between portability and innovation

phoenix — Fri, 11 Mar 2011 03:46:19 +0000

As has been shown by the BSDs, you do not need PulseAudio, or any other sound daemon, sound server, etc, in order to have non-blocking, multi-source/single-output sound setup.

Just because Linux can't do it, doesn't mean it's not possible, nor that it should be avoided.

Choosing between portability and innovation

gcooper — Fri, 11 Mar 2011 01:52:58 +0000

Some history review

kqueues came before epoll and some of the equivalent functionality in inotify.
BSD sockets came before the Linux implementation (obviously)? XSI streams are basically dead, hence FreeBSD doesn't make an effort to implement them, along with some other deprecated XSI functionality.
OSS came before ALSA and was technologically superior to ALSA. It was in Linux for a while, and has been adopted in *BSD for a long period of time, but was abandoned for licensing disagreements (BSD licensing) and bad blood between Linux devs and the OSS devs and because OSS was done largely within the kernel. FreeBSD (at least) also has had software mixing for well over a decade now (vchans) and Linux is still struggling with this functionality (why outsource to PulseAudio)? What about arts, ESS, etc?
systemd is another take on something that upstart was supposed to answer by itself, which is in turn something that SysV init did back in the day (and a simple init clone already does to a large degree on FreeBSD without as much complexity for well over the past decade); granted it has to use devd and devfs for some minor help with event notification, but have you tried creating jobfiles in upstart? Oh yeah, and OSX has launchd, which is probably superior to the rest of the init-like daemons, but isn't really portable after Leopard (surprise, surprise), so it hasn't moved outside of OS X.

So with that in mind, how is tying Freedesktop to Linux better than trying to be somewhat portable (if I screwed up the history lesson, feel free to correct me)?

Code churn

Brute forcing software design through several revisions of code or several forked projects detracts from the collaboration effort and I would question whether or not true innovation is being made, or if it's just code churn.

How often does OSX, Windows, etc dramatically change system interfaces or how components interact with one another, and how do end-users receive the changes? I know based on my experience that people absolutely loathed the Windows 2000 -> XP and XP -> Vista/7 transitions and those only happened every 3-5 or so years. Freedesktop and other associated projects (Gnome, KDE, LXDE, XFCE) pulls this stuff every release, which is either biannually or annually!

Impact of churn and becoming less portable

Do the devs have the performance figures to support the fact that the new software truly functions faster?
Can the devs qualify all of the wasted hours...
- Doing the initial development and unit testing.
- Effort needed to track the versions qualify the software in multiple releases by package maintainers?
- Effort needed to track changes required for end-users who depend on Linux or other platforms like BSD, OpenSolaris (now IlluminOS), etc?

An example of good collaboration / design

I find the work that Intel is doing with GEM (even though it's a bit misguided in tying itself to Linux, oh well) to be really good work, because it's a relatively abstract framework for their display drivers. I will applaud them for this effort and others because several folks in Intel have also works on making their NIC drivers and other things sane for non-Linux Unix users.

Conclusion:

I was a devout Gentoo Linux user for 5 years and gave up running it on my primary system because of the fact that some developers that develop for core components in Linux lose sight that an OS distribution is so much more than a kernel, and stuff broke all too frequently. If Freedesktop devs aren't willing to slow down and think more -- and thinking about portability is one of those things -- then Unix won't be my primary desktop / portable platform, I won't be the only user (BSD, Linux, etc) dropping FreeDesktop software like a hot rock. There's always screen/tmux with SSH on my Mac if I'm pushed to do things that way, so even if folks go the way of Wayland I'll have a way to get my work done.

Choosing between portability and innovation

phoenix — Thu, 10 Mar 2011 23:01:12 +0000

Hrm, no, if you actually look back at the development history, you'll note that the Linux devs need to rid themselves of NIH Syndrome.

Working, stable, performant wireless networking stacks were developed on BSD first. How many wireless stacks has Linux gone through so far?

Working, stable, performant device detection and /dev filesystem was developed on FreeBSD first. How many /dev filesystem setups has Linux gone through so far, and still re-writing them to this day?

Working, stable, performant RC systems were developed on BSD first. How many RC init systems has Linux gone through so far, and is still re-writing them to this day?

Working, stable, non-root X setups were developed on OpenBSD first. Canonical got to brag about being the first Linux distro maker to accomplish that this past year. But it's still no working right.

Working, stable, performant and portable packet filtering was developed on BSD first. How many packet filters has Linux gone through now?

Working, stable, performant USB stacks were developed on BSD first. How many different USB stacks has Linux gone through now?

Working, stable, in-kernel multi-channel, no-locking, sound mixing was available on FreeBSD before ALSA was ever even thought of. How many
"sounds daemons", "sound servers", and sound stacks has Linux gone through now?

And the list goes on. Re-writing sub-systems with every release of the kernel is not "innovative".

Centralized vs Decentralized

ldo — Thu, 10 Mar 2011 01:42:12 +0000

To me, this BSD-versus-Linux thing revolves around the difference between a centralized model of development versus a decentralized one.

Linux is just a kernel, whereas each BSD is a whole distribution. Linux distros take their upstreams from thousands of different sources, whereas the BSD folks seem to want to maintain their own copies of everything. And they still seem to think CVS is a good idea for version control, though they do try to offer Subversion as an alternative for the radical young kids. While on the Linux side, look at the number of projects that have moved onto distributed VCSes like Git and Mercurialthe Linux kernel itself never used any centralized VCS.

This decentralized approach, not controlled from any single point, is why the Linux ecosystem has been able to move so much faster than the BSD ones. BSD had a thriving community back when Linux was still a snot-nosed little upstart that could very well have died in childhood; yet look at all the great things it has been able to go on to achieve, while the BSDs havent really moved much at all.

Choosing between portability and innovation

nix — Wed, 09 Mar 2011 23:44:21 +0000

Writing with portability in mind does not imply 'missing functionality' (on platforms capable of it), and *certainly* does not imply 'hundreds of #ifdefs'. Indeed, the latter is generally a symptom of software that was not written with portability in mind, but which had limited portability jammed crudely into it at a later date.

Choosing between portability and innovation

nevyn — Wed, 09 Mar 2011 20:52:48 +0000

My biggest concern with doing something like that would be how efficient it is compared to using the native interface (and why I said you'd need to redesign the kernel, so that it could be implemented efficiently), the only benchmark you have is vs. poll() (and uses ab) ... which is pretty sad.

epoll <=> kqueue is probably the best case test too, to be convincing you'd want something that benchmarked EVFILT_VNODE/SIGNAL/TIMER at least. To be really convincing you'd want PROC/USER/AIO and play with the EV_* flags.

Choosing between portability and innovation

schabi — Wed, 09 Mar 2011 17:50:27 +0000

I agree here.

But note that there is a huge difference between writing "non-portable" code targeted for an idealized platform (with a PAL in place to emulate that platform in the real environment), and writing with portability in mind (hundreds of #ifdefs, missing functionality, etc.).

Choosing between portability and innovation

mheily — Wed, 09 Mar 2011 14:48:48 +0000

Actually, you don't need to redesign your kernel to implement the kqueue API on Linux. The libkqueue project provides a userspace wrapper library that translates each kevent() call into the equivalent epoll/inotify/timerfd/signalfd/etc call for Linux. On Solaris, it uses the event port framework. On Windows, it will use the WaitForMultipleObjects() function.

(Disclaimer: I am the main author of libkqueue)

Choosing between portability and innovation

nix — Wed, 09 Mar 2011 13:49:13 +0000

There are good reasons why PA runs as you and not as a systemwide user (though it can). PA can be asked to load modules providing new features at runtime: this is obviously forbidden for systemwide daemons. PA can operate in a zero-copy mode, transferring audio data directly over shared memory: for obvious security reasons this must be avoided if the daemon may serve more than one user, forcing everything to be serialized and deserialized again.

PA itself is plainly necessary: most modern sound hardware can't mix, so the first open()er blocks all the rest. This is more than slightly aggravating to users (the difference between a very long block and a crash is not very large from a user perspective).

PA is really not that bad. Yes, it was buggy once upon a time (mostly because it used features that had never been used by anyone else), but these days it largely Just Works.

Choosing between portability and innovation

lacos — Tue, 08 Mar 2011 01:10:23 +0000

boilerplate trolling? At least try to be original

Okay, I'll try to be "original" for him.

All this desktop integration goo is getting increasingly difficult to get rid of. There are users who don't need their features, and certainly don't want their bugs, possible inter-app privacy holes, and very probable security vulnerabilities. So while these components are helpful for most people, there's a small group of "power users" who consciously want to remove them, and it is more and more difficult.

The only thing I run on my "interactive systems" (both home desktop and work laptop, different distributions), out of "ConsoleKit, PolicyKit, systemd, HAL, PulseAudio or dbus" is D-Bus; and even that only because I can't remove it. I like to know why my UID runs a process; for PulseAudio, I was unable to find any reason.

I have four unencrypted pendrives, one encrypted pendrive, one flash card reader, one dumb digital camera; obviously an optical drive, and an encrypted hard disk in a USB disk enclosure. Two non-root users can mount different sets of these, with reasonable file permissions and good IO performance afterwards (for a change). I don't buy gadgets each second day, so it's not hard to update my static config. The inability to mount unseen pieces of hardware is a feature.

Choosing between portability and innovation

lacos — Tue, 08 Mar 2011 00:49:12 +0000

people should just forget about the BSDs

You are promoting vendor lock-in. Under this aspect, it does not matter if a given application is free software: unless the end-user has the resources to make the port happen, he's forced to use the kernel/libc you have chosen for him.

Portability of application source code is about adhering to standards that were distilled with meticulous work, considering as many implementations as possible. The answer to divergent implementations is not killing all of them except one, and taking away the choice from the user. The answer is standardization, and a clearly defined set of extensions per implementation, and allowing the user to choose the platform (for whatever reasons) he'll run the application on.

The Unix landscape was splintered and balkanized during most its history. We finally are overcoming that, and that is a good thing

You seem to intend to overcome diversity by becoming a monopoly. The freeness of software (free as in freedom) doesn't matter here, see above.

Choosing between portability and innovation

intgr — Sun, 06 Mar 2011 19:04:07 +0000

> However, that doesn't change the main point I wanted to make, which is
> that it ought to be tracking fds internally instead of files

> I want it to stop watching (and tell me that it did stop watching), if
> the *fd* I originally gave is closed, not if the underlying file is closed

Seriously, if you want it to stop watching, you use EPOLL_CTL_DEL before closing a file descriptor. The implicit deregister-on-close() is just a safety net -- because it's the only sane thing that the kernel can do when you have watches on a file that's being closed. It's not intended to be used that way.

You tried to spin it as a "*HUGE* misdesign", but in practice it's just a trivial edge case that shouldn't affect real applications. Wouldn't be the first time *BSD people criticize parts of Linux they don't actually understand.

Choosing between portability and innovation

pjm — Sun, 06 Mar 2011 12:53:33 +0000

>If you have a single well reviewed implemention I think you are much better off than having 10 badly reviewed ones.

That sounds good in itself. However, your advocated position is to standardize on a kernel that's featureful and consequently full of bugs (despite having so many developers contributing to it), to the complete exclusion of any kernel with fewer bugs. That's not to say that your advocated position is a bad one; but I do believe it runs counter to a goal of combatting computer insecurity.

(Incidentally, if you really did think that 10× the number of bugs is such a bad thing, then I think you'd probably use a different kernel. But most people do use Linux even if they know it has lots of bugs, and would even if they knew it had 10× the number of bugs of some other unixy kernel, even for just one or two extra features important to them. Similarly, each of the N competing implementations presumably has one or two features or attributes not present in (and not feasible to add to) the others.)

and other featureful-but-buggy software .)

Choosing between portability and innovation

jthill — Sun, 06 Mar 2011 06:41:40 +0000

I wouldn't knock epoll, myself. I recognize the design. It seems odd to me that over the last few days I've run across a spate of gripes about epoll in various places, all of them based on misconceptions like the ones in this thread. It isn't just the original complaint (since retracted), it's the remaining gripes too.

There's an unavoidable race between event reporting and any close processing that's most efficiently handled by keeping your own books. It's not so hard to EPOLL_CTL_DEL before you close(), and defer event-tracking cleanup rcu-style until after a nonblocking epoll_wait returns 0, to catch events reported between your decision to close and the actual close.

Note that nothing you can do avoids that race. Having close do the deletion processing will not save you: the event may have already been posted.

IBM don't want their manuals accessed by non-customers so I won't link the description, but epoll is down there in big-O range of mainframe-style event reporting. The main difference I can see is that on the mainframe, the table is in your address space and sized by how many events can get backed up (posted but as-yet unhandled) before something is badly wrong -- for anything but large-scale work you just size it to handle everything anyway.

I notice that nothing in the reporting api constrains epoll to file-based events. Tying any asynchronous event at all to epoll should be possible. Timers come to mind, of course. epoll_pwait handles signals. Me, I think it'd be good to have an epolled_fork, so child termination comes in as just another thing on your events list.

Choosing between portability and innovation

nix — Sun, 06 Mar 2011 04:30:00 +0000

Quite (though it can be used properly, it's definitely in the negative half of that API correctness thing gregkh(?) put up a while back: an API for which the obvious use is wrong).

Thankfully epoll() is not widely used yet, and being Linux-specific all the users can fall back to other mechanisms. (However, the old syscall would have to stay, anyway: we'd need an epoll2() syscall that did things right.)

It's beyond even OS assumptions.

Tet — Sun, 06 Mar 2011 00:34:20 +0000

Precisely. I share my home directory, and mostly it works fine. But the morons at Mozilla don't know how to write software properly, so my Firefox plugins break every time I log in from a different machine. Gimp is slightly better (in that it at least supports the concept of running different versions and multiple instances concurrently), but still problematic.

Choosing between portability and innovation

Tet — Sun, 06 Mar 2011 00:27:57 +0000

This is one case where the pain of breaking compatibility with existing code is probably less than the pain of living with a broken design.

Choosing between portability and innovation

skissane — Sat, 05 Mar 2011 02:00:45 +0000

Personally -- forget about the Linux monoculture -- what about the Unix monoculture, or the LUW (Linux/Unix/Windows) monoculture? People forget that Unix and Windows are relatives -- Windows shows significant influence of Unix. Some examples:

Base OS only supports unstructured bytestream files, rather than base OS support for more complex file structures (such as record structure, indexed files, forked files, etc.) -- this is Unix influence on Windows via CP/M then DOS
Filesystem does not have explicit recording of file types, only convention of file extensions -- again, Unix influence on Windows via CP/M then DOS
Hierarchical directories -- DOS 2.0 explicitly modelled these on Unix
C programming language -- direct Unix influence on DOS/Windows
Berkeley sockets influence on WinSock

If you want a non-monocultural OS, don't look for yet another Un*x or Windows. They are parts of the monoculture. Examples of non-monocultural OSes would be any IBM mainframe/midrange OSes (like z/OS, z/VM, z/VSE, z/TPF, OS/400), HP MPE, Siemens BS2000/OSD, Unisys ClearPath, language-specific OSes like Lisp Machines or PICK (back when PICK was an whole OS not just a database), etc. Of course, many of this stuff is rather ancient and decaying, but it just goes to show how the IT industry was far less monocultural in previous decades than it is today

I don't like hierarchical filesystems. We end up with these massive random directory trees and cannot find anything. Better would be a filesystem where we give files tags, some unique and some non-unique, and we can find files by their tags -- in library science terms, hierarchical filesystems are like Dewey Decimal or Library of Congress classification, I think we should adopt faceted classification instead

I really like the idea of getting rid of operating systems and making applications run directly on bare hardware. Especially with virtualization, who needs a general purpose OS to run a web server or database server? Why not just run the application directly on the hypervisor, with as thin a level possible in between. This is like MIT's exokernel research.

I think Oracle's WebLogic VE is a good implementation of this idea -- run the JVM directly on the hypervisor, with just a very thin custom OS that exists solely to meet the JVM's needs, no general purpose OS needed in between. I'd like to see the same idea extended to other areas (e.g. databases). (Full disclosure: I work for Oracle but this is just my personal opinions not those of my employer)

Choosing between portability and innovation

foom — Fri, 04 Mar 2011 23:08:14 +0000

Okay, I must apologize, you are completely right.

I was getting myself mixed up with kqueue's API (which actually does have you specify a struct containing both the fd and userdata), and then mis-reread the epoll docs. (As you might be able to tell, I always passed the fd as the user data. :))

However, that doesn't change the main point I wanted to make, which is that it ought to be tracking fds internally instead of files -- I don't want it to do anything special with the value it returns, I want it to stop watching (and tell me that it did stop watching), if the *fd* I originally gave is closed, not if the underlying file is closed. Returning events on a file that I no longer have a handle for in the current process is annoying.

Choosing between portability and innovation

nevyn — Fri, 04 Mar 2011 22:11:36 +0000

> No, it really wasn't misleading. The userspace-visible key *is* the fd.
> You're required to pass that in, that's what epoll uses to look up the
> file object, it will always pass that back

Yes, you need an fd to register/change the epoll events. But it isn't ever passed back:

epoll_wait() returns a list of "struct epoll_event", which is defined as:

struct epoll_event
{
uint32_t events; /* Epoll events */
epoll_data_t data; /* User data variable */
} __attribute__ ((__packed__));

...epoll_data_t is the union I talked about before, so you can use a pointer or a number for your callback data. You don't get the fd back from the API, unless you use that as your callback data but if you do that it's just "a number" ... so I wouldn't expect the kernel to do anything special with it.

Choosing between portability and innovation

foom — Fri, 04 Mar 2011 21:45:23 +0000

No, it really wasn't misleading. The userspace-visible key *is* the fd. You're required to pass that in, that's what epoll uses to look up the file object, it will always pass that back, and that's what you have to pass to epoll_ctl for further modifications/deletions of the watch. You can of course *in addition* register arbitrary user data that it'll pass back to you. But that's optional. The fd is required.

Segue into another rant of mine....that's yet another reason why the auto-removal-upon-file-close functionality is broken. There's no way to get a notification that epoll has autoremoved an entry from the epoll set, so, it's not actually possible for you to ever *free* the data (and other resources) you allocated and passed into epoll_ctl. Sigh. (same bug exists with kqueue API btw).

Choosing between portability and innovation

nevyn — Fri, 04 Mar 2011 21:04:11 +0000

> And epoll certainly has a *HUGE* misdesign in it [...]
> epoll_ctl lets you register a file descriptor for monitoring. However,
> it *actually* registers the underlying file description for monitoring.
> But then it remembers in kernel-land the file descriptor number you passed
> in and reports all events for the file descriptor with that file
> descriptor number you passed in originally.
> No matter what you do to the file descriptor afterwards.

That's a pretty misleading description. You register for events and give the kernel a pointer or a number as your callback data ... you then get that piece of data and a set of flags, when something happens.

If you pass in your "original fd number" as the data you want back, that's what you get. Personally, when I've used it, I used a pointer to a struct as the callback data.

Choosing between portability and innovation

nevyn — Fri, 04 Mar 2011 20:59:27 +0000

kqueue is a _huge_ interface, combining a number of ideas, you basically need to design your kernel around it if you want users to use it. And much like SYSV streams, expecting other people to reimplement a giant redesign like that is often expecting _way_ too much.

On the other side, timerfd(), socketfd() and epoll() are all there to do a single thing. FreeBSD did the retarded thing when they reimplemented sendfile() roughly a week after Linux added it. TCP_CORK is still not implemented in FreeBSD. mremap() was NAKd, there's the whole MMAP_ANON vs. MMAP_ANONYMOUS, or the weird bits of mmap in general.

Cooperation is much more like to happen in the FreeBSD => Linux direction, IMO ... but that could also just be the much bigger developer pool.

Choosing between portability and innovation

dlang — Fri, 04 Mar 2011 17:44:31 +0000

I almost agree with you.

the thing is that the API you code to should be flexible and evolve over time, with some of the abstraction layers that others have mentioned being in the API -> platform level

In the end your API may not directly match any of your destination platforms.

the linux kernel does this. Linus has said in interviews that the kernel is not written directly for any type of system, but instead is written for an idealized system that doesn't actually exist, with 'shim code' on every single platform (including x86) to simulate portions of the 'system' the kernel is written for. He says that the kernel is much better as a result of this work.

Choosing between portability and innovation

jeremiah — Fri, 04 Mar 2011 16:45:58 +0000

LOL..amen. Gotta love the high black arts of Linux network administration. Maybe it's time for O'reilly and Olaf to come out with a revised version of the Linux Network administrator's guide. My version is from 1995.

Choosing between portability and innovation

trasz — Fri, 04 Mar 2011 14:37:36 +0000

Kqueue vs epoll is a pretty good example of lack of willingness to coooperate - kqueue was earlier, yet Linux folks chose to implement something incompatible and functionally inferior.

Choosing between portability and innovation

nix — Fri, 04 Mar 2011 12:26:38 +0000

Agreed. It's not like sysvinit ran on non-Linux systems either: they all have their own inits.

What *would* be nice is, when systemd calms down a bit and we start to see which of its features are used by other programs, if some other inits could start to gain those features. (I've never heard of another init-like program which could even be described as having features that other programs could use. systemd is a big step forward in that way.)