| |||||||||||||
EPOLL_CTL_DISABLE, epoll, and API designEPOLL_CTL_DISABLE, epoll, and API designPosted Oct 23, 2012 20:27 UTC (Tue) by Yorick (subscriber, #19241)Parent article: EPOLL_CTL_DISABLE, epoll, and API design
This is unfortunate. There is nothing more important to review than immutable interfaces (except for security audits). I'm going to be grossly unfair here, but in neglecting basic software engineering principles, the developers here come out as bumbling amateurs. (Of course I've made similar mistakes myself, but with slightly less severe consequences.) With a cast-in-stone policy, these APIs must be subject to extreme scrutiny. I would like to go further than the editor of the excellent article: There should be working applications present, not just tiny test cases or proofs of concept, in addition to full documentation, before any proposals can be accepted. It's not just a matter of verifying that the APIs work, but that they are useful and complete as well. The importance of checking unused parameter bits was learned dearly in the 1960s, over and over again, both for hardware and software. We should know this by know.
(Log in to post comments)
EPOLL_CTL_DISABLE, epoll, and API design Posted Oct 23, 2012 20:30 UTC (Tue) by dlang (✭ supporter ✭, #313) [Link]
even working applications aren't going to identify all the drawbacks and problems with an API.
The early applications that use an API are going to be written by people who are very familiar with the API and know not only what the API does, but how it _should_ be used.
a few years later, you get applications developed by people who don't know how it _should_ be used, and as they start trying to use it in other ways (some of them very good ways), they expose limitations that the people who were involved with the development, testing, and reviews of the API missed.
EPOLL_CTL_DISABLE, epoll, and API design Posted Oct 23, 2012 21:01 UTC (Tue) by Yorick (subscriber, #19241) [Link]
You are right, but there is not much we can do about that other than devising more sophisticated mechanisms for API evolution than the rather simplistic policy in Linux. User-space libraries (with versioned symbols or similar ways of evolving APIs) provide one possible answer, but from what I understand, the kernel maintainers aren't too keen on such layers.
EPOLL_CTL_DISABLE, epoll, and API design Posted Oct 23, 2012 21:07 UTC (Tue) by dlang (✭ supporter ✭, #313) [Link]
I disagree.
If applications will be broken, then the API shouldn't change.
If there are no applications using an API, that API can be removed.
But If applications are using that API, creating new versions of the API doesn't solve the problem. Existing applications are not going to disappear.
This attitude that "the APIs are versioned, we can drop support for old versions" is exactly what's causing the problems in the Linux Desktop Environment world.
It doesn't matter how justified you think you are in getting rid of an old version, if it breaks users it's a regression and you should not do so.
Maintaining lots of different, incompatible versions of an API is a huge amount of work, so just versioning the API isn't nearly enough, and it's questionable if it really helps in the long run.
EPOLL_CTL_DISABLE, epoll, and API design Posted Oct 23, 2012 23:04 UTC (Tue) by nybble41 (subscriber, #55106) [Link]
In cases like this one, where the problem is just unused parameters which weren't checked, it seems like a simple enough thing to fix. Just create a new API for new applications that follow the rules, and make the old one forward to the new one after masking out the unused parts. For example, epoll_ctl(epfd, op, fd, event) could call epoll_ctl_v2(epfd, op, fd, NULL), ignoring event; anything that wants to use the last parameter would call epoll_ctl_v2 directly. The same principle would work for bitfields. The new API could easily be made automatic for new programs through versioned library symbols.
However, I agree that in general maintaining multiple API versions, where the old one is not a simple subset of the new one, is not likely to go well.
EPOLL_CTL_DISABLE, epoll, and API design Posted Oct 25, 2012 9:42 UTC (Thu) by dgm (subscriber, #49227) [Link]
> However, I agree that in general maintaining multiple API versions, where the old one is not a simple subset of the new one, is not likely to go well.
For that reason we should be putting much more effort into API design. In fact, tacking into account that API _will_ outlive its implementation (maybe several of them), most of the effort should be towards getting the API right.
Re API (and ABI) design and maintenance Posted Oct 26, 2012 18:31 UTC (Fri) by davecb (subscriber, #1574) [Link]
Creating new versions is one *step* in a disciplined (anal) process of evolution. It's been done continuously since the days of the mainframe, but people don't realize it's happening.
I spent three years at the (late, lamented) Sun Microsystems doing ABI stability, and ended up sensitized enough that I notice it when it happens.
An easy example was a company's linker, which needed a different data structure for a method call that it had. We added a new "record type" in tghe linker, and converted the compilers to produce only it. We supported the old format for a year, then made its use produce a warning, and a year after that, made it require a link-line option.
We got two complaints, total, about the option. Out of all our customers, only two were so very very far behind that they ran one of the old compilers. A year or two later a hardware change made those compilers produce impossibly lousy code, and the two outliers upgraded to the new compilers. Then we retired the old interface.
If we had moved any faster, we would have annoyed at least two customers. If we had moved any slower in switching the compilers over, we would have made the OS developers unhappy. The time between the first switch and the final stages of the retirement made the maintainers unhappy, but there weren't many bugs in that interface nor were there many user of it, so the time cost of maintaining it was low.
We did have to manage it, and we had to do a fair bit of work behind the scenes to make it invisible to the users, but we succeeded at evolution.
Just as with humans, if you don't evolve, you might just die out. Homo habilis, anyone?
--dave
Re API (and ABI) design and maintenance Posted Nov 13, 2012 12:39 UTC (Tue) by k3ninho (subscriber, #50375) [Link]
> We did have to manage it, and we had to do a fair bit of work behind the scenes to make it invisible to the users, but we succeeded at evolution.
I suspect that Linus' view on never, ever, binning old binary interfaces will mean that there will be (un)dead interfaces supported forever. My thoughts on a management plan look like:
K3n.
Re API (and ABI) design and maintenance Posted Nov 15, 2012 17:15 UTC (Thu) by nix (subscriber, #2304) [Link]
That's pointless. With memory sizes continuing to shoot up as they are, the memory overhead of maintaining old interfaces is minimal: and as long as the interfaces have no significant maintenance overhead, why not maintain them? I mean, Linux still supports uselib(), which is IIRC useless unless you're still using a.out shared libraries and libc4!
Re API (and ABI) design and maintenance Posted Nov 19, 2012 6:44 UTC (Mon) by k3ninho (subscriber, #50375) [Link]
This article itself supplies an example where an older interface design has caused internal plumbing designs to be less than the best. This will happen again, and will cause a kind-of scarring to the structure of the Linux Kernel. Wouldn't you like to be able to offer end-users of the Kernel a clean set of interfaces, easy to comprehend and unhindered by history's quirks?
K3n.
EPOLL_CTL_DISABLE, epoll, and API design Posted Oct 24, 2012 4:25 UTC (Wed) by daniels (subscriber, #16193) [Link] the developers here come out as bumbling amateurs Err? They made a mistake, which in the context of something as complex and genuinely impressive as the Linux kernel, can be forgiven. How many kernels which scale from embedded to clusters have you written?
EPOLL_CTL_DISABLE, epoll, and API design Posted Oct 24, 2012 10:40 UTC (Wed) by Yorick (subscriber, #19241) [Link]
As the article points out, changes to the internal kernel code tend to get much more scrutiny than those that affect user-visible interfaces, despite the fact that those are the ones that really need to be right from the start. That comes out as sloppy, but is really more a sign of a development process in the need of improvement.
When we suffer from badly thought-out APIs made by Microsoft, say, we pour scorn over the developers who designed the mess and call them incompetent, fairly or not. Linux kernel programmers are not exempt and should not be.
EPOLL_CTL_DISABLE, epoll, and API design Posted Oct 24, 2012 10:51 UTC (Wed) by mkerrisk (editor, #1978) [Link] That comes out as sloppy, but is really more a sign of a development process in the need of improvement. Yes. Having watched what goes on for quite a while now, I consider this mainly a process problem, rather than a problem of individual developers (though obviously some do a better job than others).
EPOLL_CTL_DISABLE, epoll, and API design Posted Oct 25, 2012 9:45 UTC (Thu) by dgm (subscriber, #49227) [Link]
A good start would be to find out how is that some do a better job that others, and how we can get that to happen more often.
|
|||||||||||||