| |||||||||||||
How we learn APIsHow we learn APIsPosted Mar 17, 2009 17:34 UTC (Tue) by christian.convey (guest, #39159)Parent article: Better than POSIX?
I think this goes to one weakness we have regarding writing robust code: most of us learn APIs by experimentation and seeing how a program performs when we've written it in a certain way. This issue with the POSIX file APIs plays right into our weaknesses as developers, because we easily draw the wrong conclusions about our correct use of the API when our programs appear to work.
Unless we can get programmers to learn APIs differently, I don't see how we can avoid providing them with a more helpful API if we want them to write robust programs.
Maybe what we really need is some user-space library that provides the more robust guarantees, but that works across many different file systems and operating systems. That could get us past the concerns about writing ReiserFS-only code or Linux-only code.
(Log in to post comments)
How we learn APIs Posted Mar 17, 2009 18:19 UTC (Tue) by droundy (subscriber, #4559) [Link] Unless we can get programmers to learn APIs differently, I don't see how we can avoid providing them with a more helpful API if we want them to write robust programs.I've been wondering whether we might be able to quantitatively describe a "robust" implementation of the POSIX API in terms of the published API in a way that is relatively easy to convey to its users. For instance, I'd like to know that if my application is written so as to behave correctly (as understood by myself and my users) in any scenario in which any subset of processes are abruptly killed, and the connection to the disk is severed at any time--assuming that the disk is always in a "correct" state at the time it is severed, in the sense of reflecting all IO preceding that point, and that all IO after that point has no effect on the disk, and fsync fails after that point--then my application is also robust in the event of a system crash. This is a relatively simple criterion of *application* robustness, which then could define *file system* robustness. As an application developer, that's what I'd like to have assurance of. In the extremely common case that we don't care if data has actually hit disk (i.e. our application will work fine on a RAM disk), the latter scenario of severed disks is irrelevant, and we just want to know that the guarantees that POSIX provides with regard to the running system are preserved in case of crash. And that's something I can reason about. And it's even something I can test my application for without installing special non-robust file systems and pulling the plug on the computer.
How we learn APIs Posted Mar 19, 2009 17:35 UTC (Thu) by anton (guest, #25547) [Link] What you describe sounds very simular to what in-order semantics provides.
How we learn APIs Posted Mar 20, 2009 13:53 UTC (Fri) by droundy (subscriber, #4559) [Link]
Yes, that exactly describes the semantics I'd like--and the page even gives the same reasoning for it. It seems so obvious that this should be the goal of a file system!
(Although I can see an appeal to relaxing the in-order constraint for IO from different processes... one ought to be able in principle to do that while maintaining in-order semantics if one were to examine locks---and information flow in general---to ensure that you didn't reorder IO that could be causally related.)
How we learn APIs Posted Mar 20, 2009 18:36 UTC (Fri) by anton (guest, #25547) [Link] Although I can see an appeal to relaxing the in-order constraint for IO from different processesWell, given that Unix applications often create lots of processes that interact in lots of interesting ways (think shell scripts), I think it's just too hard to find out when two operations are independent. I also don't think that this relaxation buys much if the in-order constraint is implemented efficiently (by combining a large batch of operations and commiting them together).
How we learn APIs Posted Mar 17, 2009 20:04 UTC (Tue) by nix (subscriber, #2304) [Link]
The only programming community I know that actually works as you suggest
is the Ada community, and that's because their systems tend to be 'system fails, people die' sorts of embedded systems (or 'system fails, N-million- dollar missile falls out of the sky'). That tends to breed paranoia and a desire to actually know the damn standard before you write code.
Everywhere else, it's a minority who even know the standard *exists*, let
I, too, wish this was not true (I spend much too much of my time at work
How we learn APIs Posted Mar 17, 2009 21:44 UTC (Tue) by man_ls (subscriber, #15091) [Link] And there is a reason that critical code can be as much as 10 times as expensive as regular business code. Working that way (learning your standard by heart before starting to work) can be much more expensive than the usual corporate style of "write, then fix". On the other hand it is better to write correct code than to "write, then fix". There is a sweet spot somewhere in between.
Learning APIs by writing unit tests? Posted Mar 18, 2009 11:25 UTC (Wed) by emk (guest, #1128) [Link]
Over the years, I've become a bit cynical about standards and documentation. When I rely on a standard, I find that nobody implements it correctly. (Witness, oh, every C++ compiler from 1992 to 2002.) When I rely on documentation, I find that it's a pack of useless lies. (The worst offenders: Hardware databooks, which often have no bearing on reality whatsoever.)
So now I take a different approach: I write lots of unit tests, and I set up buildbots for any platform I need to support.
This leads to lots of interesting discoveries. For example, under Windows Vista, pretty much any file system operation can fail for no apparent reason, and it may need to be retried until it works.
There are two exceptions to this rule: Security, and data integrity. You can't ensure either with unit tests. You must also understand both the official documentation and the reality of common implementations. (My nastiest surprise so far: There are snprintf implementations on some legacy Unix platforms that ignore the buffer size parameter, exposing every caller of snprintf to an overflow attack. If you rely on snprintf on old Unix platforms, it might be worth writing a unit test that checks to see if it works.)
|
|||||||||||||