| |||||||||||||
Crowding out OpenBSDCrowding out OpenBSDPosted Nov 13, 2012 23:41 UTC (Tue) by vonbrand (subscriber, #4458)In reply to: Crowding out OpenBSD by rleigh Parent article: Crowding out OpenBSD
Sorry, but the effort to run, e.g. systemd, over a non-Linux system is definitely not "a little." Writing portable code is very hard. Keeping it portable is a lot of of extra work, requiring many able bodies.
(Log in to post comments)
Crowding out OpenBSD Posted Nov 14, 2012 1:16 UTC (Wed) by mikemol (subscriber, #83507) [Link]
Are there decent 'lint' or 'valgrind'-style tools for warning about use of nonportable APIs (or APIs without universal behavior)? I could imagine such a thing being immensely useful for those of us who write software and care.
A 'lint'-style tool would obviously be a static analysis thing. A 'valgrind'-style might be able to spot uses of various calls without POSIXLY_CORRECT defined.
Of course, not all warnings are meaningful for a program. Not all platform targets are meaningful for a program. It'd be a matter of defining the testing scope, as with anything else.
Crowding out OpenBSD Posted Nov 14, 2012 1:34 UTC (Wed) by mpiechotka (subscriber, #83514) [Link]
The problem is not in tools or lack of them. Usually it is that you need to write different code to support different APIs. Think kqueue vs. epool vs. ... - they all sort of solve the same problem but might have different edge cases etc. And this is relatively easy part - if you try to manage the USB driver you need to talk directly to OS. In the end you have a massive code which can be inflexible because you haven't thought about multi-seat at the beginning and you need to add it now. Oh and you need to maintain so better prepare to have multiple VMs to test it on different setups.
As of testing - it might detect bugs but usually a) they are relatively simple bugs not big, large bugs like "you cannot do X on system Y" b) they test one code path and the bug might be in another (I'm not saying that testing is bad but that it is supportive to portable design not a full verification).
Crowding out OpenBSD Posted Nov 14, 2012 2:07 UTC (Wed) by mikemol (subscriber, #83507) [Link]
I wasn't looking for full verification, but rather tools to point at likely problems. Similar to how a compiler warns me that some code I wrote may not do what I intended...I'd love to have an API call audit warn me that I'm calling an API not present in all my supported platform targets. Or that I'm calling them with parameters that aren't properly supported.
I wish I had time to respond to all the things you mention. Some of them are going to be true of any software project (Paradigm shifts forcing architectural changes), some are greatly mitigated by avoiding unportable APIs unless there is a strong benefit to using them, and some are mitigated by keeping code modular and unportable portions behind abstractions. But I've got to go catch a bus...
Crowding out OpenBSD Posted Nov 14, 2012 8:17 UTC (Wed) by mpiechotka (subscriber, #83514) [Link]
No, they don't apply to all programs - I would say they don't apply to at least 99% of programs. There is no point in say graphical calculator to call any unportable API. However graphical calculator-like applications is not the only class of programs.
Modern DE usually at least try to manage devices, do power management, session management etc. - those are the hard problems. As of writing portable abstraction layer - it is usually good move if many components use it. However if you are the only user you moved a problem a bit, make architecture cleaner but haven't solve the ultimate problem - someone have to maintain the portable layer and move it forward on all platforms in sych. That means that say users are limited only to support of things that are on all platforms (and if say NetBSD have some wonderful feature the DE developed by NetBSD people couldn't use it).
I'm not saying that it is impossible but that it is hard (even if you do know something is not portable you don't necessary can solve it in simple way - I run at least once in problem when on system X you had do A while on system Y you mustn't do it). And given that by definition projects are understaffed I can imagine that portability might slip as a goal.
Crowding out OpenBSD Posted Nov 14, 2012 13:57 UTC (Wed) by mikemol (subscriber, #83507) [Link]
So we agree that for 99% of programs, the tool I'm asking for makes sense? I'm OK with that.
I disagree that you need to move the portability layer in sync. It makes pragmatic sense to have first-class and second-class platforms. First-class platforms can be in sync, second-class platforms can be a bit behind. As a user of a second-class platform[1], I understand this can be a pain, but I've also seen how it ultimately ends up work out.
At least with the portability layer, you've limited the number of lines of code that a porter needs to focus on.
[1] Gentoo. I'm sure the various BSD groups have it worse, but try running a pure-stable Gentoo. You pretty much can't. And there's no official stance on whether running a mixed stable/unstable setup is supported; ask two developers, you'll get three answers.
Crowding out OpenBSD Posted Nov 15, 2012 14:27 UTC (Thu) by khim (subscriber, #9252) [Link] So we agree that for 99% of programs, the tool I'm asking for makes sense? I think you both are mixing issues. My own experience is radically different: for 99% of programs full portability is a pipe dream (think getopt: pretty popular tool on *nix, but problematic if you need to support Windows, too - you either need to provide your own implementation bundled with your program or use some third-party package, etc). What is possible is to write, say, 90% of program in a fully-portable way, cover the remaining 10% with portability layer and keep 1% as a system-specific code surrounded by ifdefs. Sadly even if for typical program just 1-2% code must be system-specific these pieces impose insane amount of maintenance work - significantly more then 1-2%. And then the obvious answer is to use OpenSSH approach (born by OpenBSD, ironically enough): just cover your primary platform and don't think about portability. If someone wants to have portable version - they can do it as a patches on top of clean-yet-unportable code.
Crowding out OpenBSD Posted Nov 15, 2012 15:04 UTC (Thu) by mikemol (subscriber, #83507) [Link]
I wouldn't dream of "full portability". To me, that includes Linux, Windows, various BSDs, FreeDOS and places with 12-bit bytes. Even outside of practical considerations, to me "full portability" is a phrase that represents a paradox.
The way I see it, you have a tester program with a database of things to check against, and you give it an operational mode for the classes you care about. I.e. "warn me about anything you know of that's incompatible between Xorg and XFree86. Give me an error for anything that's incompatible between FreeBSD, OpenBSD and GNU." or "Give me an error for anything incompatible with unix-like systems [where that would mean common Linux environments, FreeBSD, OpenBSD, QNX...]"
I'm not suggesting the programmer should be demanded to provide support for all known platforms. I'm suggesting a programmer should be able to decide what platforms he'd like to support, get an advance rough idea at the parts of his program that are incompatible with that set, cull his supported platform set down to something manageable, and manage that.
The tool I'm describing would also be useful for people doing the active porting work, since it can immediately draw their attention to the places which need work.
Crowding out OpenBSD Posted Nov 17, 2012 22:15 UTC (Sat) by cmccabe (guest, #60281) [Link]
Someone should create a language that you can "write once, run anywhere." Radical to the max, dude!
That was a 90s reference, for you youngsters...
Crowding out OpenBSD Posted Nov 19, 2012 9:03 UTC (Mon) by ortalo (subscriber, #4654) [Link]
Hmmm, strictly speaking, it seems that this should even be a reference to the 70s and... C no?
BTW, note that C is the primary unifying factor between Linux and the BSDs in fact... and the main difference with the dominant proprietary OSes.
Crowding out OpenBSD Posted Nov 20, 2012 9:05 UTC (Tue) by cmccabe (guest, #60281) [Link]
One of Java's slogans was "write once, run everywhere."
I guess the implication was that C programs always required tweaking to run on platforms other than the ones they had been written on, and Java programs would not. Of course it didn't quite turn out that way in practice-- for example, Hadoop is mostly written in Java, and the process of porting it to Windows is still ongoing.
It's hard to find a free lunch in computing. Even harder to find a type of free lunch that someone else hasn't already spent years looking for.
Crowding out OpenBSD Posted Nov 20, 2012 11:52 UTC (Tue) by Cyberax (✭ supporter ✭, #52523) [Link]
Uhm. I've been running Hadoop on Windows for _years_. The main stumbling block is setting up an SSH daemon, but even without it you can use Hadoop for certain limited tasks.
Crowding out OpenBSD Posted Nov 21, 2012 21:33 UTC (Wed) by cmccabe (guest, #60281) [Link]
Yes, there's a branch of hadoop called branch1-win that has been modified to work well on Windows. There's a JIRA tracking its progress here: https://issues.apache.org/jira/browse/HADOOP-8645.
Crowding out OpenBSD Posted Nov 20, 2012 12:11 UTC (Tue) by man_ls (subscriber, #15091) [Link] The idea was actually that a single "binary" (bytecode actually) could run, unmodified, on several platforms. Yes, "write" here is grossly misused.Given that the bytecode was in fact interpreted by the VM (and later compiled on the fly), the comparison to compiled languages was misdirected: Java had the inconvenience of having to compile and the slowness of interpreted languages, but none of the advantages. It would have been more appropriate to compare with interpreted languages such as Python or Perl, but then Java would have lost big time since VMs were only available for a handful of platforms and the competition can run, unmodified, on tens. Here by "platform" I mean a combination of OS and processor architecture. Nowadays Java is JITted and fast. But then again so is the humble javascript...
Crowding out OpenBSD Posted Nov 14, 2012 15:41 UTC (Wed) by cruff (subscriber, #7201) [Link] Writing portable code is very hard. Keeping it portable is a lot of of extra work, requiring many able bodies.I have to say that writing portable code is not really that hard, as long as you have a well defined common base, such as POSIX, to target. It just takes discipline and proper abstraction of the features you need that are not part of the base. I've written multithreaded C++ software for a variety of POSIX compliant OSes that only required some straight forward GNU make makefiles and a few OS dependent source files to fill in the missing or OS dependent parts. Didn't even need the mess of autoconf, etc. Once you stray far from the common base chasing the neat new features, whether for the wow factor or because of need, then it can be hard, but need not be complex if structured correctly.
Crowding out OpenBSD Posted Nov 14, 2012 18:28 UTC (Wed) by jwakely (subscriber, #60262) [Link]
> a well defined common base, such as POSIX
Which release? I'm not sure all the BSDs are down with some of POSIX-2008 yet (though I have only looked at recent-but-not-current releases)
> only required some straight forward GNU make makefiles
Later in the thread Espie complained [1] that GCC requires GNU make, apparently even that requirement is too much ;)
But then he also complained that GCC major releases are a few years apart (which was true a few years ago, but not now) and that changes only go in the trunk not release branches (so what? push your changes upstream and maintain patches in the ports collection against old releases, then drop the patches when a new release comes out that already has the changes ... surely that's obvious?) The implication in [1] that GCC only builds on Linux is ridiculous, plenty of other OSs manage to keep GCC trunk building without herculean effort, if OpenBSD can't it's not GCC's fault. I've made sure NetBSD does myself, not because I ever use it, but just because I had access to a box.
Crowding out OpenBSD Posted Nov 14, 2012 18:49 UTC (Wed) by jwakely (subscriber, #60262) [Link]
Sorry, wrong link, [1] should be http://article.gmane.org/gmane.os.openbsd.tech/30747
Crowding out OpenBSD Posted Nov 14, 2012 23:54 UTC (Wed) by wahern (subscriber, #37304) [Link]
It doesn't matter whether every Unix supports all of POSIX. What matters is that they everybody aims for common or similar abstractions, and that people test, test, test. "Portability" is a process.
I wrote a C library, wrapped in Lua, which does O(1) polling, async file notification, async DNS, async SSL, wrapped almost all of OpenSSL's X.509 key bindings, async descriptor passing, and async thread management in just about one week's worth of man power. The only dependency is OpenSSL.
Not only that, but I wrote it and maintain it for Linux, OS X, Solaris, FreeBSD, NetBSD, and OpenBSD.
http://25thandclement.com/~william/projects/cqueues.html
I learned one very important thing doing that: the BSDs generally have the better abstractions (particularly in kqueue), though Linux has more comprehensive POSIX API support. But in any event, it simply wasn't that difficult to write all of that portably, and in a small amount of code. And with VMs, it's been trivial to maintain the build for all the present releases of those systems.
The reason why people have a problem with portability is because even the best programmers fall into the habit of cargo cult programming. They're new to some API and scour usenet (or more likely these days, stackoverflow.com) looking for descriptions and usage examples. Then they copy the examples and move on.
But if you take the time to study the APIs, to study POSIX, and to study the implementation code (thank you, open source), then portability just isn't that big of deal. And the end result is that your program is more robust, because not only are more code paths tested, you actually understand what the heck is happening, as well as understand how and in what direction the APIs will evolve in the future.
It's important to constantly test on different platforms, though, and not to let portability issues fester and metastasize. But that's, sadly, what is happening with most open source software these days. People not only just don't care, they openly reject the idea portability. And that's a damn shame.
As for GNU Make, writing portable make is easy, but I allowed myself that single convenience, mostly because GNU Make is readily available everywhere. The BSD makes are almost as powerful, and I've even begun writing a make parser+compiler to translate between them. Mostly because GNU Make is darned ugly, and some of the BSD make syntax features are easier to understand while remaining just as powerful; so I want to write in a subset of of my favorite features from the different makes and then translate to each accordingly.
Crowding out OpenBSD Posted Nov 14, 2012 23:56 UTC (Wed) by wahern (subscriber, #37304) [Link]
And FWIW, signal notification. Although, Solaris doesn't have a pollable signal facility.
Crowding out OpenBSD Posted Nov 15, 2012 2:52 UTC (Thu) by nix (subscriber, #2304) [Link] Later in the thread Espie complained [1] that GCC requires GNU make, apparently even that requirement is too much ;)Well, that's politics, isn't it. GNU Make is GPLed, therefore bad (even though it doesn't contaminate projects built with it), therefore all projects should work with BSD make as well. (That the program being compiled is both GPLed and a flagship GNU project itself apparently does not affect this policy in its case, though common sense says it probably should.)
Crowding out OpenBSD Posted Nov 15, 2012 14:57 UTC (Thu) by jwakely (subscriber, #60262) [Link]
Indeed it is politics, and common sense should indeed let the GNU compiler use GNU make.
My point is that espie@ seems to want to have his cake and eat it. If you refuse to work with upstream (for technical or political reasons) then don't be surprised if upstream forgets about you.
On the plus side, as he says he is happy to leave kettenis@ to deal with GCC, and he recently pushed a load of patches from OpenBSD to GCC, which were gladly accepted (although I did forget to apply one until reminded, but I'd hardly call that "a fight".)
|
|||||||||||||