LWN: Comments on "A tempest in a tty pot"

Why not fix the tty interface?

yuhong — Wed, 12 Aug 2009 18:55:21 +0000

Well, the good news is that Plan 9 gets rid of this tty mess altogether.

Please

obi — Wed, 12 Aug 2009 04:25:09 +0000

I wish I could highlight your post; it cheers me up knowing at least some people understand this point. ;-)

Then there are no point and case is closed, right?

obi — Wed, 12 Aug 2009 04:12:17 +0000

My first reaction would have been to install Debian's 2.6.30 kernel backport - that way I get all the bugfixes and security updates too.

A tempest in a tty pot

jonshouse — Tue, 11 Aug 2009 15:24:54 +0000

Almost all of the real world problems here could be solved if Clib/libc had more real world user functions. What about something like

if (stdio_raw(1)==0)
{
// I am in raw mode
}
else
{
//something went wrong
}

The c runtime can even track if raw is on or off, or simply reset to non raw on crashes.......

Its piss poor API for userland applications not the kernel that makes *nix a bit of a suck to code.

out of tree

vadim — Mon, 10 Aug 2009 16:43:35 +0000

Let me give you my opinion as an user: As far as I'm concerned, if your hardware doesn't have a driver in the mainline kernel, it doesn't have one. And that means I don't buy it. And that's not a huge hindrance, since Linux has modules for everything these days. A device that only has an out of tree module is a rare exception and easily avoided.

Having the driver in-tree actually increases the burden to the manufacturers because now they can't control what their customers are getting.

But I'm not that interested in you controlling what I'm getting. If I have a 3Com card, I need it to work. Who makes it work isn't particularly important. If it stops working, I don't go to 3com.com, I expect my distro to ship a fix. It's actually something I love about Linux - on Windows every driver comes with random crap attached. Webcams can't just be webcams, they have to ship some horrible interface as well, without which it's impossible to have full control of the device. I much prefer the Linux model: The webcam is an image capture device, nothing more. Every webcam works identically via a common API. If I remove one and plug in another, capabilities of the hardware aside, nothing changes. That's precisely how I like it.

Then there are no point and case is closed, right?

khim — Sun, 09 Aug 2009 11:46:20 +0000

Well, I am talking about a source level API, so I don't see your point really.

What makes API so much different from ABI? In my experience breakage occurs not at format ABI/API level, but at level of semantic: old driver presume that something is done in some specific way and it does not work in exactly this way after small fix in kernel (I've personally seed cases where driver stopped working with NP4SP5=>NT4SP6 and XPSP1=>XPSP2 upgrades) - at that point the only solution is some person who can maintain driver... and if you have such person why do you need stable ABI or API?

Plus, I disagree with your example.

How can you disagree with truth?

The vast majority of drivers did work between Windows versions.

The wast majority of drivers work with Linux out-of-the-box. Speaking as a Linux user.

I was asked to troubleshoot a Debian Lenny image on a new batch of POS machines. Well, guess what, Lenny's kernel 2.6.26 does not support the network adapter (Intel 82567LM-3).

How is it different from my example above - where Windows XP SP2 broke drivers for Pinnacle DV500?

So, WTF am I supposed to do?

You know the answer - you just refuse to resign yourself to it.

Should I perhaps install a newer kernel version?

Yes. Either that or ask your vendor for older motherboard.

What about security updates?

What about them? Latest version of kernel always have the latest version of security updates applied - it's up to your distribution to package them.

Remember that bug in driver can make your system volnerable (case to the point - and conveniently it's about e1000 as well) so there are only two ways to live:
1. Use drivers from your distribution kernel (and forgo latest and greatest hardware), or
2. Use latest and greatest version of kernel (and live with possible bugs).

Both approaches don't need any stable API or ABI...

P.S. Infromation from Windows campus agrees with me: substantial percentage of problems with the (in)famous Windows (un)stability are caused by drivers.

out of tree

Los__D — Fri, 07 Aug 2009 22:31:57 +0000

This is not much different on Windows.

Stupid devices which needs stupid programs to do what a perfectly capabable standard Windows configuration tool should be doing seems to be more the rule that the exception.

- The worst part is that Windows seems to support that ridiculous way of doing it. i.e. the Wi-Fi configuration tool has an option to transfer control to a 3rd party program.

(To be fair, this seems to be getting a bit less frequent lately).

GNU + Linux = broken development model

poison — Fri, 07 Aug 2009 18:57:12 +0000

Just a few random unfiltered thoughts that popped up while reading through
this thread. Please forgive me^^
* What do you mean no stable API? I will spend WAY more time adapting to
API changes compared to implementing the features myself and weeding out
most bugs? WHY ON EARTH SHOULD I USE IT?
* Innovation through rewriting and completely breaking APIs? Ok, I've done
rewrites. They involved MAJOR changes to requirements. What was your
reason? The new API was neater? Was the performance improvement anywhere
near measurable and not achievable without breaking the API so badly? Was
the design so bad you couldn't implement the features on top? Oh wait,
Linux folks have some strange analogies with software design/biology and
religious nuts. I don't get the comparison but apparently it's a religious
stance that thinking about requirements and laying out an API that could
sustain a a year or two is a bad idea compared to writing some fugly code
that nobody would touch and then rewriting it every few months. Who cares,
it works (somehow).
* ... Abstraction? Whats that? Oooh, right ... forget about it.
* Out of tree is easier? Hows that? In case refactoring is REQUIRED,
wouldn't it be easier if everything is in place?
* OOOh ... out of treeeeee! The things that disappear into the mist once
you upgrade to a new kernel version. What? Recompiling won't help? The API
is gone? Not maintained? Oh well ... I'll just buy a new card.
* Windows? Like on a submarine? As soon as you open one the trouble
starts? The blue thingy thats so stable you have to reboot your server
every week? (Of course except you do nothing at all with it except letting
it sit there and brag about how stable it is.) At least it has a stable
ABI. Oh to hell with a stable ABI, I just want a stable API!
* Companies MAINTAINING ...? Sure that'll help sales. Maintaining drivers
for 10 year old hardware! Lots of profits!
* Standard Interfaces? The ones that get replaced all the time? (Wait ...
I was already there, wasn't I?)
* Beer.

Then there are no point and case is closed, right?

deleteme — Fri, 07 Aug 2009 14:34:31 +0000

But the e1000 driver is in the tree as well, right? Which has been stated before in this thread, it's nice to have the source directly from the vendor as well.

Please

efexis — Fri, 07 Aug 2009 14:14:58 +0000

The wrongness of one doesn't increase the rightness of another, and the politeness of another isn't a prerequisite for the politeness of oneself. I must say I rather see a single inpolite message in a thread than many, and many others here agree, which is why we prefer for people not to sink to squabble. I think it's fair enough to point that out. There's always LKML and slashdot for flamefests :-)

GNU + Linux = broken development model.

efexis — Fri, 07 Aug 2009 11:24:38 +0000

"Why else should Windows needs exponentially more resources with each release?"

Sloppy development, greater number of features we're greater for, and greater number of features that we're really really not.

Windows uses subsystems for different API sets (whether it's dos, posix, win16, win32, .net etc) that are loaded on use, with backward compatibility confined within the subsystem, or even different versions of the library or live patching. MS are also always pushing their newest 'bestist' API sets, which means if you're not running old software, you tend to not be needing to load the old APIs (you can have windows run with win16/dos support completely removed for example) so there's no extra resource requirement there... and if you are running old software, then of course you need the old support.

Win7 looks to have lower requirements than Vista (I can even run it on my laptop, something I couldn't dream of with Vista), but still, it's much slower for basic functionality than my 2003 install. Basic stuff like recursive file meta data scanning, more progress bars (which require two-pass operations, one to find out how much stuff there is to be done, and the second to do it) slows down the UI. Services to control CPU scheduling for media applications adds complexity there, as do the various annoyance^H^H^H^H^H^H^H^H^Hsecurity services. It's genuine complexity of the OS combined with less than optimal coding that makes it chug, backward compatibility cruft really isn't as big a player as you might think.

GNU + Linux = broken development model

efexis — Fri, 07 Aug 2009 11:01:30 +0000

"(This is a blatant, content-free, troll.)"

You kiddin? That's the most helpful thing I've read this thread! :-)

Why not fix the tty interface?

forthy — Thu, 06 Aug 2009 15:27:05 +0000

Yes, I remember that when I started writing a Forth on Unix nearly 20 years ago. The tty interface is broken. Fundamentally. It's not even fixable. Give me a proper tty interface instead, and dump the old one. Write a CUSE driver for legacy applications.

Points I have to complain:

Obtain cursor position - impossible when the user types in something at the same time, due to inherent race condition. The root cause of this problem is terminal emulation. Whoever wants terminal emulation should link to /lib/libvt100.so or whatever terminal he likes to have.
raw/cooked mode and modes that persist over application changes: This is insane. Give everybody raw terminal access by default and whoever wants it cooked should link to /lib/libkitchen.so.

The Forth terminal model (that's what I wanted to adapt to Unix, and still haven't found a way to do that stable and bug-free) is quite simple: You can type text (which can be completely uninterpreted plain characters), you can send a carriage return, clear the screen, position the cursor, query the cursor position, (even more advanced: also the size of the terminal; setting colors and styles would also be advanced), check if there's a keyboard event available, obtain one and dispatch what type of event that is (character, function key - note that the terminal event model makes the terminal side of the pty a symmetric task to writing the client side). This separates terminal data (text) from terminal metadata (formatting, cursor positions, colors), and therefore is sane interface design. Each of the functions does exactly one thing, and does it right. Writing a Forth terminal driver for raw hardware (or just GUI emulation) is a piece of cake. Writing it for Unix is a PITA. If it is more complicated to do it through the OS than without, there's something seriously wrong with the OS.

If you happen to have a real terminal (old greenish CRT attached to a RS232 cable), you have to translate the metadata into data, and adapt to the terminal emulation (vt100 or any other of 1000 terminal "standards"). This ought to be an intermediate layer, provided by the system, not by the application. In Unix, you can't position the cursor just by calling an ioctl(). You have to use termcaps to find out which arcane set of string will position the cursor for you, or use one or the other arcane library ([n]curses is arcane) to do it for you.

out of tree

farnz — Thu, 06 Aug 2009 08:10:37 +0000

I'm generalising from experience of 15 to 20 vendors, of whom only a couple had their drivers upstreamed. The remainder simply did not understand why (to use some examples) I didn't want to use their vendor- specific DVB-T tuning interface instead of Linux's DVB API, or their vendor-specific video capture interface instead of V4L2.

However, I recognise that my experience may be atypical; I therefore mention that I base this on personal experience, so that if I've just had bad luck with vendors, other people can point out vendors who get it right.

out of tree

lysse — Thu, 06 Aug 2009 06:28:04 +0000

> this is from direct personal experience

So why are you generalising from it?

Then there are no point and case is closed, right?

mikov — Wed, 05 Aug 2009 22:17:29 +0000

Sure. And we have many OSes which promised and failed to deliver that: Windows, Solaris, etc. It just does not work. Drivers from NT don't work in Windows 2000, drivers from Windows 2000 crash the Windows XP, drivers from Solaris 8 are unusabel in Solaris 10, etc.

Well, I am talking about a source level API, so I don't see your point really. Plus, I disagree with your example. The vast majority of drivers did work between Windows versions. Speaking both as a user and as an ex-Windows kernel developer.

What I do see is the wish to reduce number of problems for manufacturers by increasing number of problem for users and developers - and I can not agree that it's good idea.

I am sorry but you are dead wrong. Very ironically just today (about hald an hour ago) I encountered exactly the problem I am describing.

I was asked to troubleshoot a Debian Lenny image on a new batch of POS machines. Well, guess what, Lenny's kernel 2.6.26 does not support the network adapter (Intel 82567LM-3).

So, WTF am I supposed to do? All the "wisdom" against out-of-tree drivers and stable API expressed in the numerous responses does not help me a bit! Should I perhaps install a newer kernel version? Huh? What about security updates? Backport a driver from 2.6.28? I don't have time to do that, and I am not a network driver expert.

Fortunately for me, Intel maintains an out-of-tree driver at http://e1000.sf.net/ . An out of tree driver saved the day! As usual and as expected.

Unless somebody can offer me a better solution than this out-of-tree driver, I don't see how the arguments carry any weight.

Then there are no point and case is closed, right?

khim — Wed, 05 Aug 2009 19:40:15 +0000

The point of a stable API is precisely that - to continue using the same source with the newer OS-es which are currently supported.

Sure. And we have many OSes which promised and failed to deliver that: Windows, Solaris, etc. It just does not work. Drivers from NT don't work in Windows 2000, drivers from Windows 2000 crash the Windows XP, drivers from Solaris 8 are unusabel in Solaris 10, etc.

Right now Linux supports more hardware then Windows - sure, the "latest and greatest" gadgets are supported by "latest and greatest" release of Windows only, but sheer number is on Linux side.

Stable OS (not stable API, but stable OS - I certainly remember cases where switch from NT4SP5 to NT4SP6 was impossible because drivers had no suppoort for this version) makes support for some hardware easier and for some hardware harder (recall NT4 and USB, for example, then do the same with Pixel Shaders 4), so I see no huge advantages either way.

What I do see is the wish to reduce number of problems for manufacturers by increasing number of problem for users and developers - and I can not agree that it's good idea.

If you want to make life easier then implement stable API where it naturally exist anyway: on the border between your hardware and my hardware!

out of tree

farnz — Wed, 05 Aug 2009 18:45:34 +0000

Hauppauge cards tend to work out of the box with a recent enough kernel; when they don't, Hauppauge employees are happy to help you make the cards work. Other vendors cards require developer time to make work, and attempts to get the vendor to help tend to be fruitless (they have their own out of tree driver that doesn't work well, due to reinvention of interfaces, and don't understand why you won't simply rewrite your userspace to match their driver).

Developer time is hugely expensive, as we don't have much to spare; given the choice between a small amount of extra money per card (to the vendor), or developer time, we'd rather pay. If you're a vendor, looking to expand, this is a market, and it has money to spend on you.

out of tree

johnflux — Wed, 05 Aug 2009 18:15:04 +0000

> I don't quite follow. Can you please clarify a bit? Why do the other cards need huge amounts of money? Also, you are saying that Hauppauge is more expensive because it has to support Linux. So, how is that a good thing?

I think his point was that his company was willing to pay extra just for linux support. So Hauppauge was able to charge more money, and still get his custom. Whereas the companies that didn't support Linux didn't get his business.

From a business point of view, if the money that a company loses because of their lack of linux support is greater than the cost of opening the specs or writing the drivers, then the company should write Linux drivers. The poster was providing anecdotal evidence to indicate that they believe that this is the case for some (most?) companies.

On a personal note, I think companies perceive the cost of opening their drivers to be far higher than it really is.

out of tree

mikov — Wed, 05 Aug 2009 18:05:18 +0000

It is a problem in Windows, too; for example, very few graphics card drivers support Windows native screen rotation in XP - you need to use a vendor utility instead.

I don't want to defend Windows, but IIRC that is technically wrong. Unless I am mistaken, XP does not support native screen rotation. It was introduced in Vista.

I do agree with your general point that you can't rely on the vendors to "do the right" thing. Vendors do suck. I have seen really really horrible disgusting things working with closed source drivers. But this is an impossible battle. You can't require all developers for your OS to be perfect. People will do whatever they want to do. The best you can do is give them tools encouraging them in the right direction. Plus, hopefully GPL will shame them into better code.

Look at Google Android. They are basically doing whatever they want with the kernel and not bothering to submit anything back.

They didn't keep the graphics driver API the same from NT 3.1 to Windows XP. Newer OSes cannot run graphics cards using older drivers, pure and simple. No different to Linux, in that respect.

I am sorry but that is not quite true either. In a previous life I used to do Windows kernel development, and graphics drivers specifically. I have ported drivers from NT4 to W2K and WXP. The API remained backwards compatible and even binary compatible.

In NT4 they did a fundamental architectural change by moving the drivers into the kernel, but the source level API remained compatible with NT 3.51.

And note that vendors do need to bring the Linux community on board - my employer alone buys a significant number of TV capture cards a month, on condition that they are supported under Linux. We pay a slight premium to buy Hauppauge cards because of the Linux support; other vendors are £5/card cheaper, but cost huge amounts of money in expensive developer time keeping the card going.

I don't quite follow. Can you please clarify a bit? Why do the other cards need huge amounts of money? Also, you are saying that Hauppauge is more expensive because it has to support Linux. So, how is that a good thing?

out of tree

farnz — Wed, 05 Aug 2009 10:32:42 +0000

> The thing is, and this is from direct personal experience, companies > maintaining out of tree drivers don't put their driver through the Linux > code review process; as a result, they ignore standard Linux interfaces > in favour of inventing their own. I have no opinion about this because I haven't personally experienced it, but I believe you. But why isn't this a problem in Windows?

It is a problem in Windows, too; for example, very few graphics card drivers support Windows native screen rotation in XP - you need to use a vendor utility instead. The difference is that Windows users are used to this, and don't complain as loudly about it. It's becoming less of an issue as things like USB standardize more and more device classes, and manufacturers use the standard classes, because that means that the device is Plug And Play under currently shipping Windows systems (Vista, future Windows 7 installs).

> Add to that the fact that they lose interest in maintaining their driver > at all when they've moved onto the new shiny (in the case of one vendor > I've dealt with, the driver stops being maintained as soon as they have > a new chip out, even though they're still selling the old chip), and the > fact that I don't have the time to do more than bugfix drivers that have > already been ported to new APIs, and I find myself thinking that all the > advantages you claim for out of tree drivers are disadvantages from my > perspective. Agreed, but what is your point? You are describing exactly the problems caused by a lack of stable API. Vendors do not want to maintain drivers for hardware which they no longer sell, because it is a never ending cycle of completely pointless but expensive work (from their standpoint). That would not be the case if there was a stable API.

Oops - my bad. I forgot to mention that the case in question was migrating from NT4 to XP in an embedded product. We could still buy the chips in question, but the NT4 driver didn't work in XP, even after we recompiled it against the new DDK (don't know why - I thought there was a stable API?). We ended up having to buy new ASI capture chips, and scrapping the older hardware.

> reminds me of a > subset of nVidia graphics card owners, who complain bitterly (regardless > of technical reasons) when X.org releases a new version of the X server > that isn't backwards compatible with the ancient version of the nVidia > drivers that supports their older hardware; I would like to see an open source Nvidia driver as much as the next guy, but I have to disagree with you. How did Microsoft manage to keep their graphics driver API the same from NT 3.1 to Windows XP? To think that it is a bad thing is to ignore reality.

They didn't keep the graphics driver API the same from NT 3.1 to Windows XP. Newer OSes cannot run graphics cards using older drivers, pure and simple. No different to Linux, in that respect.

And note that vendors do need to bring the Linux community on board - my employer alone buys a significant number of TV capture cards a month, on condition that they are supported under Linux. We pay a slight premium to buy Hauppauge cards because of the Linux support; other vendors are £5/card cheaper, but cost huge amounts of money in expensive developer time keeping the card going.

A tempest in a tty pot

nix — Tue, 04 Aug 2009 21:06:39 +0000

The fact remains that POSIX does not require process exit signal / TTY
flush synchronization, and I know of no Unix that ensures that these
things are synchronized. Your musings about what constitutes 'the device'
are thus not terribly relevant. The system simply does not work the way
you would like it to (and the way some app authors assume).

out of tree

mikov — Tue, 04 Aug 2009 20:51:56 +0000

> The thing is, and this is from direct personal experience, companies
> maintaining out of tree drivers don't put their driver through the Linux
> code review process; as a result, they ignore standard Linux interfaces
> in favour of inventing their own.

I have no opinion about this because I haven't personally experienced it, but I believe you. But why isn't this a problem in Windows?

> Add to that the fact that they lose interest in maintaining their driver
> at all when they've moved onto the new shiny (in the case of one vendor
> I've dealt with, the driver stops being maintained as soon as they have
> a new chip out, even though they're still selling the old chip), and the
> fact that I don't have the time to do more than bugfix drivers that have
> already been ported to new APIs, and I find myself thinking that all the
> advantages you claim for out of tree drivers are disadvantages from my
> perspective.

Agreed, but what is your point? You are describing exactly the problems caused by a lack of stable API. Vendors do not want to maintain drivers for hardware which they no longer sell, because it is a never ending cycle of completely pointless but expensive work (from their standpoint). That would not be the case if there was a stable API.

First, it would be very cheap for them to continue the maintenance, so they would likely do it. Second, even if they didn't, the driver should just continue to work. And lastly, a volunteer can pick it up at any moment.

> I want one place to go to for drivers. Not twenty trees, but the one
> tree on kernel.org. I have found it easier to backport drivers from
> 2.6.30 to 2.6.23 than to integrate vendor drivers, so I would prefer in-
> aiming to move into the tree (before GregKH's staging area), and that
> was quite enough pain.

Well, that is you but you are the exception. 99.9% of the Linux users and most of the developers can't backport a driver. While I personally also could do it, there is more than that - for example I don't really have time or support such backported driver in a business setting.

There is definite value in a central repository for drivers. The problem with the current approach is that the release cycles of all drivers are tied together. Ideally I would like to see a central repository where vendors are free to upload their drivers whenever they want.

> I'm also afraid that your admission that you're stuck on an old kernel
> due to an out-of-tree driver from your board vendor [...]

I have never said that. I mentioned a problem where newer webcam drivers are only available for new kernels. This has nothing to do with in-tree vs out-of-tree drivers; it is precisely a matter of lack of stable API.

It seems that many people who responded negatively to my comments here are confusing a couple of related by separate issues:

* Stable API. It is equally beneficial to in-tree and out-of-tree drivers because in either case it decreases the maintenance burden. In fact contrary to the common wisdom, a stable API gives more freedom to core developers to make changes because they can keep them isolated behind the facade of the stable API.

* In-tree vs out-of-tree drivers. It is true that a stable API decreases the incentive to submit drivers to the kernel because the "free maintenance" red herring is gone. But if people perceive value in having a central point for driver development, there is nothing stopping them.

* Closed source drivers. This is completely unrelated to a stable source level API. They are prohibited by the kernel license and suffer from their well known problems.

> reminds me of a
> subset of nVidia graphics card owners, who complain bitterly (regardless
> of technical reasons) when X.org releases a new version of the X server
> that isn't backwards compatible with the ancient version of the nVidia
> drivers that supports their older hardware;

I would like to see an open source Nvidia driver as much as the next guy, but I have to disagree with you. How did Microsoft manage to keep their graphics driver API the same from NT 3.1 to Windows XP? To think that it is a bad thing is to ignore reality.

> I do feel that you should be
> exerting pressure on your vendor to make this happen, either by getting
> their drivers into the kernel, or getting them to step up and do the
> work of maintaining a stable in-kernel API. It's not going to come for
> free, so why shouldn't the people who want such an API do the work of
> maintaining it?

Agreed in theory. But you are forgetting that these vendors don't need to support Linux at all. It is less than 1% of their market, but it takes more work to support that Windows. So, why the hell would they do it at all?

So, they don't have to do anything. It is not their problem. If the Linux community wants to bring these vendors fully on board, it should make it easy for them. Easier than Windows. The major factor for "easiness" is stable APIs.

GNU + Linux = broken development model

mcmanus — Tue, 04 Aug 2009 17:43:23 +0000

I've not personally come across single application which works in XP but not Vista.

Here's a good one: http://msdn.microsoft.com/en-us/library/aa376635%28VS.85%29.aspx

Packet Filtering Functions

[These functions are available for use in the operating systems listed in the Requirements section for each function. In Windows Server "Longhorn", these functions return ERROR_CALL_NOT_SUPPORTED. The MprConfigInterfaceTransportSetInfo function and the Windows Filtering Platform API Management Functions provide similar functionality.]

GNU + Linux = broken development model

nye — Tue, 04 Aug 2009 16:02:35 +0000

I've not personally come across single application which works in XP but not Vista. The primary problem with Vista is that it's noticeably slower.

Comparing 95/98 with XP/Vista is fairly absurd. They're not different versions of the same OS; they're completely different OSes which both have very-nearly compatible implementations of some important APIs, most notably win32. In fact, come to think of it, I can't name a single win32 application that works in Windows 95 and not Vista. The only applications that have broken along the way either interact with the OS at a lower level, or are 16-bit, or actually DOS applications. That would be a bit like expecting Linux to provide perfect binary compatibility with Minix.

The only substantiated bitching I've heard has been about drivers, but you were talking about userland apps; I contend that the number of broken userland apps is vanishingly small.

A tempest in a tty pot

alex_the_d — Tue, 04 Aug 2009 08:46:21 +0000

> The device and buffering for it are kernel entities.

No, 'kernel entities' are 'device driver(s)'. THE device is that beige box on the table or a piece of
code emulating it (emacs).

out of tree

farnz — Tue, 04 Aug 2009 08:06:19 +0000

The thing is, and this is from direct personal experience, companies maintaining out of tree drivers don't put their driver through the Linux code review process; as a result, they ignore standard Linux interfaces in favour of inventing their own.

Add to that the fact that they lose interest in maintaining their driver at all when they've moved onto the new shiny (in the case of one vendor I've dealt with, the driver stops being maintained as soon as they have a new chip out, even though they're still selling the old chip), and the fact that I don't have the time to do more than bugfix drivers that have already been ported to new APIs, and I find myself thinking that all the advantages you claim for out of tree drivers are disadvantages from my perspective.

I want one place to go to for drivers. Not twenty trees, but the one tree on kernel.org. I have found it easier to backport drivers from 2.6.30 to 2.6.23 than to integrate vendor drivers, so I would prefer in- aiming to move into the tree (before GregKH's staging area), and that was quite enough pain.

I'm also afraid that your admission that you're stuck on an old kernel due to an out-of-tree driver from your board vendor reminds me of a subset of nVidia graphics card owners, who complain bitterly (regardless of technical reasons) when X.org releases a new version of the X server that isn't backwards compatible with the ancient version of the nVidia drivers that supports their older hardware; I do feel that you should be exerting pressure on your vendor to make this happen, either by getting their drivers into the kernel, or getting them to step up and do the work of maintaining a stable in-kernel API. It's not going to come for free, so why shouldn't the people who want such an API do the work of maintaining it?

out of tree

mikov — Tue, 04 Aug 2009 04:47:33 +0000

Yeah right. Which parts of your Mac did you buy from a different hardware vendor?

out of tree

foom — Tue, 04 Aug 2009 00:38:46 +0000

I'm pretty sure I don't use any 3rd party drivers on my Mac laptop, and I certainly haven't missed
them or thought MacOS was not ready-for-the-desktop because of it...

out of tree

mikov — Mon, 03 Aug 2009 23:55:21 +0000

the people that you need to convince are not here, you need to go elsewhere to convince them.

Do you mean this in the sense "shut up and go away"?

I don't want to convince anybody. I think that a stable API is a lost cause, like Linux on the desktop, or Linux gaming. It is time to wake up and smell the flowers. The desktop PC is almost gone.

I am curious about intelligent objections against a stable API, but I am not crazy to go get flamed on LKML. (If you do a search, there have been others, brave or stupid enough to post about a stable API there. LOL. Poor suckers. They deserved what was coming to them.)

The people who would need convincing are the big companies who pay for Linux kernel development. It is more than a bit naive to think that volunteers in their free time can influence Linux at this point of its development. But I have a pretty good theory why RedHat, Novell, etc, aren't interested in a stable API.

out of tree

mikov — Mon, 03 Aug 2009 23:38:35 +0000

Why can't it be upgraded? Because of an out-of-tree driver? :-D

Laugh all you want :-) But it is a real problem that should not exist... Ignoring it doesn't make it any less important for the developers and users of this product.

I find that unlikely :-) I've bought 4 webcams randomly, 2 built into laptops, and all just worked in linux.

Well, I am not making it up. BTW, I am running Kubuntu LTS and Debian Stable on my desktops. Perhaps you see my problem? ;-)

Nah, I meant that if I re-install, for an unrelated reason, then I have trouble getting the drivers.

Well, that is the problem of closed source, no argument from me there.

BTW, you can still save the drivers. You have to copy the INF file and the files referenced from it and you can reinstall them on a new system without any tricks. Some things in Windows really are that easy :-)

A tempest in a tty pot

nix — Mon, 03 Aug 2009 22:30:14 +0000

The process of sending the data through a pty or tty is desynchronized
from the state of the process doing the sending. If a process sends data,
then exits, it can be gone (and SIGCHLD fired) *long* before the buffers
are flushed to the receiving app. This has always been true (I first
remember noticing it on SunOS 4, but it was old even then) and is
essential for good performance over slower lines or under high load.

(Emacs is not the device. The device and buffering for it are kernel
entities. Emacs is merely the entity on the *master side* of the device.)

out of tree

johnflux — Mon, 03 Aug 2009 22:29:10 +0000

Well, you are implying that it is somehow "bad" to have an existing working GPL driver, which just happens to be out of tree.

_Compared_ to having it in tree. That's all I've ever said.

> That is the fallacy. In fact it cannot possibly negatively affect the desired outcome, which is to have a working driver.

Compared to not having it at all, sure.

> That is a funny example because I have exactly the opposite experience with an embedded project. It is not possible to buy a working camera for it because its kernel can't practically be upgraded.

Why can't it be upgraded? Because of an out-of-tree driver? :-D

> And even with a modern kernel, I can't find a single webcam from Fry's to work on my Linux desktop.

I find that unlikely :-) I've bought 4 webcams randomly, 2 built into laptops, and all just worked in linux.

> I don't get it. Why would you have to re-install Windows for a single driver?

Nah, I meant that if I re-install, for an unrelated reason, then I have trouble getting the drivers.

out of tree

dlang — Mon, 03 Aug 2009 22:19:49 +0000

what makes you think that the pool of developers to maintain the drivers in the kernel is finite while the pool of drivers is not?

the number of people who contribute to the kernel continues to grow.

you believe that the unstable API is a drawback, many others believe that is is a strength.

the people that you need to convince are not here, you need to go elsewhere to convince them.

out of tree

mikov — Mon, 03 Aug 2009 22:14:25 +0000

There's no logical fallacy there. I stated that an out-of-tree driver requires a lot of work to get into the kernel if it has to be done once the company has gone bankrupt, compared to if the driver was in the tree in the first place. But you're stating that an out-of-tree driver is easy than no driver at all which is true, but not really relevant to an in-tree or out-tree discussion.
I do get where you're coming from, but I do think that in the long run, Linux will have the bigger advantage of having all the drivers in tree.

Well, you are implying that it is somehow "bad" to have an existing working GPL driver, which just happens to be out of tree. That is the fallacy. In fact it cannot possibly negatively affect the desired outcome, which is to have a working driver.

I actually think that there should be no drivers in tree. That is an unmanageable situation - keeping all possible drivers in one monolithic source tree, and upgrading them at every kernel release. It should be obvious that this simply cannot scale. You cannot expect a finite number of developers to maintain a potentially infinite number of drivers for ever.

The other day, for example, I need to get a webcam working on an ARM device. In linux it was easy because all the drivers were just there. I plugged in the webcam and it just worked. That sort of thing is simply not possible in Windows.

That is a funny example because I have exactly the opposite experience with an embedded project. It is not possible to buy a working camera for it because its kernel can't practically be upgraded. And even with a modern kernel, I can't find a single webcam from Fry's to work on my Linux desktop.

AFAIK, there is a very high turnover rate with webcam chips, so it simply does not make economic sense to maintain a driver for hardware whose life is 6 months. Now, if only there was a stable driver API ... :-)

Likewise I dread re-installing Windows because the drivers for my wireless card are difficult to get. The company that made the drivers has since gone and their website is now down.

I don't get it. Why would you have to re-install Windows for a single driver?

Let's for a second forget about the labels Linux and Windows and look at it as impartial scientists performing an experiment. We have "OS A" which owns 99% of a market, and "OS B" which has 0.5% and doesn't seem to be growing even though it is much cheaper (!?!). What reasons could we have, as impartial scientists, to believe that "OS B" is somehow superior?

Personally I obviously think that Linux is superior in many respects, or I wouldn't be here. But I am sure that that it is despite the lack of stable API, not because of it.

out of tree

johnflux — Mon, 03 Aug 2009 20:08:30 +0000

> I am sorry but this is a logical fallacy. This volunteer is free to develop a driver of his own using his superior coding standards from day one. Plus, surely having an existing working driver is much better than starting from scratch.

There's no logical fallacy there. I stated that an out-of-tree driver requires a lot of work to get into the kernel if it has to be done once the company has gone bankrupt, compared to if the driver was in the tree in the first place. But you're stating that an out-of-tree driver is easy than no driver at all which is true, but not really relevant to an in-tree or out-tree discussion.

I do get where you're coming from, but I do think that in the long run, Linux will have the bigger advantage of having all the drivers in tree.

The other day, for example, I need to get a webcam working on an ARM device. In linux it was easy because all the drivers were just there. I plugged in the webcam and it just worked.
That sort of thing is simply not possible in Windows.

Likewise I dread re-installing Windows because the drivers for my wireless card are difficult to get. The company that made the drivers has since gone and their website is now down.

out of tree

mikov — Mon, 03 Aug 2009 17:48:46 +0000

Out-of-tree drivers don't have to be GPL (well, that's a up to debate, but there are certainly proprietary drivers).

That is not really true. I am not aware of anything that allows a Linux kernel driver to be non-GPL. The only gray area is if the driver has been ported from another OS, and this is also questionable. I have seen more than one clear statement from Linus Torvalds on this subject.

There is no question that people violate the Linux GPL all the time. It is ethically disgusting (although I actually understand very well why they do it).

But in any case, a stable source API has not relevance to that problem.

And even if it's GPLed, if it's out of tree then there's a good chance that it doesn't meet the coding standards for inclusion into the kernel - for example a company typically might have a single source for Windows and Linux, with hundreds of #ifdef's to switch between them. There is also often a custom build system, revision control system, etc.

And that is bad because? The current standards are needed because the kernel developers have to be able to "maintain" (more like coax into compiling) a mountain of drivers they don't care about. If the drivers are maintained by interested 3rd parties, it doesn't matter in the least, and those 3rd parties should choose whatever standards of their own they prefer.

So this means that once the company goes bankrupt, some volunteer has to take code that he's probably never looked at before and undertake what could be a large task to clean up and adapt the driver for inclusion into the kernel.

I am sorry but this is a logical fallacy. This volunteer is free to develop a driver of his own using his superior coding standards from day one. Plus, surely having an existing working driver is much better than starting from scratch.

Look, I am well aware that the idea for a stable API is not popular and for sure I am not the best advocate for it. I actually don't think that a stable API will happen. But I also know what I am seeing in my professional life and it doesn't look good.

Also (sorry to repeat myself) it is clear to me that with the current situation Linux will never be really successful on the desktop. The stable API is just one of the several reasons for that, but it isn't a small one.

A tempest in a tty pot

clugstj — Mon, 03 Aug 2009 13:02:02 +0000

Anything as "old/mature" as Emacs probably has other bad assumptions lurking in it. It should come as no surprise.

Assuming an order between read() and signal reception is ALWAYS a bad idea.

Greg KH steps up as the TTY maintainer

csamuel — Mon, 03 Aug 2009 09:46:23 +0000

Just merged by Linus:

commit 57d7f282271a83fe4ca4bd15eee79be577210210
Author: Greg Kroah-Hartman <gregkh@suse.de>
Date:   Fri Jul 31 21:28:16 2009 -0700

    TTY: Maintainer change

    Clearly, I am a glutton for punishment.  I'll see if I can see Alan's
    changes through to the end, otherwise I'll be fending off a lot of bug
    reports for usb-serial devices.

    Cc: Alan Cox <alan@lxorguk.ukuu.org.uk>
    Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
    Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

out of tree

johnflux — Mon, 03 Aug 2009 06:38:29 +0000

Out-of-tree drivers don't have to be GPL (well, that's a up to debate, but there are certainly proprietary drivers). And even if it's GPLed, if it's out of tree then there's a good chance that it doesn't meet the coding standards for inclusion into the kernel - for example a company typically might have a single source for Windows and Linux, with hundreds of #ifdef's to switch between them. There is also often a custom build system, revision control system, etc.

So this means that once the company goes bankrupt, some volunteer has to take code that he's probably never looked at before and undertake what could be a large task to clean up and adapt the driver for inclusion into the kernel.