The interface IS the source code [LWN.net]

The interface IS the source code

Posted Apr 20, 2006 13:46 UTC (Thu) by elanthis (subscriber, #6227) [Link]

This "get it into the mainline kernel" line is just as broken today as it ever was.

Here's an example:
I built a new box a month or two ago. I wanted to install Ubuntu. However, the very most recent stable Ubuntu release *wouldn't install* because various bits of the hardware had no drivers in the ancient (by Linux standards) kernel in the Ubuntu Breezy install disc. The same went for the most recent Fedora release at the time. In short, it was impossible for me to install any of the OSes I want onto this machine without downloading and installing a beta/pre-release version of an OS (Ubuntu Dapper, in my case).

Now, let us assume that the kernel had a stable driver ABI. Or, heck, we don't even need that; let's pretend it had a stable driver API. I could have downloaded the additional and updated drivers necesasry to install Linux on my new box, put them on a floppy, and told the installer to use those driver binaries or even to compile those drivers at install time. And voila, I could've installed just about any Linux onto this machine without resorting to installing buggy unstable pre-release software for hardware which was already close to 6 months old in shelf-time.

The "get it into the mainline kernel" line essentially means that in order to ever support new hardware, you have to upgrade your kernel. And, because no distro in its right mind supports wholesale kernel upgrades in a distribution release's lifetime (look at how much testing and effort has to go into getting a simple kernel patch package pushed out from a major distro vendor), that essentially means that you have to upgrade your entire operating system - every library, application, and so on - to a new version of the distribution release to get the newer kernel that supports your new graphics card or sound card or network card or whatever. And, if there hasn't been a new release of your distro with the new kernel yet... too bad. You are forced to not use some bit of new hardware, *even if a completely Free GPLd Linux driver has existed for several months for the hardware*.

Your only options become: (a) don't use new hardware, even if you have a genuine need for it; (b) install the kernel from sourcecode yourself, which requires a huge investment of time and energy, and requires knowledge most people don't have and shouldnt' realistically be forced to have, and which means you lose automatic kernel security updates from your OS vendor; or (c) install a beta version OS to get the new kernel.

Again, you don't even need a stable ABI or non-Free drivers; I just want to be able to take a Free GPLd driver that came out five months ago and install it on my OS which came out six months ago. I'm perfectly fan with completely banning non-GPL drivers. Wanting non-Free binary drivers is NOT the issue here!

"Get it in mainline" isn't an answer that actually helps users. The only people it helps are the developers, because it gives them an excuse to not need to put forth the effort for a stable API. It's the developers' project, and I in no way feel that I have a right to tell them how to do things unless I'm paying them to do it. But please stop blathering on with the nonsense that "getting drivers into mainline" actually solves the issues the *users* have with binary drivers. It doesn't.

The interface IS the source code

Posted Apr 20, 2006 14:17 UTC (Thu) by rknop (guest, #66) [Link]

You are wrong -- it DOES solve the problem, at least some of the problems.

A year from now, when all (or most) of the distributions have done new releases, that hardware *will be* supported. Two years from now, it will be supported. But will it be supported if you have to rely on getting binary-only drivers from a vendor? Likely not; the kernel will have changed, and the old driver won't work.

It's ESSENTIAL to get your drivers into the mainline kernel if you few your hardware as anything other than a flash-in-the-pan, not to be used on Linux distributions that were released after six months after the release of the hardware device.

And a workaround to the problem you mention can go with in-kernel drivers. Redhat used to have their secondary archive of "addons". You can make packages for popular distros, or make source packages available, with added kernel drivers. After a few months, when a new distro release comes, everything will get simpler. This happened with the ipw2200 drivers (although we still need an annoying firmware blob... grrr).

Very few traditional vendors realize that the free software/open source development model is fundamentally *different*. A stable kernel API for low-level things like drivers really *is* less crucial, because the *right way* to do things is for everything in the kernel to be free, and thus integratable into the main kernel. They have all lived in the MIcrosoft world where they can list which versions are supported, and write drivers for those-- well, Linux just doesn't work that way, and the fact that proprietary vendors have always worked that way does NOT mean that it's the only way one can work. And, the Linux way really is better when it comes to older hardware-- the drivers generally do stay along and get updated for newer releases, whereas with vendor-supplied binary drivers, you have to count on enough backwards compatability, or the vendor releasing a new driver.

-Rob

The interface IS the source code

Posted Apr 20, 2006 14:18 UTC (Thu) by tetromino (subscriber, #33846) [Link]

Overall, I agree with you, but just to nit-pick:

You say "no distro in its right mind supports wholesale kernel upgrades in a distribution release's lifetime"

Gentoo does. For instance, Gentoo 2005.1 was released in early August 2005. After it was found that it didn't install on some newer AMD64 systems, Gentoo updated the kernel and released 2005.1-r1 in mid-November.

The interface IS the source code

Posted Apr 20, 2006 16:05 UTC (Thu) by cventers (subscriber, #31465) [Link]

I invite advocates of the stable API theory to
read /usr/src/linux/Documentation/stable_api_nonsense.txt. I understand
that your scenario regarding distributions is a problem; the difference
is that it's actually not a kernel developer problem at all.

The problem kernel people have with a "stable API" isn't that they are
somehow lazy -- the problem is that a stable API flies in the face of the
agility of kernel development. This agility means that I can walk into
LKML, identify a bad design, propose a better design, and then in one
sweeping series of patches, replace the entire mechanism. It keeps the
kernel from getting cruft.

If you have a stable API, the time is soon going to come where it gets in
the way. Then you have a driver author hacking around the stable API,
because that's the way to talk to the kernel. And then you have the
subsystem maintainer hacking around the stable API, cause that's the way
to talk to the driver.

So you end up with crufty code talking to crufty code over a rickety old
bridge.

The correct answer is for the distributions to either accept the
responsibility of backporting important modules (because the reality is
often that while the api is not stable, it doesn't change *dramatically*
often enough that backporting would really be a huge effort), or,
alternatively, offer up-to-date kernel packages that users can accept
under a 'beta' headline in order to support their new hardware.

People say the bleeding edge kernels are buggy and unstable. Of course,
something that just hits the road is going to naturally have some
problems. But I've been on the bleeding edge for a long time, and rarely
has it led to any kind of problems whatsoever. In years of using Linux on
servers and desktops, I've never seen the kernel actually crash on me,
with the exception of one time when I had failing RAM. The reality is
that most desktop users that use bleeding edge hardware can probably get
by just fine on bleeding edge kernels, if only the distributions would
step up and accept responsibility of delivering them to their users in a
useable way.

The interface IS the source code

Posted Apr 20, 2006 20:46 UTC (Thu) by oak (guest, #2786) [Link]

> However, the very most recent stable Ubuntu release *wouldn't install*
> because various bits of the hardware had no drivers in the ancient
> (by Linux standards) kernel in the Ubuntu Breezy install disc.

Hm. What was preventing you from upgrading to newer kernel/modules
in Breezy repositories?

> The same went for the most recent Fedora release at the time. In short,
> it was impossible for me to install any of the OSes I want onto this
> machine without downloading and installing a beta/pre-release version
> of an OS (Ubuntu Dapper, in my case).

Or updating just the kernel from Dapper?

(I just installed Breezy on P166 and updated just the X server to Dapper
because Breezy's ATI driver was segfaulting because it was probing for
my PCI gfx card from the ISA bus.)

The interface IS the source code

Posted Apr 21, 2006 17:35 UTC (Fri) by dododge (subscriber, #2870) [Link]

> Hm. What was preventing you from upgrading to newer kernel/modules
> in Breezy repositories?

He may have been talking about the installer kernel itself. For example
if the kernel that boots from the install CD doesn't have a recent enough
driver to be able to reliably access your SATA chipset, you're going to
have a hard time installing the base system onto a SATA drive in the first
place.

The interface IS the source code

Posted Apr 28, 2006 4:39 UTC (Fri) by turpie (guest, #5219) [Link]

How often does that actually happen?
I would think that usually you would be able to install an older distribution and then upgrade the kernel to get support for peripherals.

Planning ahead

Posted Apr 21, 2006 0:08 UTC (Fri) by man_ls (subscriber, #15091) [Link]

How long do you think it takes to bring a product to market? Or, once it is announced (if they don't want to lose the element of surprise), to distribute it in quantity? If it's something like six months, then the manufacturer should have time to submit their driver to lkml and have support in the kernel, in time for the next release of your favorite distro.

Even if the window is just one or two months, they would have time to submit their code to kernel developers and have it in the next kernel release, so that you could just download the kernel in source form and compile it yourself.

Hardware manufacturers have a responsibility to their users, so this process should not be too much of a burden. Certainly less than maintaining binary drivers themselves.

The interface IS the source code

Posted Apr 22, 2006 19:35 UTC (Sat) by davecb (subscriber, #1574) [Link]

Just for everyone's information, the kernel interface
in non-Linux OSs aren't stable either. In Solaris, when
I was on the ABI team circa 2.6, we discussed a kernel
ABI and API, but drivers were acively evolving, so it
didn't happen.

The Solaris 10 driver interface is much more stable, but
it can still be changed if necessary.

--dave

The interface IS the source code

Posted Apr 25, 2006 20:13 UTC (Tue) by rickmoen (subscriber, #6943) [Link]

elantis wrote:

And, because no distro in its right mind supports wholesale kernel upgrades in a distribution release's lifetime....

This, of course, is a pile of foetid dingo's kidneys. You could have:

Installed using the standard Breezy image, provided at least your mass storage host adapter was supported, and then fetched a newer kernel to support the rest of your bleeding-edge hardware.
Used a third-party Debian-compatible image (e.g., a netinst) using a recent kernel, and then cut over using apt-get.
Used the Dapper pre-release installation image. (And why not, sir? It's quite good -- and probably a lot less buggy than the hardware in that "new box" you built.)
Temporarily put your hard drive into an older box, installed Breezy, fetched a newer kernel, moved your hard drive back.
Compiled the one or two drivers you needed, put them on a USB flash drive, and insmod-ed them into the Breezy installer when needed.

The reason nobody's going to hand you a guaranteed-stable binary driver interface, no matter how much you complain, are compelling and amply explained. If you insist on using obscure or bleeding-edge hardware, you have ways to deal with the problem you've imposed on yourself -- far more than you claimed were your "only options".

Alternatively, you're more than welcome to fork the kernel and maintain a "stable API", yourself. Please do: It'd be fun to watch.

Rick Moen
rick@linuxmafia.com