A tempest in a tty pot

Sure, this is the NFL, wear your pads, and all that crap.

As several people just pointed out to me, rather strenuously, in the sexism thread, that's really not a good enough answer. My experience of Alan is not that he's a shrinking violet; perhaps Linus could have approached that in a... slightly more mellow fashion?

I happen to be on the "if there's really a 'right' way for the kernel to do it, and that breaks user apps that made questionable assumptions, then break them" side, myself, but I don't think it's black and white, either.

Particularly in this case, where the odds of that kernel getting into distributions are effectively non-existent, I don't think this was resolved in the best way for all concerned. Though perhaps that observation is obvious enough to be puerile...

If one of the BSDs wanted to fix their TTY code, they would coordinate the changes in the kernel, core userland, and ports. This allows you to change existing bad behavior with minimal risk of breakage, since users are encouraged to upgrade their entire operating system (kernel, userland, and ports) at the same time.

Perhaps there should be a "reference implementation" of a complete GNU/Linux operating system that that is blessed by both Richard Stallman and Linus Torvalds, and where extra effort is made to ensure that all the pieces work together. Existing distributions would be free to fork this implementation and add their own differentiating features, like Ubuntu forks Debian every six months and polishes it up for desktop users.

Sure, I understand why Linus would do that. He has to cope with bug denial from hackers and maintainers all the time. Yet it seems to me he's started to develop a hair-trigger towards developers complaining about the abuses of userspace.

Sure, Alan grumbled about the behavior of TTY consumers in the wild. Even so, he never showed any resistance to fixing these issues. My perception is that things had been moving along well, with regressions being closed left and right.

IMHO, I think Linus should "man up" and offer an apology.

The idea that a kernel can maintain perfect userland ABI/API accross all versions for ever and ever is just _wrong_ thinking. They can't do it. It's impossible and is something that they have consistantly failed at over and over again.

Ideally it should be like that, but it isn't and won't ever be. Sorry. Can't do it. It's impossible. Nobody is that perfect. Soooo.... What should they do differently?

Version Numbering the ABI

Every 4 years or so introduce another new userland-ABI compatibility version. This gives people a chance to clean up /proc, /sys, bad sys call designs, odd tty insanity, break compatibility with old X drivers, clean up file system behavior, etc etc.

Then try to do your best to support the previous ABI. So that the kernel supports 2 versions at most. So that gives you about 8 years of ABI/API compatibility.

Because right now what is happenning is that the Linux kernel isn't supporting 1 ABI version.. it's supporting _hundreds_. Each time a kernel developer introduces a bug or a odd behavior and applications start taking advantage of that then that is a new feature that Linux will have to support for the rest of the time. (which it will fail at eventually)

Whether or not the kernel should keep compatibility (i.e. the fix for the
regressions) is a different question. If two widely used applications show
problems with a new kernel version, then there's a non-zero chance other
applications are "broken" as well. This $unknown needs to be offset with
the cost of maintaining the incorrect behaviour.

This again has little to do with personal issues, which play an important
role in this story.

Alan, you don't know me, I am just a Linux fanboy, but out here in Ubuntuland, I have been rooting for you.

You're the one they sent in where lesser mortals fear to tread.
Nobody else could possibly grok the tangle of code that is the TTY layer.
To me you are the 'asset' dispatched to eliminate the BKL.

Back in the real world :-) ... you are one of the elders of GNU/Linux and you have devoted (almost) half of your life to it because you believe in it.

Today, at this moment of stress, when you have been publicly slighted by the boss, please just remember that we, the people, really are counting on you to do a difficult job, that there really is nobody else eager to step in to your shoes and take the huge workload and _apparently_ small reward.

Most of all please know that your efforts are more widely appreciated than you realise.

1: Work up a new interface ('nu-serial') that communicates serial data and related out-of-band information in a sensible, linear way. Don't design this to comply with RS-232, RS-422 or anything - design it to work with data and userspace requirements.
2: Convert userspace programs across to this new interface.
3: Write the TTY driver to connect to the nu-serial interface.
4: Optimise the nu-serial interface to work well with threads and across cores.
5: Phase out the old TTY interface.
6: Profit.

The problem seems to me to be that everyone's got a slightly different interpretation of what a TTY should do, and it's being bolted into a whole bunch of things like ppp, KDE and emacs that are needing quite different things from the serial interface. If there's already something like nu-serial, then so much the better.

I'm not a kernel developer, though, so I have no ability to either implement it or guide the project. Some might argue that I shouldn't be saying anything. It does seem to me, however, that this is the approach that's been taken in the past with other things - libata, memory allocators, or schedulers being the first (bad) examples that pop into my head.

Have fun,

Paul

Linus wants to make sure all applications works even though you can say they are buggy. Alan says that applications can't rely on on certain old behaviour and must be fixed.

I disagree with Linus: You must be allowed to change things once in a while. If the applications assumes some non-standeard behaviour it is broken. To determine what non-standeard behaviour is, look at how it behaves on OTHER kernels. What can Emacs assume when it runs on BSD, Solaris, etc.? If they also comply with the old behavior of TTY then the new behavior is wrong. But if Emacs already has code to handle the new behaviour because the other kernels already do that, then the new behaviour is ok (this is unlikely as Emacs would probably have no problem with the new tty code.)

In short: If there is no written standeard look at the other OS'es. Applications should not expect special behaviour from the Linux kernel over other Unix kernels unless there is a really good reason to.

http://www.linusakesson.net/programming/tty/index.php

And the answer was "do this insane set of ioctls". Followed by "oh and make damn sure your program can NEVER EVER crash, because you must undo this before exit or other programs will be screwed up.

This was obviously insanely wrong then, and it still is now.

With shells using readline-style libraries now, there are NO programs left that use the TTY in anything other than uncooked mode (programs like cat that are mostly used as destinations of pipes do not count!). It does seem like it is time to scrap this. This would also allow "pty" to be scrapped as well, because you could use plain old pipes for this.

There should be a trivial uncooked raw io interface that you open as "/dev/newtty" or whatever. The drivers should be about 10 lines long I think.

The old tty/pty interface should be a CUSE api of some sort that emulates it atop the new interface. And that can be vastly reduced. For instance we can scrap the entire key assignment thing, as nobody ever alters it, and finally (after almost 4 decades!) get both Backspace and Delete to work at the same time!

1. "Also, no matter how many bytes are requested in the read() call, at most one line shall be
returned." (in canonical mode)

2. "process to close a terminal device file shall cause any output to be sent to the device and any
input to be discarded."

I don't think the problem is in userland, neither emacs nor kdesu. Can someone comment?

This new mpsafe tty version will be in the future 8.0 version of FreeBSD. Why Linux developpers cannot do the same thing ?

if (stdio_raw(1)==0)
{
// I am in raw mode
}
else
{
//something went wrong
}

The c runtime can even track if raw is on or off, or simply reset to non raw on crashes.......

Its piss poor API for userland applications not the kernel that makes *nix a bit of a suck to code.