The idea that a kernel can maintain perfect userland ABI/API accross all versions for ever and ever is just _wrong_ thinking. They can't do it. It's impossible and is something that they have consistantly failed at over and over again.
Ideally it should be like that, but it isn't and won't ever be. Sorry. Can't do it. It's impossible. Nobody is that perfect. Soooo.... What should they do differently?
Version Numbering the ABI
Every 4 years or so introduce another new userland-ABI compatibility version. This gives people a chance to clean up /proc, /sys, bad sys call designs, odd tty insanity, break compatibility with old X drivers, clean up file system behavior, etc etc.
Then try to do your best to support the previous ABI. So that the kernel supports 2 versions at most. So that gives you about 8 years of ABI/API compatibility.
Because right now what is happenning is that the Linux kernel isn't supporting 1 ABI version.. it's supporting _hundreds_. Each time a kernel developer introduces a bug or a odd behavior and applications start taking advantage of that then that is a new feature that Linux will have to support for the rest of the time. (which it will fail at eventually)