Btw, if you care at all about your data, you will not run
Ubuntu's release that doesn't fix the kernel and instead
turns timestamps off.
If you turn timestamps off, at rates of 1GB/s and above you
are exposed to possible sequence number wraparound. This in
turn can lead to data corruption. Without timestamps there is
no PAWS protection (Protection Against Wrapped Sequence numbers)
and thus at high enough data rates new data can be interpreted
as old data and vice versa, corrupting your data stream.
Ubuntu made the wrong decision, there is simply no argument for
the way this was "handled."
I don't understand why everyone gets their tits in a knot when
even the slightest suggestion of slipping a release is suggested
in order to fix a serious bug like one of this magnitude. It is
always the right thing to do, and it avoids crap like what is
happening here.
To reiterate, if timestamps are off, you are exposed to possible
data corruption.
Posted Oct 29, 2008 22:04 UTC (Wed) by jspaleta (subscriber, #50639)
[Link]
So its a data corruption issue on top of a security issue.
Has this been communicated into the Ubuntu bug tracker?
-jef
Networking change causes distribution headaches
Posted Oct 29, 2008 22:17 UTC (Wed) by nick.lowe (subscriber, #54609)
[Link]
Yes, I have quoted and posted a link for that very reason.
Networking change causes distribution headaches
Posted Oct 29, 2008 22:32 UTC (Wed) by jspaleta (subscriber, #50639)
[Link]
In the context of the problem the workaround attempts to fix... data corruption is a significant problem because it impacts your ability to get updates. Wasn't that the underlying motivation for the quickfix as implemented? Or am I misreading?
-jef
Networking change causes distribution headaches
Posted Oct 29, 2008 22:54 UTC (Wed) by nick.lowe (subscriber, #54609)
[Link]
"data corruption is a significant problem because it impacts your ability to get updates"
No, not at all. :)
It is a seperate issue.
The workaround -introduces- a data corruption problem at high data rates because it disables PAWS protection in the TCP/IP stack by virtue of the timestamps no longer being there.
The issue here is that the server release will go out with this, which will be run on machines highly likely to see these data rates!
Networking change causes distribution headaches
Posted Oct 30, 2008 3:02 UTC (Thu) by njs (subscriber, #40338)
[Link]
>The issue here is that the server release will go out with this, which will be run on machines highly likely to see these data rates!
Obviously this whole situation is unfortunate, but... is your suggestion really that there are large numbers of people with GB/s equipment who are likely to jump to a non-LTS Ubuntu release, on the first day, and don't read release notes, and don't install updates? Because that seems like a relatively narrow slice of the userbase to me -- not so narrow it should be ignored, so I'm glad you're continuing to help the ubuntu devs keep on top of things, but narrow enough that some of the other folks in this thread could maybe stand to relax a bit...
Networking change causes distribution headaches
Posted Oct 30, 2008 3:40 UTC (Thu) by nick.lowe (subscriber, #54609)
[Link]
Fair point :)
Networking change causes distribution headaches
Posted Oct 30, 2008 6:27 UTC (Thu) by davem (subscriber, #4154)
[Link]
Feel free to ignore the security implications of this change
as I detailed in an earlier comment.
I just mentioned the data corrupter just to show how absolutely
insane this was on just about every level.
Want to know the litmus test of how stupid this is? Not one
damn ubuntu kernel developer asked any of the core networking
folks for guidance on how to handle this problem. They didn't
know the implications, and they didn't bother to ask people
who did.
That's the definition of failure.
Networking change causes distribution headaches
Posted Oct 30, 2008 16:08 UTC (Thu) by hppnq (subscriber, #14462)
[Link]
Well, you would have to be doing a distribution upgrade, boot into it
immediately and go into production without looking for updates. I think it
is fair to assume that not too many people would run into TCP timestamp
related corruption. If they really care about their data,
obviously their scripts would notice the absence of TCP timestamping with
this new release.
Here's a simple explanation for Ubuntu's decision.
As a side note: for home users -- who are extremely unlikely to be running
at high enough data rates -- there is (also) the option to revert to the
last working kernel. Maybe in a next release, this specific kind of
distribution problem will actually be "solved" by Ubuntu. Which would be
very nice.
Second side note: a couple of years ago PAWS users were vulnerable -- on a
rather big scale -- to a remote Dos. Your mileage will always vary.
Phasing out the broken-workaround procps
Posted Oct 30, 2008 9:49 UTC (Thu) by ncm (subscriber, #165)
[Link]
I still haven't seen a plausible story of how they're going to phase out the buggy workaround. Will there be a good procps package that conflicts with the bad kernel, and a good kernel that conflicts with the workaround procps version? Will they be in the security-updates repository?
Phasing out the broken-workaround procps
Posted Oct 30, 2008 9:56 UTC (Thu) by njs (subscriber, #40338)
[Link]
> Will there be a good procps package that conflicts with the bad kernel, and a good kernel that conflicts with the workaround procps version?
Sure, maybe. Is your objection literally that you don't know how they're going to phase it out, or that you're worried that in fact they won't phase it out? I don't know how they're planning to do it (though the suggestion somewhere upthread of checking the runtime version of the kernel sounded plausible to me, and they can just drop it altogether in 9.04 in any case), but I'm pretty confident that they don't want to carry this annoyingness around and will find some way to get rid of it, and the exact mechanism they choose doesn't affect me, so I don't really care what it is.
Phasing out the broken-workaround procps
Posted Oct 31, 2008 1:43 UTC (Fri) by jamesh (subscriber, #1159)
[Link]
I just updated my system, and got a new procps package (1:3.2.7-9ubuntu2.1) that drops the sysctl changes and a new linux kernel package (2.6.27-7.15) that fixes the options order. Both packages came through the intrepid-security repository.
Phasing out the broken-workaround procps
Posted Oct 31, 2008 2:11 UTC (Fri) by ncm (subscriber, #165)
[Link]
Okay, then.
At what point will the download CD/DVD images get the updates?
Networking change causes distribution headaches
Posted Oct 31, 2008 18:50 UTC (Fri) by Cato (subscriber, #7643)
[Link]
If you really mean 1 Gigabyte/second transfer rates, that means a 10G Ethernet link or the equivalent using SONET/DWDM, from a single host, and probably over a long fat pipe (e.g. satellite), which sounds exceedingly unlikely for Ubuntu, which is aimed at consumer and business usage - more like something CERN would do.
IMHO, anyone who is not running Ubuntu on a supercomputer using that type of network connection can safely ignore the chance of the PAWS issue corrupting their data.
Even if you meant 1 Gbps, that's an impressive sustained rate for a high latency network session (i.e. the ones where sequence numbers matter, hence over a WAN). A Gigabit Ethernet LAN would almost by definition have low latencies (a few milliseconds).
I'm generalising here, but I really think the PAWS issue is irrelevant to people who are likely to use Ubuntu. If it was an HPC distro it would be quite different.