EuroBSDCon 2005 (NewsForge)

My experience suggests that cleanly disconnected wire pairs are the trivial special case--wires that have been corroded, compressed, stretched, or otherwise damaged so that they are *almost* open or shorted, or have compromised signal integrity, but still work just enough for link detection in at least one direction, cause all kinds of really annoying problems that just get worse when there are dynamic failover systems in the mix.

I've had cables that would do interesting things like negotiate 100mbit-FD at one end and 100mbit-HD, 10mbit, or nothing at the other, until I cut off one end and re-crimped it. Other cables would carry short packets but get frame errors on long ones. This confuses simpler systems like ifplugd and "ip route add ... nexthop" ARP-based failover, because the cable works for the simple signal tests they are doing but fails to work for real live data packets.

I'd imagine that a number of these marginal-signal-quality cases might arise at various instants during molestation of the network cables with an axe.

On the other hand...

Standard CD-audio frames are 2352 bytes long and take 13.3ms to play. An Ethernet frame is 1500 bytes minus a bit of overhead and takes 8.5ms to play. Assuming a TCP connection carrying the data with a window size of 64K, the failover system has to recover *and* get an ACK packet through from the player to the audio source within 370ms of the first lost packet, or the audio player will run out of data. That's pretty easy to do with no buffering in the sound card at all (although since there is considerably more than 64K of buffer in both the sound driver and the TCP stack, you'd have to turn all those buffers off to produce this result in real life).

Now, if the demo involved streaming 192kbit/s Ogg data like the recent OpenBSD song, then one Ethernet frame takes 62.5ms to play, and a 64K TCP window needs to recover within 2.6 *seconds*. ARP-based failover can recover in that kind of time if you set the cache parameters low enough. And of course typical buffering will keep the music playing for much longer.

OTOH, if he's firing UDP audio packets through the firewall in real time with no ACK path, then then he's got a pretty impressive demo after all.

However, I can tell you that sub-microsecond failover is certainly doable, if it is either:

"Hot Standby" (ie: nothing needs to be loaded or configured at the time, although you do need to have some sort of start/stop operation)
• Mirrored connection (ie: ALL network devices are live at the same time, the grouped outbound connections have the same MAC address, as do the grouped inbound connections, with duplicate packets being dropped either side by filtering hardware)

Hot Standby is what you'd use under any kind of "real world" scenario, but if you plan on building a mach 20 aircraft, or want a probe that will RELIABLY reach Mars or a comet, then you'd be better off mirroring the connection as you really can't afford the latency in Hot Standby, and/or can't go out there to physically debug the code if the stop/start hangs.

All depends on just what sort of system you're building.