"Opera was first to have HTTP pipelining on by default."
How long ago was this "first"? Because Mozilla spent (arguably wasted) a bunch of time trying to get to a point where blacklisting was enough to turn on pipelining and comprehensively failed. There were reverse proxies, corporate firewalls, all sort of stuff that felt it was OK to violate the end-to-end rule when it came to HTTP or try to share sockets across threads with no locking and the experience was that both proprietary software vendors AND end sites felt that "it works in Internet Explorer, stop bothering us about this" or just outright ignored bug reports.
Over at Google they also spent a bunch of time fighting one of these protocol violatons, and eventually they gave up and switched off the speed-up for any site that doesn't speak their new non-HTTP protocol.
It's great news if today Opera has this _working_ but your reference to blacklists suggests that they just haven't done enough research/ got enough users to notice that blacklists aren't enough to actually get pipelining working reliably out on the public web. Certainly they weren't when Mozilla experimented with this.
(Some people will say this is a great failing of HTTP, but actually the same happens even down in the lower levels, there are big companies where it's strictly forbidden to use IP multicast, a core feature of the protocol for decades, because their network hardware crashes when exposed to "too much" multicast and they can't get the supplier to fix it.)
Posted Feb 17, 2013 17:37 UTC (Sun) by raven667 (subscriber, #5198)
[Link]
> IP multicast
Tangent alert: Multicast is broken because it requires the network equipment to be smart which fundamentally breaks a core assumption of IP networking that the endpoints are smart but the network is dumb.
Opera moves to WebKit and V8
Posted Feb 17, 2013 19:41 UTC (Sun) by butlerm (subscriber, #13312)
[Link]
Nothing about IP requires such an assumption to be made. Multicast is about the only reasonable way to accomplish a large scale IPTV deployment, and it works well for that. It is only within the Internet backbone that IP multicast is impractical, at the edges it works fine.
Opera moves to WebKit and V8
Posted Feb 18, 2013 3:44 UTC (Mon) by raven667 (subscriber, #5198)
[Link]
I think you are agreeing with me, multicast on the Internet is impractical, although I will also point out that most video on the Internet is unicast (Netflix, YouTube, Hulu, etc.) and not multicast. Multicast seems only to be used in campus and other large scale environments within a single network/administrative domain and even that seems to be fading out.
Opera moves to WebKit and V8
Posted Feb 18, 2013 4:42 UTC (Mon) by dlang (✭ supporter ✭, #313)
[Link]
multicast video only works if everyone wants to watch the same thing at the same time.
most people who could use IP TV are not going to be doing this. They will watch the show they want to watch at the time they want to watch it, and the number of other people who start watching the same show at the same time is so small as to be meaningless.
For broadcasts of live events (political speeches, Sports events, etc) there may be a small niche, but is that really worth the effort of implementing it across such a large infrastructure?
remember that if people aren't consuming the content, all you are doing is wasting available bandwidth.
Opera moves to WebKit and V8
Posted Feb 18, 2013 12:12 UTC (Mon) by ewan (subscriber, #5533)
[Link]
It works if everyone is OK with receiving the content at the same time, which is not the same thing as watching it. A lot of people are OK with subscribing to something in advance, caching the content when it's released, and then watching it shortly afterwards at their convenience - it's what people do with PVRs and traditional broadcast TV.
A model that worked like that over multicast IP, plus a relatively smaller number of unicast streams for 'catch up' services, should be more bandwidth efficient than unicasting to everyone.
Opera moves to WebKit and V8
Posted Feb 18, 2013 13:27 UTC (Mon) by tialaramex (subscriber, #21167)
[Link]
Unfortunately the cost saving from multicast assumes a traditional IP network. In this respect DSL still looks like dial-up did, with every subscriber at the end of a PPP connection. An ISP gets a whole pile of point-to-point links from a nearby POP. The physical infrastructure is largely hidden from ISPs, so there is no way to save money on the expensive long-haul. If you've got data in London, and it needs to go to 600 subscribers in Cardiff, you have to send and pay for 600 copies of the data.
Opera moves to WebKit and V8
Posted Feb 18, 2013 16:35 UTC (Mon) by butlerm (subscriber, #13312)
[Link]
For IP multicast to be useful on a large scale, it must be implemented by the last mile ISP. That is easy enough though - locate a distribution node on the ISP network, unicast across the backbone to it, and then multicast from it to the ISP customers. Every wireline television system in the world is likely to look like this in a few years. Television meaning real time, not stored content distribution.
Opera moves to WebKit and V8
Posted Feb 18, 2013 14:47 UTC (Mon) by nye (guest, #51576)
[Link]
>For broadcasts of live events (political speeches, Sports events, etc) there may be a small niche, but is that really worth the effort of implementing it across such a large infrastructure?
There actually seems to be a fairly large market for live-streamed video, like Twitch and Ustream, or YouTube's live option. However, I don't think any services like this actually try to multicast over the internet; it's unicast all the way[0].
So I guess the answer is 'no' - it's not really worth the effort, even when live video broadcast to tens of thousands of destinations is your principle business; possibly once it scales up to millions is where things start to look different, but at that volume you can invest in the infrastructure to avoid having to send it over the internet.
[0] Last year's edition of TCP/IP Illustrated still describes multicasting over the internet as "ongoing effort...for more than a decade", which seems to correlate with the general consensus I got from Google of "don't even try"
Opera moves to WebKit and V8
Posted Feb 18, 2013 15:54 UTC (Mon) by anselm (subscriber, #2796)
[Link]
So I guess the answer is 'no' - it's not really worth the effort, even when live video broadcast to tens of thousands of destinations is your principle business; possibly once it scales up to millions is where things start to look different, but at that volume you can invest in the infrastructure to avoid having to send it over the internet.
Here in Germany, Deutsche Telekom is happy to sell you access to IP-based high-definition broadcast television, in competition with traditional cable TV providers. They do have a couple of million users. In this context, IP multicast, which is in fact being used, makes a great deal of sense.
Opera moves to WebKit and V8
Posted Feb 18, 2013 16:02 UTC (Mon) by johill (subscriber, #25196)
[Link]
Yep, but that doesn't contradict nye's point -- this isn't on the Internet, it's entirely on their own infrastructure.
Opera moves to WebKit and V8
Posted Feb 18, 2013 19:12 UTC (Mon) by dlang (✭ supporter ✭, #313)
[Link]
but don't they also sell the functional equivalent of a DVR along with that? and don't most users heavily use the DVR functionality to watch shows on their schedule?
If so, it's not really a broadcast situation.
multicast
Posted Feb 18, 2013 2:14 UTC (Mon) by tialaramex (subscriber, #21167)
[Link]
This is link-layer multicast. Because early Ethernet is actually a broadcast network, multicast worked fine there, and because so many subsequent network link layers are pretending to be Ethernet (including 802.11 "WiFi") they all have link-layer multicast too. Today that might mean smart "snooping" modes in switches, but in principle dumb equipment will work just fine, and in practice I've never had any problem with cheap home GigE or Fast Ethernet hardware too dumb to do snooping.
Any manufacturer that does even a little testing will notice if they've actually completely broken link-layer multicast on a LAN because such a variety of things break in that scenario, including features in famous brand name products like Office and Mac OS X.
But if their implementation does something dumb with multicast that only shows up under /load/ then these features (which are mostly discovery mechanisms, using multicast to avoid wasting CPU on uninterested nodes) won't trigger it. A multicast video payload, even on a LAN (so no actual multicast routing) will run into problems though, as will pseudo-reliable multicast file delivery and other techniques. Whoops.
Link Layer Multicast
Posted Feb 20, 2013 2:25 UTC (Wed) by butlerm (subscriber, #13312)
[Link]
Given that less than desirable state of affairs, have there been any proposals to add a new type of Ethernet frame that does not multicast unless the switch is multicast aware? i.e. something that an existing switch will ignore?
Or short of that an IP multicast option that does not use link layer multicast addresses, to avoid this problem? Where an actual router does all the multicasting?
Link Layer Multicast
Posted Feb 20, 2013 5:54 UTC (Wed) by dlang (✭ supporter ✭, #313)
[Link]
There are two layers of multicast.
There is IP based multicast (224.x IP addresses). This layer is managed by the routers knowing what downstream routers need copies of the traffic and sends it to all of them
There is ethernet link-layer multicast (a '1' in the least significant bit of the first byte of the MAC address, i.e. 01:00:00:00:00:00), traffic to these MAC addresses get handled by the ethernet switch.
These two can be used in combination with each other, but you can use the link-layer multicast with any IP address (and with no modification to the sending or recieving software), and I assume that you could use the IP based multicast without using the link-layer multicast, but I also wouldn't be surprised to learn that most of the time things default to using link-layer multicast if you are using the IP multicast range.
I also think that you will find that even the rather cheap switches handle link-layer multicast nowdays.
Link Layer Multicast
Posted Feb 20, 2013 17:59 UTC (Wed) by butlerm (subscriber, #13312)
[Link]
IP multicast (224.x.x.x) addresses have been hardwired to a range of Ethernet multicast addresses (01:5e:00:xx:xx:xx) for a very long time, and unfortunately overriding that mapping doesn't seem like it would accomplish anything, because any Ethernet switch is just going to broadcast anything it doesn't recognize anyway.
A practical example of this is where you have multicast IPTV traffic arriving from a ISP network into a home/office network. You generally have to filter that out and direct it over a separate network of some kind to every IPTV device, because if you just forward it directly onto the local subnet, inexpensive switches will broadcast everything to every port, which is a problem.
Does anyone really want to run a separate network to their set top boxes simply because link layer multicast is synonymous with link layer broadcast? It makes it difficult to watch television on ordinary desktops because they are connected to the wrong network, for example. Perhaps inexpensive Ethernet switches will implement IGMP snooping in the future for this reason. It isn't common yet though.
Link Layer Multicast
Posted Feb 21, 2013 4:03 UTC (Thu) by foom (subscriber, #14868)
[Link]
Well, at least it's possible to get IGMP snooping these days without paying a hugely excessive amount of money.
E.g. Netgear GS108 ($53) has no IGMP snooping, while Netgear GS108T does ($80).
Opera moves to WebKit and V8
Posted Feb 17, 2013 21:01 UTC (Sun) by ibukanov (subscriber, #3942)
[Link]
> your reference to blacklists
Once I talked to a person at Opera who had being involved into their networking stack. Their approach was to make sure that they could always restart connections in non-pipelining mode after the first sign of troubles. And troubles could mean unexpected timing in response, suspicions header content or their order etc. He claimed that after a couple of years they got enough information to make this works.
To apply such extensive blacklisting at many implementation levels to Mozilla's networking stack would require a very substantiate code rewrite.