LWN: Comments on "Layers and abstractions"

Layers and abstractions

riking — Sun, 24 Mar 2019 02:24:27 +0000

This has been on my mind recently as well. The way I summarized it was thus: "Adding layers [of abstraction] improves velocity [of development]; punching through layers improves performance [of runtime]".

I think my version is a little more violent 🤔

Layers and abstractions

perennialmind — Sun, 24 Mar 2019 02:10:02 +0000

The vast majority of web clients have been upgraded for IPv6, but most of them have lousy IPv6 access. IPv6 required upgrades along the entire path to be useful. Even then, anything short of full adoption makes use risky. For a decade, I tried to join in the 128-bit fun: tunnelbroker.net, 6to4, teredo (once I had the twin IPs for a relay), and finally a /48 of ~~my own~~ my company's from an ISP happy to provide it but bemused that I cared. And still, I have to grit my teeth and turn off IPv6 when some website we care about turns on IPv6 and the road gets all bumpy. "Happy Eyeballs" is all about compensating for the tendency of IPv6 to be so very bumpy. It's not the hosts at either end: it's the unloved links inbetween.

A larger address space in the 90's was primarily useful to client networks that were already resorting to NAT. That's where the incentive was, so start by addressing their problems. In the first phase, you don't run public servers beyond the IPv4 edge – I agree with you there! That happens once (1) there are few legacy IPv4 clients left and (2) all of your server software has support for the address extension. You don't disable the extended-address support on your IPv4.1 OS, because it's so very harmless. It's just one more upgrade like any other. No need for AAAA records at that point. 32-bit IPng destination addresses are normal. 🙃

It took so long before userspace had decent support for IPv6. Presumably because it was largely irrelevant. Once you have servers automatically upgrading connections to IPng, it becomes relevant! Upgrade your client OS and server OS, but keep the routers and userspace on IPv4 and still get bigger addresses on the wire? I have to think adoption would have been quicker!

Start small. Build up incrementlly. Let your chick grow up and you wind up with chicken and egg. 😊

Layers and abstractions

zlynx — Sun, 24 Mar 2019 00:16:40 +0000

IPv6 servers are backward compatible in the same way as some IPng scheme. Data centers that care (and this is the key point) are giving out IPv6 addresses to web servers and also providing an IPv4 reverse proxy. Every server gets a AAAA DNS record to their actual IPv6 address and an A record to the reverse proxy.

But there's still a bunch of people who don't see the point and think it's just fine to use 10.x.x.x addresses in the data center and only rely on proxies. Forever. But those people would be a problem in any IPv4 backwards compatibility scheme for any new addressing. They'd never see the point in upgrading past the compatibility solution.

Layers and abstractions

Cyberax — Sat, 23 Mar 2019 06:53:05 +0000

> NAT is bound to be the fallback in any scheme.
Whatever system you end up will not be functionally different from dual stack IPv6 + IPv4 NAT.

This is what is happening right now - you still have legacy IPv4 infrastructure with NAT-ed clients and newer IPv6 infrastructure without NATs. They both live together side-by-side seamlessly, thanks to Happy Eyeballs RFC.

> But even if you drop the stateful NAT option, under this scheme, or a 6to4-only-IPv6 phase, clients with mere IPv4 access can talk to the full IPng internet. For servers, there's no good reason not to opt-in to IPng.
No they can not. IPv4 clients can only connect to IPv4 addresses, so the server will have to use IPv4. Peer-to-peer with IPng is still impossible.

Layers and abstractions

perennialmind — Sat, 23 Mar 2019 04:29:06 +0000

NAT is bound to be the fallback in any scheme. If you start there and negotiate up, you can have a smooth on-ramp. Under the options-can-work scheme, the initial packet in any given flow exits the border router with it's address information intact. That packet is effectively "dual stack". (A heisenpacket?) It's only when it lands at an unenlightened IPv4 endpoint that you are stuck with stateful nat. Stateless transformations I don't mind.

But even if you drop the stateful NAT option, under this scheme, or a 6to4-only-IPv6 phase, clients with mere IPv4 access can talk to the full IPng internet. For servers, there's no good reason not to opt-in to IPng.

I do agree that IPng-blind IPv4 hosts could only talk to IPv4 servers. But that just makes sense, doesn't it? If I fired up Netwcape 4, I'd be able to load webpages over SSL3, but not those that only support TLS 1.2+. I'd be able to load crusty.gif, but not snazzy.svg. The web has undergone gradual transitions, with ample consideration for legacy systems. IPv6 didn't do it that way.

Layers and abstractions

smitty_one_each — Fri, 22 Mar 2019 23:11:44 +0000

Jake, I preferred the skimmability of your summary to the video, for reasons I can't pin down. Thank you.

Layers and abstractions

smitty_one_each — Fri, 22 Mar 2019 23:04:05 +0000

> That's the bar. Any new routing protocol has to be better than NAT.

(1) My observation is that technology adoption is driven by how much better the new is than the old.

(2) If IPv6 were that much of an improvement over v4, then Egress-Only Ingernet Gateways[1] would make no sense.

(3) Maybe CIDR is "good enough".

[1] https://docs.aws.amazon.com/vpc/latest/userguide/egress-o...

Layers and abstractions

Cyberax — Fri, 22 Mar 2019 20:56:49 +0000

The problem is, IPv4 clients can't talk to IPng nodes without what amounts to NAT64. So even IPng-capable servers will be forced to remain on IPv4.

This makes everything even worse, a dual stack IPv4/ng client won't know which protocol to use to connect to a V4 address.

Layers and abstractions

perennialmind — Fri, 22 Mar 2019 18:30:40 +0000

An IPv4 option field won't work simply because there are too many middleboxes that mangle or discard unknown option fields or entire packets containing them.

Oh there's no chance of such a scheme working now! The fun thought experiment supposes an alternative history diverging in the early 90's. Maybe it was already too late then. I'm awfully curious on that point. :-) As I understand it, unrecognized options are supposed to be ignored per RFC1122 and RFC7126. So a conformant IPv4 router should not need replacing or upgrading. In theory. 😊

IPng-to-IPng-over-link is obviously native IPv6.

Yup.

IPng-to-IPng-over-IPv4 is covered by 6to4, Teredo, 6rd, and 6in4.

Covered, yes. Miserably. The 6to4 scheme would have worked beautifully, if all the other IPv6 addresses were also 6to4. 6rd trades the anycast address for a unicast address provided by the ISP. 6rd is a mechanism for an ISP to provide IPv6 access over it's IPv4 infrastructure - a worthy goal, but a much less ambitious one. Teredo avoids upgrades to the NAT gateways but would only work well if IPv6 network operators consistently provide Teredo relays. Due to that, the complex path, added latency and limited reliability, it never reached critical mass.

... any protocol which relies on embedded IPv4 addresses will be unusable since (a) one side doesn't have a public IPv4 address to embed...

The fact that the IPv6 address space doesn't extend IPv4 is precisely the problem. I still don't understand why they did that. They put IPv4 addresses at the bottom (::127.0.0.1); if they'd put them at the top, the problem you describe wouldn't exist.

and (b) there is no room to store an extended address

If you can't use the extension mechanism built into the original design (options), then the best you can do is encapsulation a la 6to4. 6to4 without the anycast route would still have been a huge improvement over NAT44. If you're referring to the sockaddr_storage type, yes, that would still be a painful, costly effort. Preferably one resulting in a sane id/locator split. Maybe replace sockaddr_in with a "locator handle". <shrug>

Hey, I just think it's a fun counterfactual! 😊

Layers and abstractions

nybble41 — Fri, 22 Mar 2019 16:38:16 +0000

An IPv4 option field won't work simply because there are too many middleboxes that mangle or discard unknown option fields or entire packets containing them. Rather than replace all these routers with ones that handle the new option field properly, you might as well just replace them with ones that can handle the new protocol (IPv6) natively, or set up a tunnel.

IPng-to-IPng-over-link is obviously native IPv6.

IPng-to-IPng-over-IPv4 is covered by 6to4, Teredo, 6rd, and 6in4.

IPng-to-IPv4, without the option field, is NAT64. DNS translation is also mandatory, working in concert with the NAT64 gateway, if you intend to allow IPv4-only hosts to establish connections to "IPng" servers. Like *any* simplistic address extension scheme—including NAT44 and CGNAT—it has numerous shortcomings due to the fact that one side of the link is blissfully unaware of the need for address extensions. For a start, any protocol which relies on embedded IPv4 addresses will be unusable since (a) one side doesn't have a public IPv4 address to embed, and (b) there is no room to store an extended address even if the NAT64 gateway were modified to intervene and translate addresses at the application layer. The best you can hope for in that case is a new "IPng"-compatible protocol with space for address extensions and an application-level proxy running on the gateway to perform the translation.

Layers and abstractions

perennialmind — Fri, 22 Mar 2019 14:50:13 +0000

I don't see that. I see that NAT has been and still is the real-world solution to the address crunch. That's the bar. Any new routing protocol has to be better than NAT.

For an IPng upgrade to beat NAT44, you'd want IPng-to-IPng-over-link, IPng-to-IPng-over-IPv4, and IPng-to-IPv4. The first one is easy*. The last two need to be compatible with IPv4-only routers in the path.

IPng-to-IPng-over-IPv4 can use encapsulation, so that can also be easy. With encapsulation, the layer underneath changes from link-to-IPv4, but that's normal - boring, even.

IPng-to-IPv4 is where, if you want to beat NAT, you have to embrace NAT. Upgradable NAT. The NAT we have today is expensive, brittle and lossy. To support true legacy IPv4 nodes, it has to be. If you set aside that requirement, you can have fully stateless NAT with address extension by IPv4 option field. If you need to support legacy IPv4, start with stateful NAT but include the address extension option. Then you can tear down the connection tracking and port reservation after the first reply in the flow. TCP would work especially well: only OS upgrades would be strictly required for an IPv4 TCP server. An upgraded TCP/UDP kernel+userspace would have full connectivity with the IPng world, without any help from an ISP.

Of course, if you decide to do IPng-to-IPv4 that way, you might as well take the same approach with IPng-to-IPng-over-IPv4. Once you take a translation approach, you can splice disjoint routing domains together. It's an inter-net after all! 😁

* If you adopt CLNP instead of inventing a brand-new "quick" "simple" design based on IPv4.

Layers and abstractions

perennialmind — Fri, 22 Mar 2019 14:21:36 +0000

A workable encapsulation strategy looks like a tunnel in that you have a layer 4 protocol over another layer 4 protocol. However, a tunnel abstraction usually implies a virtual interface for each "link". For an overlay network, while you can pretend the substrate is a link layer rather than another routing layer, at that point you are fighting your abstraction rather than fixing the discordance. In an more ideal world, abstractions evolve the way a model evolves to better fit the reality it describes.

Look at filesystems. Filesystems take a block device and provide files and directories. Except maybe they sit atop RAM, like tmpfs. Or other filesystems, like overlayfs or nfs. Or maybe they sit atop a single file... except they don't unless you construct an illusory loopback block device. That is, until Dave Chinner's new mapping layer teases apart the block address space concept from the block device layer. That is exactly the kind of well-defined "leak" we want between layers.

But I don't think that IPng-to-IPng-over-IPv4 should have used encapsulation. I think translation is a better approach. I think that using the extension facility built in to IPv4 should have been used. I think that new options fields should have been blessed as backwards-compatible first-class headers and a new minor version, IPv4.1 created. If IPng had benefits other than bigger addresses, then perhaps it could creep in from the edge. But if not, that's fine too.

There are other options, but the key enabler is the co-rooted address space. 6to4 worked great, except for the anycast bit, which was god-awful, a black hole of failure. If the new address space had embedded IPv4 near the top and refused to hand out anything not under it, we'd still have needed all of the host-side software changes, DNS, and routing protocol upgrades, etc, but the parties with no incentive to make costly changes would not have had to make costly changes.

To your last point: I am very glad that the entire operation of the internet does not depend on an IPSec substrate.

Layers and abstractions

zlynx — Thu, 21 Mar 2019 19:13:13 +0000

You know that using IPv4 as a compatibility layer for IPv6 is just tunneling, right?

There have been tunnel solutions ever since IPv6 started. 6to4, Teredo, 6rd. And NAT64, 646XLAT, etc.

There's just not much advantage to it unless it's used natively and that's the hold-up.

Really, there's a ton of slow and stupid solutions if all we wanted were more IP addresses. My favorite proposal I heard someone make was automatic IPSEC tunnels and new DNS records to get the second layer of IP once you had a tunnel up. Because that way you got the encryption *and* the tunnel.

Layers and abstractions

Cyberax — Thu, 21 Mar 2019 18:59:23 +0000

I'm sorry, but DJB has no idea what he's speaking about. There's no way to create a backwards-compatible IPv6, attempting to do automatic V4-to-V6 translation (that he's promoting) makes migration even worse.

Layers and abstractions

perennialmind — Thu, 21 Mar 2019 18:04:02 +0000

One quibble: I don't think it was the network-layer-scope limitation that hamstrung IPv6. I was just re-reading DJB's old take. He did an excellent job calling out the problems with attempting to institute a replacement without making compatibility a top priority. QUIC, soon to be HTTP3, works so well because it works with the existing infrastructure instead of taking a hostile "We will bury you!" attitude. Imaging an alternative history in which the IETF adopted TUBA and kept the IPv4 address space at the root makes for a fun exercise.

Layers and abstractions

mm7323 — Thu, 21 Mar 2019 11:47:18 +0000

QUIC was bound by history and the fact that TCP/IP and UDP/IP are still the only real choices for Internet spanning protocols - IPv6 uptake being a demonstration of the difficulties and slow progress in trying to change the network layer alone.

Layers and abstractions

nim-nim — Thu, 21 Mar 2019 08:33:57 +0000

IMHO the reality is more that humans can only cope with a fixed amount of complexity (statistically; of course you have outliers).

To do new things you need to slice up the new problem space in layers, to keep the complexity per layer to a manageable amount. But, managing layers is itself a complexity source.

So, the cost of doing new things, is to simplify the existing layer pile, to free up the complexity budget required to handle the new layers. And simplifying mostly means giving up on some layer combinations that seemed to have potential in the past but proved not worth the layering cost.

You see this a lot in human languages. All human languages are different. Pretty much any human language will be simpler and more regular than others in one aspect. But this simplification usually freed up a complexity budget, that humans used to make the language less simple than others on some other aspect.