Layers and abstractions

In software, we tend to build abstraction layers. But, at times, those layers get in the way, so we squash them. In a talk at SCALE 17x in Pasadena, CA, Kyle Anderson surveyed some of the layers that we have built and squashed along the way. He also looked at some of the layers that are being created today with an eye toward where, how, and why they might get squashed moving forward.

When he thinks about layers, he thinks about abstractions and the separation of concerns principle. Those two are "kind of the same to me", he said. To set the stage, he put up some quotes about abstraction from computer scientists, which can be seen in the YouTube video of the talk. He also mentioned Rich Hickey's "Simple Made Easy" talk, which Anderson said was "kind of the opposite" of his talk, so he encouraged attendees to watch it as a counterpoint.

Squashing layers

The first of the layer squashes he wanted to talk about was DevOps. He said he started with that because it is more or less non-controversial; since 2009, the software industry has agreed that DevOps is a good thing. DevOps came about as a reaction to the traditional thinking that the responsibilities for a deployed software stack should be broken up into a bunch of layers, each of which had its own person or group responsible for it. So deployment, build/release, QA, code and unit tests, task decomposition, UI wireframes, and features were each owned by their own entity (e.g. product manager for features, developers for the code and unit tests, operations for deployment).

That all looks nice on the slide, he said, but whenever he shows something like that, he would like the audience to ask: "what if we got rid of some of the layers?" That's what happened with DevOps: the developers became responsible for most of that stack (other than features and UI wireframes). Developers are responsible for breaking things up into tasks, writing code and tests, doing QA, making sure it all builds, and deploying the result into production.

What are the tradeoffs involved with squashing those layers? He works at Yelp, which uses the DevOps model. The company has found that DevOps leads to increased performance, at least in terms of the number of deployments per day. Deployments are faster, as are rollbacks, which gives a faster "time to recovery". It is somewhat harder to hire for, however, as some developers are not interested in handling production and deployment. There is also more inter-team communication, which adds overhead. Overall, though, Yelp, like much of the rest of the industry, has found DevOps to be a beneficial change.

The next layers he described were those for filesystems. In a traditional Linux filesystem stack, there are neat layers starting with the block device at the bottom, RAID and volume-management above that, dm-crypt for encryption, an ext4 or other traditional filesystem layer, and finally the files themselves available at a mount point. As instructed, the audience said: "what if we got rid of some of the layers?"

If you did so, that would look a lot like ZFS, which is a non-traditional filesystem that has squashed those layers into the filesystem itself. ZFS handles RAID, encryption, pools for volume management, and so on. There clearly is no separation of concerns for ZFS. Is that a good thing? It does lead to better error detection and recovery because ZFS is concerned with the block layer all the way up to the files themselves; it has checksums at each level. ZFS can also do deduplication more easily and take faster snapshots.

Sympathetic abstraction

To explain what is going on with ZFS, Anderson introduced the idea of a "sympathetic abstraction". It can be thought of as an "intentional leakiness" where layers are sympathetic to those above and below. For example, ZFS is sympathetic to whether the block device it runs on is spinning rust or an SSD. ZFS is also sympathetic to errors that can occur on the raw block device, while ext4 and other traditional filesystems are not: they leave that kind of error handling to some other layer below.

These kinds of sympathetic abstractions happen all the time, he said. For example, SSH sets the TCP_NODELAY option (to disable Nagle's algorithm) for interactive sessions. But SSH should not care what transport it is running on; it should be able to run on top of UDP tunneled inside DNS packets. The Apache Cassandra database wants to know if it is running on spinning rust or SSD; shouldn't that be the concern of layers below it?

He introduced "Kyle's Abstraction Law", which he has come up with, though it is "not canon", he cautioned. It says that over time, layers tend toward becoming more sympathetic (or merged) in the pursuit of performance. We start off with "beautiful layers", each with a single purpose; over time, we realize the picture is not quite so clear cut, so we squash the layers some.

Next up was HTTP and HTTPS. In order to talk about HTTPS in terms of layers, he wanted to start with the OSI seven-layer model for networking. It came out in 1984 and when he learned about it, it made a lot of sense; it was a "great way to think about how we build our networks".

So he showed how HTTPS "fit" into the OSI model. On his laptop, both the physical and link layers (layers 1 and 2) are 802.11, which already shows some layer squashing. The network and transport layers (3 and 4) are IP and TCP, but then things (further) break down. Layer 5 is the session layer, but HTTP is "stateless", so that doesn't seem to apply. HTTPS runs atop SSL/TLS, but where does that encryption go? As layer 6, since there is no other place to put it? That puts HTTP in layer 7.

That all looks a bit weird, he said. But it is because the IETF, which oversees the internet protocols, does not intend them to conform to the OSI model. It turns out that there are several layer models out there, with differing numbers of layers. So he thought the OSI model was the territory, but it was really just one of the maps of it.

What if we got rid of some of the layers? We have, he said, and it is called QUIC. It sits atop UDP, rather than TCP, and handles everything from there, including encryption, multiplexing, error correction, and so on. There is no separation of concerns at that level, but you can't argue with the performance. It also means that you can't use tcpdump to look at the traffic and you can't simply swap out the crypto implementation with something else—you have to have faith in the QUIC crypto. As he noted, if you are watching a video on YouTube or checking Gmail on your phone, you are almost certainly using it now.

Counterexample

Services and microservices were up next. If you were to start writing Twitter today, you might start with a Ruby on Rails application with a clean separation between the model, view, and controller . Add a MySQL database for storage and that might be some early version of Twitter.

In a change from the usual pattern, he asked with a chuckle: "what if we added some more layers?" His slide showed a bunch of microservices, each with their own storage, and an API gateway that routed various requests to them. This is a "service-oriented architecture" that is what Twitter has today.

Each of those microservices may talk with the others, leading to a complicated "death star" call graph, but it is useful for scaling, both in terms of the components and in terms of development. Each of the services can be scaled up or down depending on need and each can be developed and deployed by independent teams. It also has a cost, of course, but he suggested that attendees keep this counterexample in mind through the rest of the talk.

Kubernetes

He then moved on to Kubernetes. Ten years ago, a deployment of an application in the Amazon cloud had just a few layers. There was the bare metal layer that Amazon controlled and one could pick a virtual machine (VM) size to run on top of that. That VM would run the operating system (Ubuntu, say) and the application (perhaps in a Java JAR file), which would both be built into an Amazon Machine Image (AMI) file. That was four layers.

"So what if we added a bunch more layers to this?" For Kubernetes, the stack starts the same if it is deployed in the Amazon cloud; there is the bare metal and infrastructure-as-a-service (IaaS) VM. On top of that is still the operating system, but after that is where things diverge. For Kubernetes, there will be a kubelet layer and, on top of that, a pod layer. Then there are one or more containers running in the pod and, finally, the Java JAR running inside one of the containers.

"That's a lot of layers", he said. Are we just "architecture astronauts" that want to build something with lots of layers, with a clean separation of concerns? It is hard to argue against the momentum that Kubernetes has, but he does wonder: what if we got rid of some the layers?

Amazon and other cloud providers offer many different-sized VMs as a way for their users to scale their applications, but the Kubernetes world scales in a different way. So he thinks the future is to eliminate the VMs and simply run Kubernetes on the bare metal. That would squash the lowest two layers.

You could also squash all of the layers above by moving to a unikernel model, but he does not think that is necessarily the way forward. Unikernels remove many of the benefits that we get from containers, including speed of deployment and ease of iteration. In addition, unikernels are more difficult to work with, since there is no real OS that one can SSH into, for example.

Another path would be to move to a "serverless" model, though he prefers the term "function as a service" (FaaS). In that model, the hardware layer would encompass the bare metal (and VM if needed) and the next layer would provide the Amazon Kubernetes pieces (OS, kubelet, and pod). Above that would be the container layer; for Amazon's Lambda FaaS offering, that would be Firecracker, but for Kubernetes it would be Docker or a similar container runtime. Layer 4 would be the application code.

Application focus

This allows developers to focus on their code, rather than having to keep in mind all of the substrate that makes their code run. In the unikernel world, all of that infrastructure is essentially part of the application, so developers have to be aware of it. But, isn't this new model right back to where we started? Prior to Kubernetes, there were four layers and the developer really mostly needed to be concerned with their application. He believes that there are three main reasons why this architecture based on FaaS and Kubernetes is going to be beneficial for the industry.

The first is that lack of concern for the rest of the stack. Ten years ago, developers would need to build an AMI, which is "slow to bake", to iterate on their code. Troubleshooting required making an SSH connection into the instance and changing the size of the amount of CPU to be used required picking a different size of VM at the cloud provider. These days, Docker and other containerization tools have matured so that it is easy and fast to iterate on code changes. Scaling is handled at a higher level, as well, so changes do not require an entirely new VM.

The second reason is that routing and dispatch can be handled better with sympathetic abstractions. In the traditional cloud deployment, an Amazon auto scaling group would be responsible for ensuring that N copies of a service (e.g. Apache) are running. There is a different entity, the elastic load balancer, that is responsible for routing requests to these different service instances. In the "new world", the request routing and compute dispatch are combined.

Because the API gateway is sympathetic to the kinds of services being run, it can scale up or down as needed. The old way calculated the need for more service instances from CPU usage, but is not cognizant of the differences between, say, Java and Go applications. Kubernetes allows that recognition because it has squashed the routing and dispatch layers together. That should provide a lot better performance and many features that are hard or impossible using the old mechanisms, Anderson said.

The third reason is that Kubernetes operators provide a way to do compute scheduling in a sympathetic manner. While an AWS auto scaling group can bring up a new instance of MySQL, it has no conception of the fact that the new instance needs to join the MySQL cluster and to start replication. Users need to add tooling to make sure that it doesn't get added to the load balancer before it is actually ready to serve queries. Due to separation of concerns, the auto scaling group only cares about keeping N copies running, the rest is someone's else responsibility. It can be made to work, but it is difficult to do.

But by using an operator, Kubernetes can be aware of the needs of new MySQL instances. The operator is concerned with the health of the MySQL cluster; it is sympathetic to the workload. At this point, operators for specific applications are not generally production ready, but they are getting there. Until the advent of operators, there was no layer that could handle "this application-specific knowhow", he said. It is the first time in the software industry where there is a place to specify a "really tight, application-specific control loop".

Anderson concluded his talk by noting that he believes sympathetic layers are a good thing, in general. QUIC and ZFS provide two good examples of that; sympathetic layers provide better performance, which comes at a price, but he believes the cost is justified in those cases. On the other hand, separating out layers, such as running Docker containers on Kubernetes versus AMIs on VMs, can also provide more capabilities.

Layers are not good or bad, per se, but depend on how they are used, what they cost, and so on. It is up to us as engineers to look at layers (or their lack) and to ask what would happen if they were merged (or added). He also thinks that Kubernetes operators and FaaS are "re-slicing where the layers are", which is to the good. Merging dispatch and routing brings a lot of new capabilities, but means that traditional load balancers can no longer be used.

He reiterated that attendees should watch "Simple Made Easy" before making up their minds. It is a great talk that will help guide thinking about when it makes sense to "'complect' things together or split them apart". He suggested that attendees consider both talks and then come to their own conclusions.

[I would like to thank LWN's travel sponsor, the Linux Foundation, for travel assistance to Pasadena for SCALE.]