Moving past TCP in the data center, part 1

In my experience, HTTP is mostly used between the frontend, the load balancer, and the end user (who is not actually in the data center, one assumes). This is more than it sounds because the "frontend" may be comprised of many interdependent microservices rather than just being one giant monolith, so you could have more datacenter HTTP than you might expect (at least, for folks going the microservice route, anyway). But backend stuff is generally RPCs or RPC-like-messages, because:

1. HTTP is overkill. For many services, you don't need e.g. If-Modified-Since, Cache-Control, etc., and if you do want that sort of thing, you probably want to fine-tune exactly how it works rather than having the HTTP library try to handle it.
2. HTTP is underkill. While it is technically possible for the client to present an X.509 certificate to the server to establish its identity, these certificates are (usually) designed to be long-lived and signed by a CA (which could be your own private CA). If you want to get into the business of minting your own bearer tokens, doing so in a way that is performant and not hilariously insecure or SPoF'd on the CA starts looking pretty dicey. Sometimes, you might also want to present multiple credentials for securing different aspects of the request (e.g. "this token proves I am the XYZ service, and this token proves that user 1234 has authorized me to access or manipulate their PII"), and doing that over HTTP is fiddly at best.
3. HTTP is text-based, so you have to finagle all of your data into text (or something reasonably resembling text, like JSON), and then unfinagle it at the other end. For every RPC. This ends up being ludicrously expensive when you consider the sheer scale that we're talking about here.