Development

What's new in HTTP 2

A fourth draft of the in-development HTTP 2.0 protocol was published by the Internet Engineering Task Force (IETF) on July 8. The update to HTTP was first published in draft form in late 2012, but this revision is "marked for implementation," with interoperability tests slated to begin in August. HTTP 2.0 is largely a remapping of existing HTTP semantics that is designed to improve throughput of client-server connections. Consequently, it should not significantly affect the design or implementation of web applications. Browsers and web servers, however, can use HTTP 2.0 to replace many of the tricks they are already employing to decrease HTTP latency.

HTTP 2.0 is adapted largely from earlier work done on SPDY, an experimental extension to HTTP cooked up at Google, but supported by a number of other prominent software vendors (e.g., Mozilla). The newly-published IETF draft splits the changes up into three categories: HTTP Frames, HTTP Multiplexing, and a new "server push" feature. HTTP Frames are the message format for HTTP 2.0; they are designed to serialize HTTP requests, responses, and headers in a manner suited to faster processing. HTTP Multiplexing interleaves multiple requests and responses over a single HTTP connection. The server push feature allows servers to pre-emptively initiate the transfer of a resource to clients. The intent of the push functionality is that the server would know that a request for the resource is likely to come soon (such as another resource on the same page), so starting the transfer ahead of time makes better use of the available bandwidth and reduces perceived latency.

"Perceived latency" is an oft-recurring phrase in the new protocol; most if not all of the changes are tailored to allow servers to optimize delivery of web content to clients. Some of these changes mimic techniques already found in other protocols (such as TCP), which is perhaps what has prompted criticism of HTTP 2.0 from several bloggers and social media commenters. After all, TCP already has multiplexing and flow control, so it seems fair to ask whether implementing them in HTTP as well really adds anything. The presumed answer, of course, is that the web server is in the position to do application-level flow control and make quality-of-service (QoS) decisions while the underlying TCP stack is not. How effectively that works out in practice will presumably be seen once interoperability testing begins in earnest.

Frames and headers

HTTP 2.0 connections are initiated in the same way as they are in HTTP 1.x. The first difference comes in the enclosure of requests and responses in frames over the HTTP connection. Frames begin with an eight-byte binary header that lists the length of the frame payload, the frame type, any necessary flags, and a 31-bit HTTP stream identifier. A "stream" in HTTP 2.0 is one of the independent sequences that is multiplexed over the HTTP connection; in general each resource (HTML page, image, etc.) could get a separate stream, with the frames from all of the streams interleaved over the same HTTP connection. A stream can also be kept open and used to move subsequent requests and resources. However, the stream IDs are not reused over the connection; once a stream is closed, a new ID is created for the next stream. The IDs also reflect who initiated the stream: even numbers for server-initiated streams, odd for client-initiated.

The adoption of a binary header is certainly a difference from HTTP 1.x, even if the requests and responses that HTTP 2.0 frames encode are unchanged. But the differences are designed to decrease apparent latency. Marking the frame size at the beginning allows the connection endpoints to better manage multiplexing the connection. In addition, HTTP HEADERS frames can be compressed to reduce the payload size—a feature that was previously available only for HTTP resource data.

The frame types defined include the aforementioned HEADERS, the DATA frames carrying page resources, SETTINGS frames for exchanging connection parameters (such as the number of allowable streams per connection), RST_STREAM frames to terminate a stream, PING frames for measuring the round-trip time of the connection, WINDOW_UPDATE frames for implementing flow control, PRIORITY frames for marking a specific stream as important, and the new server push frames.

Server push uses two frame types; PUSH_PROMISE is a server-to-client frame that designates a stream ID that the server wants to push to the client. An uninterested client can reject the suggestion by sending a RST_STREAM frame, or accept the DATA frames that the server sends subsequently. For dealing with an overly pushy server, the other new frame type, GOAWAY, is provided. Clients can send this frame to a server to tell it to stop initiating new streams altogether. The nightmare scenario of mobile web users paying by the kilobyte, of course, is chatty servers pushing unwanted streams; the HTTP 2.0 specification mandates that a server must stop initiating streams when it is told to shut up via a GOAWAY frame.

Multiplexing

Intelligently managing the stream IDs on a connection is the essence of HTTP 2.0's multiplexing. Both the client and the server can create new streams, and can use their streams to send resource requests or to return data. By multiplexing several streams over one HTTP 2.0 connection, the idea is that several requests and responses can share the bandwidth, without setting up and tearing down a separate connection for each request/response pair. To a degree, bandwidth sharing is already available in HTTP 1.1's pipelining feature, but it has serious limitations.

For example, HTTP 1.1 requires that servers answer requests in the order that they arrive. In practice, browsers often open up multiple HTTP connections to a single server, which consumes additional memory on both sides, and gives the server multiple requests to handle at once. Naturally, the web application has other mechanisms available to decide that these connections are part of a single conversation, but by shuffling them into a single HTTP connection, the server has less guesswork to do and can do a better job of optimizing the connection. Furthermore, it can do this without maintaining a large collection of open ports and without spinning up separate processes to handle each connection.

QoS features are enabled by HTTP 2.0's WINDOW_UPDATE and PRIORITY frames. The client can increase the amount of data in flight by widening the flow-control window with WINDOW_UPDATE. Or, rather than telling the server to send more, the client can decrease the window to throttle back the connection. The server, in turn, can designate a stream ID with a PRIORITY frame, which allows it to send the most important resources first, even if the requests came in in a different order.

It is also noteworthy that the flow control options offered in HTTP 2.0 are defined to be hop-to-hop, rather than being between the server and browser endpoints. This enables intermediate nodes like web proxies to influence the throughput of the connection (presumably for the better).

Last, but not least, the HTTP 2.0 draft mandates one change to HTTPS support. Supporting endpoints are required to implement the Application Layer Protocol Negotiation Extension to TLS (TLSALPN), which is itself an in-progress IETF draft specification. TLSALPN is a mechanism for TLS clients and servers to quickly agree on an application-level protocol; mandating its presence means that HTTP 2.0–capable endpoints can agree on the use of HTTP 2.0 and begin taking advantage of its throughput from the very start of the conversation.

Apart from these new features, HTTP 2.0 introduces no changes to the established semantics of HTTP: the request methods, status codes, and headers from HTTP 1.1 remain the same. The stated goal is to provide backward compatibility; the hope being that server and browser implementations can provide an HTTP 2.0 encapsulation layer and begin offering throughput improvements without changing web pages or applications themselves. Both the Google Chromium and Mozilla Firefox teams have expressed their support for the protocol, as have several of the proprietary browser vendors. At this point, the big questions still to be answered are things like how HTTP 2.0 will affect WebSockets, which also implements multiplexing and QoS, but is usually considered part of HTML5.

Microsoft had pushed for a different approach to HTTP 2.0—reusing the framing, multiplexing, and encoding semantics of WebSockets—but at this stage is the standardization process, the draft resembles SPDY considerably more. There will no doubt continue to be other means for clients and servers to reduce the perceived latency of the web (for instance, by shuttling binary data over WebRTC's data channels), but as the fundamental layer of web connections, HTTP has more places to make an impact—not just browsers and servers, but proxies, gateways, and all of the other nodes along the route.

Comments (13 posted)

Brief items

Quotes of the week

— Philip Guenther, as cited by wahern.

— Miguel de Icaza

Comments (none posted)

LXDE-Qt Preview (LXDE blog)

The LXDE desktop environment project has a preview of its new Qt-based development branch (including a screen shot). It uses Qt 4, but there are plans to move to Qt 5.1 eventually. "To be honest, migrating to Qt will cause mild elevation of memory usage compared to the old Gtk+ 2 version. Don't jump to the conclusion too soon. Migrating to gtk+ 3 also causes similar increase of resource usage. Since gtk+ 2 is no longer supported by its developer and is now being deprecated, porting to Qt is not a bad idea at the moment."

Comments (26 posted)

Ganeti 2.7.0 available

Version 2.7.0 of the Ganeti VM cluster–management utility has been released. This version incorporates quite a few changes, such as automatic IP address allocation, support for arbitrary external storage, and opportunistic locking to speed up the creation of new instances. As the release notes, explain: "If the ``opportunistic_locking`` parameter is set the opcode will try to acquire as many locks as possible, but will not wait for any locks held by other opcodes. If not enough resources can be found to allocate the instance, the temporary error code :pyeval:`errors.ECODE_TEMP_NORES` is returned. The operation can be retried thereafter, with or without opportunistic locking."

Full Story (comments: none)

photographer.io source release

The creator of the photographer.io photo-sharing site has announced that the source for the site is now available under the MIT license. "On a more selfish note; I’m really excited to have something I feel is worth open sourcing. I’ve spent 2 months working nearly every evening to get the site to a stable point and I’m generally pleased with where it is (other than the dire test suite)."

Comments (none posted)

GNU Radio 3.7.0 released

Version 3.7.0 of the GNU Radio software-defined radio suite is out. "This is a major new release of GNU Radio, culminating a year and a half of side-by-side development with the new features added to the GNU Radio 3.6 API. With the significant restructuring that was occurring as part of the 3.7 development, at times this seemed like rebuilding a race car engine while still driving it."

Full Story (comments: none)

HTTP 2.0 draft available

A "marked for implementation" draft of the HTTP 2.0 protocol specification has been released. "The HTTP/2.0 encapsulation enables more efficient use of network resources and reduced perception of latency by allowing header field compression and multiple concurrent messages on the same connection. It also introduces unsolicited push of representations from servers to clients."

Comments (43 posted)

KDE considers a three-month release cycle

The KDE project is contemplating a proposal to move to three-month releases. The announcement of the proposal says: "Basically the idea is to cut testing time and compensate it by keeping master always in a 'releaseable' state, now that two major components are frozen it looks like it is a good time to get used to it." The resulting discussion looks to go on for a while; see also this thread where the openSUSE project registers its disapproval of the idea.

Comments (31 posted)

Xen 4.3 released

Version 4.3 of the Xen paravirtualization system is out. New features include ARM support (both 32- and 64-bit), use of upstream QEMU, NUMA affinity in the scheduler, and more; see the Xen 4.3 feature list and the release notes for details.

Full Story (comments: 4)

GNU Health 2.0 released

Version 2.0 of the GNU Health hospital information system is available. This version adds a number of new features, starting with an OS-independent installer, and including compatibility with Tryton 2.8. New modules add support for working with World Health Organization essential medicines, American Society of Anesthesiologists physical status classifications, and in-home health care.

Full Story (comments: none)

Newsletters and articles

Development newsletters from the past week

• Caml Weekly News (July 9)
• What's cooking in git.git (July 5)
• What's cooking in git.git (July 9)
• Haskell Weekly News (July 3)
• OpenStack Community Weekly Newsletter (July 5)
• Perl Weekly (July 8)
• PostgreSQL Weekly News (July 7)
• Ruby Weekly (July 4)
• This Week in Rust (July 6)
• Tor Weekly News (July 10)

Comments (none posted)

Harlan: A new GPU programming language (The H)

The H introduces a new Scheme-based programming language for GPUs. "Scheme serves as the basis because it has history at Indiana University, where some previous experience of building Scheme-based compilers has been gathered. [Erik] Holk has also gained GPU programming experience working with the Rust programming language; unlike Harlan, however, this language works much closer to the hardware. Holk reveals that the name Harlan comes from a mishearing of fried chicken icon Colonel Sanders' first name, Harland, and this association is also why all the file extensions for Harlan programs are .kfc."

Comments (57 posted)

A Year of the Linux Desktop (KDE.News)

KDE.News has an interview with Malcom Moore, who is the network manager for Westcliff High School for Girls Academy (WHSG) in the southeast of England. WHSG switched the students to Linux desktops (openSUSE 12.2 and KDE Plasma 4.10) a year ago, and Moore talks about the motivations for the switch, the transition, the reactions to Linux and Plasma, hurdles that were overcome, and so on. By the sounds, the conversion from Windows has been quite successful, though not without its challenges. "The Senior Leadership Team grilled me in two long meetings which was fun! Once you actually take a step back from the misconception that computers = Windows and actually seriously think about it, the pros clearly outweigh the cons. The world is changing very quickly. There is a survey that reports in 2000, 97% of computing devices had Windows installed, but now with tablets and phones, etc., Windows is only on 20% of computing devices, and in the world of big iron, Linux reigns supreme. We specialize in science and engineering and want our students to go on to do great things like start the next Google or collapse the universe at CERN. In those environments, they will certainly need to know Linux."

Comments (87 posted)

Page editor: Nathan Willis
Next page: Announcements>>