LWN: Comments on "TCP Fast Open: expediting web services"

Examples for the speed of light

Lennie — Fri, 12 Oct 2012 12:04:19 +0000

You can see the routes here:

http://www.cablemap.info/

Examples for the speed of light

ncm — Tue, 02 Oct 2012 08:17:41 +0000

Do fibers follow great-circle routes already? I expected that to take longer to happen.

Examples for the speed of light

paulproteus — Fri, 10 Aug 2012 01:30:47 +0000

Interesting point!

http://blog.advaoptical.com/speed-light-fiber-first-build... suggests that one should expect approximately a 33% increase in these times for the fiber optics.

(Additionally, they should be doubled due to round-trip time, as per my follow-up comment.)

Examples for the speed of light (Correction)

paulproteus — Fri, 10 Aug 2012 01:28:44 +0000

One correction to the above note: One should *double* these numbers for *round*-trip time. These are one-way times.

Examples for the speed of light

dlang — Fri, 10 Aug 2012 01:27:11 +0000

it's actually a bit longer than these times as the speed of light you are listing is the speed of light through a vacuum, going through fiber is noticeably slower.

Examples for the speed of light

paulproteus — Fri, 10 Aug 2012 00:28:22 +0000

The article says:

At intercontinental distances, this physical limitation means that—leaving aside router latencies—transmission through the medium alone requires several milliseconds

To be more concrete:

1 mile is 5 microseconds
6 milliseconds from New York to Florida (1152 miles)
15 milliseconds from New York to San Francisco (2917 miles)
36 milliseconds from New York to Tokyo (6735 miles)

Source for time conversion: GNU Units.

You have: 1 mile
You want: light second
* 5.3681938e-06
/ 186282.4

TCP Fast Open: expediting web services

jthill — Thu, 09 Aug 2012 13:48:32 +0000

I think just allowing a server to detect whether the connection is still in fast-open state should be enough. If the server sees a non-idempotent request it can make sure the initial ACK has arrived before proceeding. Make it detectable by say making {aio_,}fsync not complete until it's arrived, or fabricating a redundant EPOLLOUT edge.

TCP Fast Open: expediting web services

cvrebert — Thu, 09 Aug 2012 06:11:06 +0000

You're gonna have to wait until 2208 for that.

TCP Fast Open: expediting web services

kevinm — Wed, 08 Aug 2012 14:45:06 +0000

...and you could request fastopen by setting TCP_CORK before connect(); write(); then uncorking.

TCP Fast Open: expediting web services

johill — Wed, 08 Aug 2012 08:02:23 +0000

I don't think it would affect the effectiveness a lot -- presumably the backend server and LB are close by each other, so the latency between them matters less than the latency between the LB & client.

TCP Fast Open: expediting web services

raven667 — Wed, 08 Aug 2012 03:41:30 +0000

Doing so would presumably add latency and reduce the effectiveness of fast open. It would make more sense for the LB to just do NAT rather than proxying the connection. Is there any special handling in conntrack needed for this?

TCP Fast Open: expediting web services

butlerm — Wed, 08 Aug 2012 00:05:52 +0000

That depends on the design of the load balancer. A well designed one should have no problem converting a TCP Fast Open connection between the client and the LB and a standard connection between the LB and the servers behind it. Assuming the servers and the load balancer(s) are colocated, there should be very little penalty for doing so.

TCP Fast Open: expediting web services

drdabbles — Sat, 04 Aug 2012 23:16:55 +0000

How do you perceive this working with load balancing hardware in the future? Vendors will have to get behind TFO and patch firmwares, but hardware behind the LBs will also need to be patched. Do you expect a chicken-and-the-egg situation here, or do you know something that perhaps you aren't or can't share with us?

TCP Fast Open: expediting web services

drdabbles — Sat, 04 Aug 2012 23:12:41 +0000

This may be true in theory (I'm not sure), but in practice it's completely wrong. Bandwidth tells you only how much data can be passed through a link in a given time period. Saying a link is capable of 1Gbit/sec means if you consume every possible bit for every possible cycle for 1 second, you'll have transferred 1Gbit of data over the wire. Many links have a frame/second limit, so if your frames aren't completely full, you've wasted bandwidth and decreased the utilization of the link.

Router latency is caused by many factors. Some can be router CPU shortages, memory resource shortages, the time it takes to transfer a frame from "the wire" to the internal hardware and vice verse, how quickly a packet can be processed, whether packet inspection is happening, etc. This, relatively speaking, can be a very long time. Typically it's microseconds, but certainly not always. Either way, it represents a minimum time delay with only a practical ceiling (IP timeout / retransmission). So increasing bandwidth to an already slow router only makes the problem worse.

Also, if you have a link that passes through 1000 routers, it's bound to hit a dozen that are oversubscribed and performing horribly. This is especially true as your distance from the "core" increases and your distance to the "edge" decreases. This is why major datacenters are next to or house the major peering points of the Internet.

TCP Fast Open: expediting web services

Jannes — Sat, 04 Aug 2012 13:34:46 +0000

actually TFO should be a huge improvement in a bufferbloated situation. If there is 1 second of bufferbloat, then 'Saving one RTT' means saving 1 second.

Not saying it's a solution to bufferbloat of course.

TCP Fast Open: expediting web services

nix — Sat, 04 Aug 2012 12:20:34 +0000

Warning: work in this area can be dangerous. See e.g. the series, ahem I mean feasibility study, bookended by <http://www.amazon.com/The-Collapsium-Wil-McCarthy/dp/0553...>, <http://www.amazon.com/To-Crush-Moon-Wil-McCarthy/dp/05535...>.

(Though admittedly the Queendom did go to rather more extreme lengths to increase the speed of light than mere vacuum tubes, and displayed a cavalier degree of carelessness, indeed insouciance, regarding the fact that keeping trillions of black holes in your solar system in very large arrays moving at well below orbital velocity is insanely dangerous.)

TCP Fast Open: expediting web services

dlang — Fri, 03 Aug 2012 21:16:40 +0000

That depends on how you define REST

many groups make everything a GET request, especially for APIs that are not expected to be used from browsers, but rather called from other applications, especially in B2B type situations.

TCP Fast Open: expediting web services

epa — Fri, 03 Aug 2012 15:35:19 +0000

I thought that part of making a RESTful web service was deciding which operations are read-only and which affect the state; and splitting them into GET and POST requests accordingly.

TCP Fast Open: expediting web services

cesarb — Fri, 03 Aug 2012 14:04:48 +0000

> Surely you could get around that by including the servers boot time in the MAC key used to generate the cookie?

A better option would be /proc/sys/kernel/random/boot_id (see http://0pointer.de/blog/projects/ids.html).

TCP Fast Open: expediting web services

ras — Fri, 03 Aug 2012 04:36:51 +0000

> Being unaware of the reboot, the client will timeout and retransmit SYNs.

For that to happen the server must accept the cookie. Surely you could get around that by including the servers boot time in the MAC key used to generate the cookie?

TCP Fast Open: expediting web services

butlerm — Fri, 03 Aug 2012 02:22:31 +0000

Yes, both client and server need support for TCP Fast Open. That support amounts to a change to both the TCP stack and the TCP socket API, both of which are implemented by the kernel. Without kernel support (or a user space TCP implementation and the privileges necessary to use it) neither endpoint can make use of TFO.

TCP Fast Open: expediting web services

butlerm — Fri, 03 Aug 2012 02:17:19 +0000

If I am not mistaken, most modern TCP stacks do not hold data sent with a SYN, and do not send it either. For most applications, there would be relatively little advantage if they did. Requests usually fit in an MTU (or MSS) worth of data, and in the absence of something like TCP Fast Open, the target endpoint has to wait for an acknowledgement that can carry the full sub-MSS sized request without a problem. Where on the other hand, holding the data simply makes it easier to conduct SYN attacks.

TCP Fast Open: expediting web services

butlerm — Fri, 03 Aug 2012 02:02:51 +0000

The normal technique is to mark all non-idempotent requests with a unique id, and keep track of which ones have already been processed. That is practically the only way to get once-only execution semantics.

This could presumably be done (up to a point) at the transport layer with TCP Fast Open by having the initiating endpoint assign a unique identifier to a given user space connection request, attaching the identifier as a TCP option, caching the identifier for some reasonable period on the target endpoint, and throwing away SYN packets with connection identifiers that have already been satisfied. The more general way to do that of course is to do it at the application layer, in addition to anything the transport layer may or may not do.

TCP Fast Open: expediting web services

Lennie — Thu, 02 Aug 2012 23:44:43 +0000

So when will we see companies building vacuum tubes used to speed up the light when crossing large parts of land or maybe the atlantic ?

That is the thing I care about. ;-)

Is anyone doing research on that yet ?

TCP Fast Open: expediting web services

Lennie — Thu, 02 Aug 2012 23:28:17 +0000

For HTTP the patch is a SPDY-/HTTP/2-like protocol, the performance of HTTP is currently limited because it does not do multiplexing.

SPDY/2 is currently supported by Firefox, Chrome. An Apache module from Google, there is a beta patch from the nginx developers, node.js module, a Java server implementation and some beta C-client- and server-libraries and implementations.

TCP Fast Open: expediting web services

nix — Thu, 02 Aug 2012 22:56:27 +0000

Quite. The set of errors in the manpages (and, indeed, in POSIX) is not a total set -- the kernel is allowed to return other errors, though it is perhaps unwise to expect callers to expect those errors.

TCP Fast Open: expediting web services

dlang — Thu, 02 Aug 2012 20:42:08 +0000

you are making the assumption that the RESTful web service is read-only.

How can you do a RESTful web service where the client is sending information to the server without causing these sorts of problems?

TCP Fast Open: expediting web services

jengelh — Thu, 02 Aug 2012 19:06:57 +0000

>Google [...] to implement TCP Fast Open

Great, now GTFO is going to get a new subentry in the Urban Dictionary.

TCP Fast Open: expediting web services

pr1268 — Thu, 02 Aug 2012 18:46:39 +0000

Perhaps I'm not totally understanding what's going on here (high-level): TFO seems more like a user-space fix (i.e. Apache HTTPD, Microsoft IIS, etc.). Are changes to the system calls socket(2), connect(2), etc. the reason this article is on the LWN Kernel Development page?

Also, do I assume correctly that both client and server have to support TFO to realize the speed-up mentioned in the article?

I certainly don't mean to criticize this article (or its placement here on LWN), just curious instead. Great article, thanks!

TCP Fast Open: expediting web services

ycheng-lwn — Thu, 02 Aug 2012 17:33:42 +0000

Our draft creates this confusion that HTTP on TFO will break if the request is not idempotent because we use the word "idempotent transactions". But today the client may send a non-idempotent request twice already with standard TCP. For example, the link may fail after the server receive a non-idempotent request. the client will retry the request on another connection later since the original is not acknowledged.

TFO makes such a case possible in the SYN stage: the server reboots between when it receives request in SYN-data and when it sends the SYN-ACK. Being unaware of the reboot, the client will timeout and retransmit SYNs. If the server comes back and accepts the SYN, the client will repeat the request. But IMO the risk is minimal especially if the server defers enabling TFO until a reasonable connection timeout after reboot, e.g., 5 min.

Cheers,

-yuchung (tfo developer)

TCP Fast Open: expediting web services

tialaramex — Thu, 02 Aug 2012 16:32:42 +0000

Maybe I don't understand. Internally we have a great many RESTful web services which all work roughly like this:

The client asks if this particular combination of data (provided as GET query parameters) is found in the database overseen by that web service. The server looks in its database (e.g. maybe it's the collection of all voter registration records for a particular country) and if there is a matching record it replies saying what was found and where.

You can run that same request again and get the same exact answer and running it many times or not at all changes nothing‡ so that seems to meet your requirement entirely.

‡ In practice some of the services do accounting, they are incrementing a counter somewhere for every query run and then we use that counter to determine the payment to a third party for the use of their data. But this is no greater deviation from the concept than the usual practice of logging GET requests and anyway most of the services don't do that.

TCP Fast Open: expediting web services

man_ls — Thu, 02 Aug 2012 16:11:02 +0000

That is not required in the RFC, and I am not sure that restful, stateless web services would be even possible. After all, web services are supposed to change state in the server; otherwise what is the point?

TCP Fast Open: expediting web services

gdt — Thu, 02 Aug 2012 15:57:39 +0000

"Router latencies" isn't really an issue, as they are easily enough solved by increasing the bandwidth (which reduces the time to receive and transmit a packet). (And yeah, I'm avoiding the "bufferbloat" overprovisioning of buffers at the network edge here, because when that exists RTO is not much help -- saving one RTT when you have multiple RTT in the queue ahead of you isn't a huge win.)

The speed of light in optical fiber is the major contributor to latency. The speed of light in fiber is roughly 150Km per ms, this is much slower than the speed of light in a vacuum. The speed of light in a fiber can be improved, but at the cost of narrowing bandwidth. This tradeoff isn't available to users, but is determined during ITU standards-making. Moreover the tradeoff isn't huge, in the order of 5% of latency. But the tradeoff does have a major effect on the cost of Forward Error Correction ASICs.

Once you get out of the tail links, the lowest speed you'll encounter on a ISP backbone is 1Gbps. You've got to have well more than 1,000 router hops before you'll get 1ms of ingress playin, cell switching and egress playout. The other devices which can add significant latency are the middleboxes at the other end of the link: firewalls, load sharers and so on.

TCP Fast Open: expediting web services

epa — Thu, 02 Aug 2012 15:19:41 +0000

Yes, I was confusing idempotent (makes no difference whether requested once, or many times) with stateless (makes no difference whether requested zero, one, or many times). Ideally GET requests would be not merely idempotent but stateless, which is a stronger property. But that is not needed for TFO.

TCP Fast Open: expediting web services

alankila — Thu, 02 Aug 2012 14:04:19 +0000

Although to be fair, multiple GETs that all attempt to delete the same user aren't a problem here. So maybe the latter request crashes if website developer wasn't careful enough, but the delete itself was idempotent.

I guess cautious people can't turn TFO on unless they validate the software they runs, but now there is going to be a good reason why you want to ensure that software is idempotent-GET safe. I imagine that the vast majority of software is, actually.

TCP Fast Open: expediting web services

epa — Thu, 02 Aug 2012 13:16:12 +0000

Long ago I worked at ArsDigita where the rule was to avoid form submit buttons as much as possible. So the user management page on a site would have 'delete user' not as a POST form submission, but simply a hyperlink. This was held to make the website look cleaner and feel faster. Then a customer using a 'web accelerator' which eagerly follows links ended up trashing their whole site. The programmer's fault, of course - GET requests should never be used for strongly state-changing operations like deleting data.

TCP Fast Open: expediting web services

colo — Thu, 02 Aug 2012 12:21:07 +0000

Ah, well yes, in an ideal world... :)

I've seen enough GET-requests with at times far-reaching side effects in the wild that I'm not convinced this will (or should) see widespread adoption, at least not for "ordinary" HTTP servers that aren't serving up static content only or something like that.

TCP Fast Open: expediting web services

iq-0 — Thu, 02 Aug 2012 12:16:44 +0000

Many 'GET' operations are not necessarily idempotent. The RFC's language is 'SHOULD NOT', which is about the mildest way requirements are described.

The problem is that often different applications run in a big webserver (especially true for massive virtual hosting setups) and that "partially enabling TFO" is not really an option.

The other way round would be for the application to signal that the request is not allowed using TFO, but that would require a retry using non-TFO which is not specced and introduces more latency than is gained using TFO.

The only real solution that is safe would be to invent (yet another) HTTP header that signifies which methods for which paths under the current vhost may be done using TFO initialized connections.
For systems that support TFO for all requests (because they have higher-level guards against duplicate requests) one could simply hint '* /' or something. Only the first connection to such a site must in that case always be made using non-TFO requests.

TCP Fast Open: expediting web services

Los__D — Thu, 02 Aug 2012 11:28:25 +0000

But if it is already possible to send data with an normal SYN, and that data gets delivered to the application later, what do you gain from throwing it away when you use TCP Fast Open? If the feature can be used for SYN attacks, they would just do it without Fast Open.

I'm probably missing something, but it doesn't really make sense to me.

TCP Fast Open: expediting web services

dan_a — Thu, 02 Aug 2012 10:59:52 +0000

I would think that the problem is the resources in the OS which you could consume by doing this - especially since the handshake has already failed one trustworthiness test.