Security [LWN.net]

TCP cookie transactions

By Jake Edge
December 16, 2009

In the currently ongoing Linux kernel merge window, for the kernel which will become 2.6.33, a new TCP feature has been added. TCP cookie transactions [PDF] are meant to eliminate various kinds of attacks, such as denial of service, while making the TCP connection handshake use fewer resources. One of the main motivations for cookie transactions is to avoid some problems that have cropped up in rolling out DNSSEC (Domain Name System Security).

DNSSEC responses are substantially larger than those of DNS, large enough that they have outgrown the default UDP datagram size of 512 bytes. UDP is generally used for DNS today, but large responses from DNSSEC over UDP result in multiple IP fragments. While it is perfectly reasonable to break up UDP packets that way, there are a large number of Network Address Translation (NAT) routers and firewalls that do not properly handle multiple UDP fragments.

When a DNS response is not received—or not received properly—a DNS resolver will typically retry the request over TCP. Because TCP is connection-oriented, there is a handshake that goes on to establish that connection before any data gets transferred. Normally, servers need to save some state between the two client packets that constitute the handshake. When handling an enormous number of requests, as the DNS root servers will for example, the storage of the state information adds up quickly. In addition, the well-known SYN-flood attack sends just the first packet of the handshake, often from a spoofed IP address, and never replies to the server to complete the connection. Enough "half open" connections can exhaust the server's resources, leading to a denial of service.

SYN cookies were created to defend against SYN flood attacks, and have been in the Linux kernel since 1997 when those attacks were raging. But, as Perry Metzger, William Allen Simpson, and Paul Vixie describe in their TCP cookie transactions (TCPCT) paper linked above, SYN cookies are only used when a system is under attack. They are a clever hack that uses the TCP sequence number to allow servers to defer using resources until they receive the second handshake packet from the client. Crucially, SYN cookies did not require client support, so they could be deployed unilaterally on the server side.

Various other mechanisms have been proposed to handle these problems over the years but, as outlined in the paper, failed to completely solve the problem. TCPCT sets out to do just that. It adds a new TCP option that contains a much larger, cryptographically secure cookie that is sent by the client in the initial handshake (SYN) packet. The server can then create a cookie for the reply that only it can decode. When the client uses that cookie in its second handshake (the third overall of the three-way handshake), the server can recover all of the information it needs to establish the connection from the cookie.

In addition, TCPCT allows for a limited amount of data to be sent in the request from the client and reply from the server, which allows for a query/response like DNS to be handled as part of the connection establishment. In those cases, the connection is torn down as soon as it is established.

TCPCT also addresses another problem that heavily used servers often have: port exhaustion. The TCP protocol requires that there be a timeout before port numbers are reused so that old messages that get delivered do not get confused with those of a newly-established connection. This is the TIME_WAIT timeout (usually four minutes) that is often annoying to those who restart server programs frequently (at least those without the SO_REUSEADDR socket flag). There are a limited number of ports available (nominally 64K, but at least 1K are reserved), an active server may have all of its free ports in the TIME_WAIT state. Because TCPCT can distinguish new and old connections based on the cookie data, it no longer has to wait on the server side. Only clients need wait out the TIME_WAIT period.

Obviously, TCPCT requires client support, and it will be some time before most operating systems have that support. As is often the case, Linux is out ahead of the pack by supporting TCPCT in the mainline. But even for Linux, it will be quite some time before 2.6.33 kernels make their way out to users via their distributions. Given that, widespread DNSSEC deployment seems quite a few years off, something that is a bit disheartening given all of the recent DNS server issues.

Comments (7 posted)

Brief items

Kretschmann: The Malware Problem (and a solution)

Amarok hacker Mark Kretschmann looks at the recent malware hidden in a GNOME screen saver. As he points out, it certainly isn't a GNOME-specific problem, as the same thing could happen to KDE and other projects. He and Ian Monroe came up with a way to help alleviate the problem by requiring public version control for Amarok scripts. "With a VCS [version control system], it's very easy to tell who inserted Malware, and when this person did this. This fact alone would provide some accountability, and I think it might prevent a good deal of attempts of messing around with the code. And even if it happened anyway, it would be trivial to revert the change, and we would just ban the person who did this from ever committing to this repository again."

Comments (57 posted)

New vulnerabilities

asterisk: denial of service

Package(s):

asterisk

CVE #(s):

CVE-2009-4055

Created:

December 11, 2009

Updated:

June 4, 2010

Description:

From the CVE entry: rtp.c in Asterisk Open Source 1.2.x before 1.2.37, 1.4.x before 1.4.27.1, 1.6.0.x before 1.6.0.19, and 1.6.1.x before 1.6.1.11; Business Edition B.x.x before B.2.5.13, C.2.x.x before C.2.4.6, and C.3.x.x before C.3.2.3; and s800i 1.3.x before 1.3.0.6 allows remote attackers to cause a denial of service (daemon crash) via an RTP comfort noise payload with a long data length.

Alerts:

Gentoo	201006-20	2010-06-04
Fedora	FEDORA-2009-12517	2009-12-02
Fedora	FEDORA-2009-12506	2009-12-02
Debian	DSA-1952-1	2009-12-15
Fedora	FEDORA-2009-12461	2009-12-02

Security

TCP cookie transactions

Kretschmann: The Malware Problem (and a solution)

asterisk: denial of service

asterisk: multiple vulnerabilities

cacti: cross-site scripting

Cacti: privilege escalation

firefox: multiple vulnerabilities

firefox-sage: insufficient input sanitizing

gpdf: buffer overflow

kdebase-runtime: missing input validation

kernel: privilege escalation

kernel: denial of service

merkaartor: symbolic link vulnerability

moodle: multiple vulnerabilities

mysql: denial of service

postgresql: multiple vulnerabilities

rt3: session hijack

rubygem-actionpack: strip_tags function weakness

webkit: multiple vulnerabilities