Weekly Edition
Return to the Kernel page
| |||||||||||||
Reserved network ports
It is not all that uncommon to have a network application which needs to be
able to bind to a specific port. Often, such requirements result from a
firewall configuration allowing incoming connections only to a specific
port, but there can be other reasons as well. When running such an
application, it can be unpleasant to discover that somebody else's
long-running ssh connection happened to stumble onto the required port. It
would be nice to be able to avoid this kind of conflict if at all possible.
This patch set from Octavian Purdila aims to make it possible. It adds a new sysctl knob (called ip_local_reserved_ports) under /proc/sys/net/ipv4. Should the system administrator write a comma-separated list of ports (or ranges of ports denoted by a hyphen) to this parameter, the networking layer will avoid those ports whenever it picks a port number for a new socket. Reserving ports in this manner will not interfere with any application which binds to those ports explicitly. This patch has been through a surprising number of revisions; chances seem good that it will show up in the mainline once the 2.6.34 merge window opens. (Log in to post comments)
Reserved network ports Posted Feb 25, 2010 11:42 UTC (Thu) by nix (subscriber, #2304) [Link]
Obvious example of processes that often grab reserved ports and block other daemons from starting: NFS daemons. Currently most distros have a hack in libc to sort-of-fix this, but doing it in the kernel is obviously a better idea.
Reserved network ports Posted Feb 25, 2010 12:36 UTC (Thu) by hppnq (guest, #14462) [Link] Hmm... You mean because NFS components can take any port number and advertise it through the portmapper? Recent Linux implementations have an option to use reserved ports only -- I guess that's the hack you refer to. ;-)I think this patch would not solve that particular problem. It does not aim to solve the problem that two separate processes might try to grab the same reserved port through bind(reserved_port): it is that of accidental use of a reserved port through bind(whatever_port). Besides, isn't this trivially solved in userspace, for instance by simply grabbing the desired port -- more complicated solutions that do this can be devised -- and complaining loudly if this fails? Of course there is a small race window, but unless you can tell the kernel that port X should be reserved for process Y, you won't be able to completely rule that out anyway. (Ironically, that portmapper is not such a bad idea. ;-)
Reserved network ports Posted Feb 25, 2010 14:51 UTC (Thu) by mrshiny (subscriber, #4266) [Link]
Isn't NFS using "any port number and advertising it" exactly the same as any other program asking for "any port", and that port happens to be a reserved port?
I have this problem on my windows PC at work: Windows assigns misc ports in a range that JBoss expects to be able to bind to, so if I don't start JBoss first it may not start at all. I work around it by shifting JBoss's ports to some other ports which are not allocated right after boot. But this doesn't really solve the problem.
This feature lets me say to the OS that it should never hand out a specific port unless an app requests it. Yes, you have a race if two processes want to open port 80* as a web server, but now you at least don't have to worry about _something else_ opening port 80 by accident and blocking your webserver.
Sounds like a good idea to me.
*(ok, that port isn't usally one of the randomly assigned ones, but it serves an a example)
Reserved network ports Posted Feb 25, 2010 17:37 UTC (Thu) by nix (subscriber, #2304) [Link]
Yes, that's the problem exactly. It makes booting unreliable, so it would be good to have a proper fix. (It would be even better to have a portable one, but something the sysadmin can tweak is better than nothing.)
Reserved network ports Posted Feb 25, 2010 23:26 UTC (Thu) by hppnq (guest, #14462) [Link] You have to have good reasons for solving this in kernel space, is what I mean. Not just because it may involve a performance hit and adds some complexity, but also because you still have to take appropriate measures in user space if you are serious about reserving an ephemeral port for a specific process.But sure, if the patch is not too intrusive, it can't really hurt. On somewhat recent Windows systems, you actually can reserve ports. ;-)
Reserved network ports Posted Feb 26, 2010 6:40 UTC (Fri) by dlang (✭ supporter ✭, #313) [Link]
how would you implement this in userspace without having to change every program (that uses ephemeral ports) on the system? Remember to include closed source and staticly linked programs in your solution.
The other thing that should be made easier is using a port for more than one thing at a time.
a connection is defined by 4 items, source IP, source port, destination IP, destination port.
a listening service is defined by the destination IP and destination port.
Thus you could have a connection from port 1234 on your local IP to some destination while still having a server listening for new connections on port 1234. The kernel can tell when a packet arrives if it is for an existing connection or not.
the only place this can't be done is for connectionless protocols like UDP, but is it really the best thing to block the port from being used by anything else? or would it be good enough to let an app (or a wrapper for the app, possibly including firewall-like logic in the kernel like IPTables uses) define a connection and deliver return traffic on that connection to one app, while delivering other traffic hitting that port to other app(s)?
even on TCP the inability to re-use ports can be a problem. With the default TCP timeouts you cannot do more than ~16,000 connections/sec before you start running out of ports. If ports could be re-used with different remote points this limit would be ~16,000 connections/sec with a single remote IP/port combination
Reserved network ports Posted Feb 26, 2010 13:56 UTC (Fri) by hppnq (guest, #14462) [Link] how would you implement this in userspace without having to change every program (that uses ephemeral ports) on the system? Remember to include closed source and staticly linked programs in your solution. Just to be clear: you need to reliably bind a specific fixed address in the ip_local_port_range to a socket, where a multitude of other processes may be competing for the same resource. If it is impossible to do this using whatever configuration options the software offers to move the desired port out of ip_local_port_range, or by starting it early enough, or through the use of another layer like inetd or a proxy and/or NAT, I wouldn't rule out admitting defeat and moving it to its own interface or even (virtual) system. The measures mentioned here need to be considered anyway unless you have good reasons to assume that no other process will attempt to bind(reserved_port), and in any case it seems a bit more reasonable to expect software to honour the concept of an ephemeral port than it is to expect a kernel to facilitate programs that completely ignore it. Of course, only a fool would run an important service on a fixed ephemeral port without taking additional measures to make sure it is actually the intended service handling the communication. So to me, the more interesting question would be: what would you do if you had more than one of those programs you mention using the same fixed port number?
Reserved network ports Posted Feb 26, 2010 18:29 UTC (Fri) by dlang (✭ supporter ✭, #313) [Link]
no, the issue here is when one program wants to bind to a specific port in the ip_local_port_range only to find out that another program was allocated that port when it had no preference and told the kernel to allocate a port for it.
if you have multiple programs attempting to explicitly use the same port you have problems, but there should be no reason for the explicitly bound ports to conflict with kernel allocated ports.
Right now ip_local_port_range is a single range of ports. This patch allows the admin of the box to tell the kernel to skip over specific ports in that range. This effectivly makes ip_local_port_range be a list of ranges rather than a single range.
the problem is that right now there _are_ no 'additional measures' that a sysadmin can take to make sure that ephemeral port requests don't get this port other than just narrowing the range of available ports to not include that port, but that can drasticly cut the number of ephemeral ports available to the system.
In the cases where I have two services that need to use the same port I add a second IP address to the interface and bind one service to each IP address. but that isn't the case that these patches are addressing.
Reserved network ports Posted Feb 26, 2010 19:29 UTC (Fri) by mrshiny (subscriber, #4266) [Link]
Exactly. The problem is that there are already programs which expect to use certain "ephemeral" ports as a well-known listening port. You can't close your eyes and wish that away. With this patch the "ephemeral ports" list gets smaller and more configurable.
Seems like an obvious improvement to me with no obvious downside.
Sure there are other problems not addressed by this patch, but I feel it _is_ progress. Heck, just put all the ports in /etc/services in here and you're already off to a good start.
Reserved network ports Posted Feb 27, 2010 14:39 UTC (Sat) by nix (subscriber, #2304) [Link]
But I want to be able to close my eyes and wish NFS away!
For a while it looked like POHMELFS was just the thing: a faster better
I don't *have* a cloud. I have one or two great big servers and I want to
Is there anything better? For a while it looked like POHMELFS would be,
Reserved network ports Posted Feb 28, 2010 22:18 UTC (Sun) by hppnq (guest, #14462) [Link] I was thinking of things like ypbind, which does not do a bind(0), but generates its own "random" port, and it will grab that port if you don't do anything in userspace to prevent it. I'm perfectly aware that this is not the case that is intended to be addressed by this patch. But if you want to solve this problem, why not the one that is addressed by the patch?I must be starting to sound like I have something against this patch, which is not the case, so I'll drop it here.
Reserved network ports Posted Mar 1, 2010 1:14 UTC (Mon) by dlang (✭ supporter ✭, #313) [Link]
huh, I wasn't aware that there was software out there that did that sort of thing. I see the fix for this being fairly simple, change misbehaving programs like ypbind to either use fixed, configurable ports, or if they don't care, do bind(0) (or just don't run them on any system you want to be reliable ;-)
Reserved network ports Posted Feb 27, 2010 0:18 UTC (Sat) by nlucas (subscriber, #33793) [Link]
I like your proposal, as a non expert on the field, but I would not be
surprised if that can't be done because in the past some (bad) security was implemented by relying on the fact port numbers can't be multiplexed.
Reserved network ports Posted Feb 27, 2010 0:33 UTC (Sat) by dlang (✭ supporter ✭, #313) [Link]
If that is the only problem, make it optional.
Reserved network ports Posted Feb 27, 2010 13:52 UTC (Sat) by nix (subscriber, #2304) [Link]
Currently, everyone who works around this (Debian, Ubuntu, possibly
others) ignores statically linked programs (they are immune to libc changes and LD_PRELOAD both so are nearly impenetrable: ptrace()-based hacks are terrible for performance and overkill for this), and just changes libc to read a list of reserved ports from a file such as /etc/bindresvport.blacklist. But changing libc is an inefficient kludge compared to doing it in the kernel: every single process that binds a reserved port has to reread this file rather than just setting this system-wide property once and for all.
Reserved network ports Posted Apr 1, 2012 0:59 UTC (Sun) by foom (subscriber, #14868) [Link]
You're conflating two different issues. The file "bindresvport.blacklist" is for the "bindresvport" function only, which allows binding a port < 1024. This function is not implemented in the kernel, it's entirely in glibc. Furthermore, programs using it are rather rare: it's basically only ever used by a few portmap-based programs, like NFS or yp.
This new kernel functionality is for the much more common case of basically every program that makes outgoing connections (or does a bind with port specified as 0).
So the kernel functionality does not obsolete bindresvport, and bindresvport does not allow doing what the kernel functionality does.
The bindresvport workaround is necessary, because there isn't really any unallocated space < 1024.
The kernel workaround should really not be necessary, since nobody should actually be allocating fixed ports > 32k. Those are reserved for ephemeral allocation, and there's plenty of <32k ports that people could be using instead. But, a glance in /etc/services shows quite a few *official* services with such ports assigned, never mind all the random ports private services might be incorrectly using. So...despite it being terrible that it's necessary, this feature does sound useful.
|
|||||||||||||