|
|
Subscribe / Log in / New account

Time to use AF_UNIX

Time to use AF_UNIX

Posted Aug 8, 2024 18:57 UTC (Thu) by ibukanov (subscriber, #3942)
In reply to: Time to use AF_UNIX by quotemstr
Parent article: 0.0.0.0 Day: Exploiting Localhost APIs From the Browser (Oligo Security)

There is no standard way to express AF_UNIX in URL. Each applications come up with its own encoding.


to post comments

Time to use AF_UNIX

Posted Aug 8, 2024 19:00 UTC (Thu) by quotemstr (subscriber, #45331) [Link]

Sure. Add this problem to my list of improvements we have yet to make.

Time to use AF_UNIX

Posted Aug 9, 2024 6:21 UTC (Fri) by jengelh (guest, #33263) [Link] (28 responses)

Just `file:///run/user/0/bus`. Since schemes cannot contain ':' or '/', normal paths like `/dev/sd0:a' cannot be misread as a URI, and so leaves room for implementation-defined behavior (usually treating it like file:).

Time to use AF_UNIX

Posted Aug 9, 2024 10:50 UTC (Fri) by leromarinvit (subscriber, #56850) [Link] (23 responses)

But where does the AF_UNIX socket path end, and the path that's sent in the HTTP request start?

Time to use AF_UNIX

Posted Aug 9, 2024 13:51 UTC (Fri) by matthias (subscriber, #94967) [Link] (12 responses)

Just resolve the path component by component. If you find a socket, that is where the HTTP request starts. There cannot be a directory with the same path as a socket.

Time to use AF_UNIX

Posted Aug 9, 2024 14:32 UTC (Fri) by leromarinvit (subscriber, #56850) [Link] (11 responses)

While that would work, I'm not sure I like it. There's something to be said for the simplicity of being able to split a URL into scheme, host, and path just by looking at the string, without touching any external resources. Giving that up seems like it could introduce subtle bugs in interesting places.

Time to use AF_UNIX

Posted Aug 9, 2024 15:40 UTC (Fri) by quotemstr (subscriber, #45331) [Link]

It's really no different from putting a web app under non-root path using regular HTTP. For example, http://myhost.example.com/~foo/bar versus http://myhost.example.com/~bar/qux. Using the GP's suggestion, we'd just make the host portion of the URL empty and interpret empty as localhost. (You'd write http:///var/run/user/1000/blah.sock or something.) This approach lets us also talk about unix sockets on other hosts if we ever want to do that, e.g. http://some-machine/var/run/user/1000/blah.sock.

Time to use AF_UNIX

Posted Aug 10, 2024 17:18 UTC (Sat) by KJ7RRV (subscriber, #153595) [Link] (9 responses)

Would this be feasible?

http:///var/run/example:unix/index.html

That way ":unix", as if it were a port, is the delimiter.

Time to use AF_UNIX

Posted Aug 10, 2024 17:30 UTC (Sat) by quotemstr (subscriber, #45331) [Link] (6 responses)

Good idea, if a bit verbose.

Time to use AF_UNIX

Posted Aug 10, 2024 17:33 UTC (Sat) by KJ7RRV (subscriber, #153595) [Link] (5 responses)

Maybe it could even just be http://var/run/example:/index.html, since there's no ambiguity as to what type of socket it is, and ports are not a relevant concept?

Time to use AF_UNIX

Posted Aug 10, 2024 17:34 UTC (Sat) by KJ7RRV (subscriber, #153595) [Link] (4 responses)

s/http:\/\//http:\/\/\/

(I had accidentally included only two slashes; it would actually be three)

Time to use AF_UNIX

Posted Aug 14, 2024 11:07 UTC (Wed) by koh (subscriber, #101482) [Link] (3 responses)

Oh noes, unterminated `s' command.

Time to use AF_UNIX

Posted Aug 14, 2024 12:43 UTC (Wed) by anselm (subscriber, #2796) [Link]

This is why sed and friends let you use delimiters other than /.

sed -e 's,http://,http:///,'

Time to use AF_UNIX

Posted Aug 16, 2024 16:23 UTC (Fri) by lkundrak (subscriber, #43452) [Link] (1 responses)

A friend once remarked that Vim's search'n'replace doesn't in fact doesn't require terminating. You wouldn't believe how many slashes I saved over the course of years.

Time to use AF_UNIX

Posted Aug 16, 2024 16:54 UTC (Fri) by mb (subscriber, #50428) [Link]

Have you carefully stashed them away for the times when your slash key might be broken in the future?
Or have you carelessly thrown those unused slashes away?

Time to use AF_UNIX

Posted Aug 10, 2024 22:04 UTC (Sat) by smcv (subscriber, #53363) [Link] (1 responses)

If I'm reading correctly, that wouldn't be compliant with <https://datatracker.ietf.org/doc/html/rfc3986>.

Time to use AF_UNIX

Posted Aug 11, 2024 1:55 UTC (Sun) by KJ7RRV (subscriber, #153595) [Link]

What part of the RFC does it violate? Would it be fixed by percent-encoding the slashes? (That would be far from ideal, but it could work.)

Time to use AF_UNIX

Posted Aug 9, 2024 15:55 UTC (Fri) by jengelh (guest, #33263) [Link] (9 responses)

>But where does the AF_UNIX socket path end, and the path that's sent in the HTTP request start?

If the uniform resource you are trying to identify is the *socket*, then its uniform resource identifier is `file:///run/abc`. If the uniform resource you are trying to identify is a HTTP *document* (and it just so happens to be reachable through such a socket), then that socket becomes the host and you actually have to have http: in front, e.g. `http://%2Frun%2Fabc/index.html`. That is also consistent inasfar that you could theoretically pass "/run/abc" to getaddrinfo() just like "lwn.net" and get a struct sockaddr out of it for use with socket()+connect().

Time to use AF_UNIX

Posted Aug 11, 2024 16:42 UTC (Sun) by deptrai (guest, #70612) [Link] (8 responses)

What’s wrong with a new scheme, e.g. httpu:? Analogous to https: being HTTP over TLS, httpu: is HTTP over unix-domain.

The issue of conformance to RFC 3986 would be solved, as the authority is specific to several schemes, and each scheme is free to specify its own syntax; if a slash-free authority is desired, the same proposed solutions such as percent-encoding can be used, but other easier-to-read solutions will still conform.

The issue remains of differentiating which path components refer to the socket versus the HTTP path, but this can be solved, including by the same manner as the other proposals in this thread.

But, a new scheme would greatly reduce the ambiguity and confusion, both for URL parsers and humans looking at the URL, over using http:, file:, or re-using other schemes.

The difference in cognitive load in trying to determine, at a quick glance, what a URL refers to, in particular that it’s over a unix-domain socket, between some of the proposals in this thread, e.g.

http://%2Fvar%2Frun%2Fexample/foo/bar
file:///var/run/example:unix/foo/bar

and e.g. something like:

httpu:///var/run/example:/foo/bar

or:

httpu://{/var/run/example}/foo/bar

or:

httpu://%2Fvar%2Frun%2Fexample/foo/bar

As a new scheme, the number of leading slashes is now open for modification as well, the extra slashes being a mistake per Tim Berners-Lee, although I favor leaving them for consistency, hence retaining the cognitively familiar:

httpu:///absolute/path:/bar
httpu://~user1/foo.socket:/bar

Rather than the simpler but unfamiliar:

httpu:/absolute/path:/bar
httpu:~user1/foo.socket:/bar

It might complicate sending HTTP over TLS over unix-domain, but I’m having difficulty imagining a use-case for this.

It does not generalize to other protocols such as ftp, imap, etc., unless the same naming convention is used there (as has already been done with the “s” suffix). It also doesn’t apply to custom protocols over unix-domain sockets, but in general custom protocols either will have custom URL formats, or not use a URL at all to refer to the service.

Both of the above-mentioned drawbacks exist equally for all of the file: proposals, and in fact are worse since they don’t specify a protocol.

All that said, given that Linux has a “fast path” that eliminates the TCP protocol when connecting to localhost, something that I only learned from reading the comments on this article, the need for HTTP over unix-domain seems much less useful to me – I’ll be looking into that.

Time to use AF_UNIX

Posted Aug 11, 2024 17:57 UTC (Sun) by intelfx (subscriber, #130118) [Link] (7 responses)

> What’s wrong with a new scheme, e.g. httpu:? Analogous to https: being HTTP over TLS, httpu: is HTTP over unix-domain.

Separation of concerns. UDS is not a protocol, it's an address namespace.

Are we going to add `ftpu://`, `davu://`, `imapu://` too (and also "-s" variants of all of the above for <protocol> over TLS over UDS)?

Time to use AF_UNIX

Posted Aug 11, 2024 18:55 UTC (Sun) by deptrai (guest, #70612) [Link] (6 responses)

> Are we going to add `ftpu://`, `davu://`, `imapu://` too

Well, I did mention in my GP post that this we already have a precedent for this in `ftps:` etc., and also that this drawback is shared by all of the `file:` proposals.

One proposal, which is very similar to `httpu:` but a little more formal, is "composability" of schemes, e.g. `http+unix:`, `ftp+unix:`, etc. See https://github.com/whatwg/url/issues/577 . IMO it's a little too verbose and differs from the precedent of `https:`, but I think it's better than the other proposals in this thread. Does that address your concerns?

> (and also "-s" variants of all of the above for <protocol> over TLS over UDS)?
As I mentioned in my post, I'm having a difficult time thinking of a real-world use-case for TLS over unix-domain. I'm not saying it doesn't exist, but can you suggest one?

Time to use AF_UNIX

Posted Aug 11, 2024 22:59 UTC (Sun) by intelfx (subscriber, #130118) [Link] (4 responses)

> One proposal, which is very similar to `httpu:` but a little more formal, is "composability" of schemes, e.g. `http+unix:`, `ftp+unix:`, etc. See https://github.com/whatwg/url/issues/577 . IMO it's a little too verbose and differs from the precedent of `https:`, but I think it's better than the other proposals in this thread. Does that address your concerns?

No, it doesn't: all of this still ignores the fact that UNIX domain sockets are *not a protocol*, they are an address namespace. We don't have distinct URL schemes for HTTP over IPv4 and HTTP over IPv6, do we?

Time to use AF_UNIX

Posted Aug 11, 2024 23:04 UTC (Sun) by intelfx (subscriber, #130118) [Link]

(Although if we end up _having_ to shove it into the URL scheme somehow, then I concede that the "+unix" proposal is the least bad one.)

P.S.: apologies for the noise — LWN really ought to have an edit/retract feature

Time to use AF_UNIX

Posted Aug 11, 2024 23:09 UTC (Sun) by Cyberax (✭ supporter ✭, #52523) [Link] (2 responses)

There is actually a question: why _would_ you use regular HTTP over Unix sockets? Unix sockets already provide sequential packets, they don't need the "Host" header, compression is not just irrelevant, but actively harmful, etc.

A cut-down protocol deserves its own scheme.

Time to use AF_UNIX

Posted Aug 11, 2024 23:12 UTC (Sun) by intelfx (subscriber, #130118) [Link]

> There is actually a question: why _would_ you use regular HTTP over Unix sockets? Unix sockets already provide sequential packets, they don't need the "Host" header, compression is not just irrelevant, but actively harmful, etc.
>
> A cut-down protocol deserves its own scheme.

Certainly so, but that is a completely different question and a completely different discussion.

Time to use AF_UNIX

Posted Aug 12, 2024 10:49 UTC (Mon) by mathstuf (subscriber, #69389) [Link]

My use case was a user-local proxy (versus a network-wide one listening on a TCP port).

Time to use AF_UNIX

Posted Aug 12, 2024 4:28 UTC (Mon) by wtarreau (subscriber, #51152) [Link]

> As I mentioned in my post, I'm having a difficult time thinking of a real-world use-case for TLS over unix-domain. I'm not saying it doesn't exist, but can you suggest one?

It's already used. TLS permits to convey extra information such as SNI, ALPN etc. If you want the same level of service as with TCP, you may need TLS as well. It also allows to offload TLS processing from a TCP connection to the local process listening to the UNIX socket.

As previously mentioned, it's important not to mix up protocols and address spaces. It's the same reason the "http" scheme doesn't change between IPv4 and IPv6.

Time to use AF_UNIX

Posted Aug 9, 2024 15:17 UTC (Fri) by smcv (subscriber, #53363) [Link] (3 responses)

OK, you've identified a socket. But now what protocol do you speak over it? (Sending HTTP requests to a D-Bus server like the socket you used as your example is not going to work; sending D-Bus requests to a HTTP server, or ftp requests to an IMAP server, won't work any better.)

Representing the AF_UNIX socket in the authority part of the URL (somehow) seems more in the spirit of how DNS domain names, IPv4 addresses and IPv6 addresses, together with their port numbers, can all appear in the "authority" part of the URL. (Designing the exact representation so that it's reasonable to write by hand without introducing syntactic ambiguity is the hard part of this, because "/" isn't allowed to appear unescaped in the authority part of the URL.)

That way, HTTP over AF_UNIX would be http://<something>/path?query, where <something> represents some sort of encoding of the AF_UNIX path (so you would connect to an AF_UNIX address identified by <something>, and then send "GET /path?query" as the HTTP request).

That would also allow https://<something>/, ftp://<something>, imaps://<something> and so on, for speaking different protocols over an AF_UNIX socket.

Time to use AF_UNIX

Posted Aug 10, 2024 17:26 UTC (Sat) by mussell (subscriber, #170320) [Link] (2 responses)

Designing the exact representation so that it's reasonable to write by hand without introducing syntactic ambiguity is the hard part of this, because "/" isn't allowed to appear unescaped in the authority part of the URL.
IPv6 had a similar ambiguity issue as ":" is used to both separate parts in a IPv6 address and to separate the host from the port number in the URL authority. The solution was to require literal IPv6 addresses to be enclosed in square brackets. The same idea could be extended for local paths, eg. http://{/run/foo.sock}/bar?baz would connect to /run/foo.sock and send GET /bar?baz as an HTTP request.

Time to use AF_UNIX

Posted Aug 10, 2024 22:05 UTC (Sat) by smcv (subscriber, #53363) [Link] (1 responses)

I don't think https://datatracker.ietf.org/doc/html/rfc3986 (the generic syntax of all URIs) allows unescaped slashes to appear in the "authority" component in any form, unfortunately.

Time to use AF_UNIX

Posted Aug 10, 2024 23:54 UTC (Sat) by mussell (subscriber, #170320) [Link]

It would necessarily be a breaking change to the URL format. The changes to the ABNF would be something like

local-name = '{' *(unreserved / '/' ) '}'
host = IP-literal / IPv4address / reg-name / local-name


Note that unreserved in RFC 3986 corresponds to the Portable Filename Character Set from POSIX with the addition of '~'. This should allow using '/' in the host the same way IP-literal allows ':' in hosts.

Interestingly it seems that RFC 3986 is no longer the canonical standard for URLs as WHATWG maintains a URL standard of its own. In fact, the issue of UNIX sockets was brought up on its tracker, but the issue is closed for reasons I struggle to understand.


Copyright © 2025, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds