LWN: Comments on "The trouble with struct sockaddr's fake flexible array"

Backwards compatibility at any cost?

anton — Sat, 23 Nov 2024 18:15:06 +0000

How would you do one better?

By looking at what Ken Thompson did, i.e. Plan 9. The Berkeley socket API appears totally alien to Unix, and was probably designed to have as little to do with the rest of the kernel as possible. E.g., send() instead of write() and recv() instead of read(); at least that was corrected later, but we still have send() and recv().

Backwards compatibility at any cost?

smurf — Mon, 18 Nov 2024 06:18:38 +0000

Streams are *way* cool when you do them The Right Way.
They can also eat your machine's performance for breakfast (and have room left over) when you do them naïvely.

Backwards compatibility at any cost?

smurf — Sun, 17 Nov 2024 21:13:38 +0000

> The IETF is working on a "successor to the sockets API":

Well. Eight people have written 26 RFC drafts, during the last 6 years, for this one.
That's four people, five years, and 20 drafts too many IMHO.
Also, the latest of those drafts expired two months ago, but at least it's in Approved state and in the RFC editor queue by now.

> Although who knows who will actually implement it.

IMHO the comment by Roman Danyliw (see the RFC history) is telling:

> […] I am abstaining on this document because there is a degree of specificity in the abstract API in this document that I cannot reconcile with proposed standard status. […]

… meaning that it's way not specific enough that, given this RFC, somebody'd be able to write a program that's expected to work with any specific implementation. Thus, more RFCs and system-specific (thus mostly-incompatible) scaffolding will be needed.

Thus, again IMHO, this will not replace the Socket API any time soon.

Backwards compatibility at any cost?

Cyberax — Sun, 17 Nov 2024 21:05:08 +0000

STREAMS, but different!

Backwards compatibility at any cost?

skissane — Sun, 17 Nov 2024 19:35:01 +0000

> the socket API is not a great by modern standards.

The IETF is working on a "successor to the sockets API": https://datatracker.ietf.org/doc/draft-ietf-taps-interface/

Although who knows who will actually implement it.

Interesting problem

skissane — Sun, 17 Nov 2024 19:24:47 +0000

> the Naval Research Lab's library for dealing with struct sockaddr

Is that available somewhere still? Would be an interesting bit of history to look at.

Backwards compatibility at any cost?

Cyberax — Thu, 14 Nov 2024 21:29:44 +0000

It kinda is...

A lot of the sockets API needs to be rewritten. It needs to support Happy Eyeballs, it needs to expose the DNS data explicitly (TXT queries, etc.), it needs to be able to deal with changing/multiple addresses for the bound sockets, etc.

Backwards compatibility at any cost?

smurf — Thu, 14 Nov 2024 07:01:49 +0000

The Sockets API isn't the problem. It happens not to embed a sockaddr in anything, much less in the middle of another struct.

Backwards compatibility at any cost?

dvdeug — Thu, 14 Nov 2024 05:40:49 +0000

You can always have a BSD sockets API translation layer. It could be constrained to libc, but the kernel would probably support those system calls indefinitely. But that wouldn't stop a new set of system calls and the old system calls being wrappers around the new system calls.

sockaddr used in the kernel?

johill — Mon, 11 Nov 2024 12:34:26 +0000

Well, no, as I said - I'm removing that part too now for WiFi7 hardware. There are some technical reasons for that even.

As for the question about why it "must" be coupled with changes in the CLI, well ... it doesn't really need to be, but when writing a completely new tool that for the most part has completely new semantics, keeping the CLI intact really isn't the first priority.

If you want to submit patches to make 'iw' have a mode where it behaves like iwconfig/iwlist for a subset of functionality, I guess I'd even apply them if they're reasonably well implemented.

But you can keep using it for eternity, for all I care, just don't expect new hardware support etc., and then you can just keep using an old kernel version too.

sockaddr used in the kernel?

mb — Mon, 11 Nov 2024 10:36:11 +0000

cfg80211 has always included a wireless-extensions API emulation to keep iwconfig and lwlist working.
So it's exactly the opposite of what you say: The fundamental API has changed, but the obsolete CLI and its obsolete API have not.
So, if almost two decades is not enough for you to adapt to the "new" iw CLI, then you can keep using iwconfig/iwlist.

sockaddr used in the kernel?

Sesse — Mon, 11 Nov 2024 09:21:31 +0000

I'd happily switch to iw, except… I really like the output of iwconfig and iwlist? I don't understand why we need to couple API changes with how a CLI works (especially when said CLI moves more towards the needs of low-level expert users).

Backwards compatibility at any cost?

smurf — Sun, 10 Nov 2024 21:35:38 +0000

While the Berkeley socket API is in fact not that great by today's standard, it's also not interesting WRT this discussion because it does *not* have sockaddr structs embedded somewhere in the middle of other objects.

sockaddr used in the kernel?

johill — Sun, 10 Nov 2024 21:32:49 +0000

A surprisingly large number of tools still use(d) wext, but I'm kicking them out - WiFi7 hardware no longer supports wext in any way. Let's see if I can win that fight ...

Backwards compatibility at any cost?

intelfx — Sun, 10 Nov 2024 21:31:16 +0000

> If we go that way we might just as well deprecate the entire socket API right away and invent an entirely new networking API, because frankly, the socket API is not a great by modern standards.

How would you do one better? (Architecturally, not just minor C technicalities like the one discussed in the article.)

sockaddr used in the kernel?

smurf — Sun, 10 Nov 2024 21:18:37 +0000

Ah, so ioctls for the IPv4 ARP table, legacy routing, and similar stuff that frankly I'd deprecate in a heartbeat. Does anybody (except /sbin/arp AFAICT) still not use netlink for that, these days?

Oh well.

sockaddr used in the kernel?

johill — Sun, 10 Nov 2024 19:20:49 +0000

Yeah, it's just you? ;-)

You can take a look at https://lore.kernel.org/netdev/cover.1729802213.git.gusta... for probably most/all of the APIs affected.

Backwards compatibility at any cost?

lunaryorn — Sun, 10 Nov 2024 18:33:18 +0000

If we go that way we might just as well deprecate the entire socket API right away and invent an entirely new networking API, because frankly, the socket API is not a great by modern standards.

sockaddr used in the kernel?

smurf — Sun, 10 Nov 2024 09:17:29 +0000

Exactly which API are we talking about here? (Presumably one that doesn't support IPv6 …)

I'm not aware of any system call that embeds a sockaddr in some other data structure, but maybe that's just me.

sockaddr used in the kernel?

johill — Sat, 09 Nov 2024 23:15:49 +0000

None requires 16 bytes, probably?

But for some APIs the address is _embedded_ into other structures (which is what most of the article is about), and changing the size now would change the position of everything that comes after it, breaking the ABI.

Interesting problem

jd — Sat, 09 Nov 2024 13:08:34 +0000

The non-existent uptake of the Naval Research Lab's library for dealing with struct sockaddr (IIRC, it provided a portable replacement structure and hid all the fine details) suggests that, at least in the mid 90s, there wasn't much interest in reliable memory handling.

It would have required a fair bit of work for legacy code, yes, but you might have expected new code to use it as IPv6 was quite difficult to handle.

This suggests that there's a degree of conservatism in programming, a preference for The Old Ways. This is going to complicate efforts to do anything new that is visible.

Backwards compatibility at any cost?

smurf — Sat, 09 Nov 2024 08:38:00 +0000

IMHO the only incompatibility is that for addresses smaller than 16 bytes the kernel will no longer zero out the padding when it copies an address to userspace / require the padding to even exist when it reads an address.

Everything else should literally stay the same. I have no idea how many programs go splat because their address padding is no longer overwritten but if the number is nonzero then a compatibility mode can be added to the place where the address is copied to userspace. Not spread across the whole of the kernel.

sockaddr used in the kernel?

smurf — Sat, 09 Nov 2024 08:33:38 +0000

Exactly which userspace API requires a 16-byte sockaddr? Using that type always requires a socket type. Some addresses are larger than the traditional sockaddr_t (Unix, IPv6, …) some are smaller (IPv4); there's nothing that requires zeroing out the padding in the latter.

So why does the kernel really need padded addresses anywhere? Just use / copy however many bytes the address's type requires and be done with it.

sockaddr used in the kernel?

magfr — Fri, 08 Nov 2024 21:31:42 +0000

I am actually also curious about why the padding is needed?

Would it be confirming if the kernel returned a size of 8 and only filled in the sin_family, sin_port, and sin_addr fields?

I suppose the kernel does nothing with the padding as it is today so beeing explicit about it should be ok.

It should be noted that posix says sa_data must exist but it says nothing about it's size.

Then there is the issue that all the sockaddr_xx types must have the same alignment.

What I am going for is if the following is standards conforming:

typedef alignas(16) int16_t sa_family_t;
struct sockaddr { sa_family_t sa_family; char sa_data[]; };
struct sockaddr_in { sa_family_t sin_family; uint16_t sin_port; struct inaddr sin_addr; };

with an obvious caveat for what the alignment of sa_family_t should be.

In particular this makes sizeof(struct sockaddr) == 2 and sizeof(struct sockaddr_in) == 8 but note that
sizeof(struct sockaddr[2]) == 32 due to the alignment.

Backwards compatibility at any cost?

rweikusat2 — Fri, 08 Nov 2024 20:25:44 +0000

The chances that people will modify all existing software ever developed for the BSD sockets API just because someone thinks he has a better idea for representing network addresses are - politely put - not very great.

:-)

Backwards compatibility at any cost?

jeremyhetzler — Fri, 08 Nov 2024 19:48:08 +0000

I would love to read a book about the history of API backward compatibility in OSes, and how we got to where we are today.

Because intuitively I see your point, but demonstrably Linux is very *very* committed to the concept. MS maybe less so, but still more than I imagine is convenient for them.

minimum size for flexible arrays

magnus — Fri, 08 Nov 2024 13:19:40 +0000

I learned something today! Thanks..

minimum size for flexible arrays

jengelh — Fri, 08 Nov 2024 10:20:31 +0000

Common, but not required. It's not the size that is mandated, but the bit width/value range. Think sizeof(long long)=1 with CHAR_BIT=64.

Backwards compatibility at any cost?

taladar — Fri, 08 Nov 2024 08:57:50 +0000

Sounds to me like a good example why backwards compatibility should sometimes be broken even if it causes great pain once so we won't be stuck with bad design decisions from literally decades ago forever.

sockaddr used in the kernel?

johill — Fri, 08 Nov 2024 07:58:43 +0000

Because ... history. The API to userspace was defined that way (some of it perhaps even back when 14 "was enough for everyone"), so it cannot be changed now.

Sure, wireless extensions wouldn't need 14 bytes for a MAC address there, only ever 6. But now it's stuck with 14 and can't use 6, and shouldn't use "14 or variable" either because it's embedded in other types.

sockaddr used in the kernel?

smurf — Fri, 08 Nov 2024 07:02:33 +0000

Well. But several addresses are longer anyway, so presumably the called functions restrict themselves to the sa_family-specified length.

So why do the short variants need padding up to an arbitrary 14, which happens to not match *any* actual address type's length?

minimum size for flexible arrays

intelfx — Fri, 08 Nov 2024 06:57:26 +0000

> on older archs where short is 2 bytes long

Isn't short more-or-less always 2 bytes long? (Checked x86-64, aarch64, mips64 and ppc64, what did I miss?)

minimum size for flexible arrays

magnus — Fri, 08 Nov 2024 06:23:27 +0000

I guess the data length of 14 in the sockaddr was also chosen to get a nice even size of 16 bytes on older archs where short is 2 bytes long.

Beside legacy issues, could it make sense to have a minimum size for flex arrays in some cases, like to know you can always use a bigger access to fetch it or use the first n bytes as a hash or something (provided the unused bytes are always 0)?

sockaddr used in the kernel?

iabervon — Fri, 08 Nov 2024 03:41:50 +0000

I was wondering the same thing, so I looked at some of the patches. The answer is that it's an address from a specific family. E.g., your wireless card's MAC address is always a MAC address, not an IPv6 address. However, storing those 6 bytes next to the right 2 bytes and 8 unused bytes allows it to be passed in situ to functions that can take all sorts of addresses, without needing a ton of MAC-address-specific functions or using two function arguments per address in all of the functions.

sockaddr used in the kernel?

clugstj — Fri, 08 Nov 2024 02:59:27 +0000

"There are many places in the kernel where struct sockaddr is embedded within another structure"

How can this be anything but a bug? It's not big enough for many address formats.

sockaddr API design

wahern — Fri, 08 Nov 2024 01:37:59 +0000

> In fact, we could have limitness AF_UNIX paths already if we did not constrain ourselves to using sockaddr_un (in both userland and kernelspace):

Some systems, like macOS, do allow providing a path larger than sizeof .sun_path (104 on my laptop). On those systems, when querying an address using getsockname or getpeername, you should check the returned addrlen and try again if it's larger than the provided buffer. Though in the case of macOS the path length isn't limitless, but 252 characters (address structure size of 255, including a trailing NUL character).

sockaddr API design

jengelh — Thu, 07 Nov 2024 23:52:59 +0000

>struct sockaddr_storage [...] is now what is often allocated, but it never appears explicitly in the system-call interface.

I am surprised no one has mentioned that the sockaddr struct family essentially forms a class hierarchy. Think of it like so:

struct sockaddr { sa_family_t family; } struct sockaddr_in6 : public sockaddr { uint16_t port; ... } struct sockaddr_storage : public sockaddr { char moarroom[126]; };

And then it's obvious why bind() and connect() take a struct sockaddr. It works with any sockaddr type, and in that sense, userland should have no need to care about the size of sockaddr::sa_data, or that this member even exists. sockaddr::sa_data (as well as sockaddr_storage) may help code reduction, fewer casts, or performance optimizations, but it is not needed conceptually.

>Code dealing with network addresses can allocate a [sockaddr_storage] and be sure of having enough space to store any address

sockaddrs are inherently variable-length (as evidenced by the length argument to bind/connect/accept), and sockaddr_storage does not fix that. It's just a conveniently-sized buffer for "this operating system's most common sockaddrs at the time of compilation", but it is not completely future-proof.

In fact, we could have limitness AF_UNIX paths already if we did not constrain ourselves to using sockaddr_un (in both userland and kernelspace):

const char *path = argv[1];
socklen_t addrlen = offsetof(struct sockaddr, sa_data) + strlen(path) + 1;
char *addr = malloc(addrlen);
((struct sockaddr *)addr)->sa_family = AF_UNIX;
strcpy(&addr[2], argv[1]);
int ret = bind(sk, (struct sockaddr *)addr, addrlen);

BSD didn't even save a size_t by doing this

khim — Thu, 07 Nov 2024 20:53:04 +0000

> I think the more plausible explanation is that somebody didn't feel like writing the extra code to initialize the size field.

The simple explanation is that was 1983. Time where backward compatibility started becoming important but no one really felt it would become critical.

Just five years before that FORTRAN removed some pieces from standard without anyone batting an eye!

Surely it's a software, it's malleable, people would just fix it later, when requirements would change?

It's only when people started dragging feet with upgrade to FORTRAN 77, when people essentially boycotted MS-DOS 4.0 (in 1988!) because it changed some internal data structures which “programs were not supposed to ever touch”, only when OS/2 completely imploded because someone decided to “improve” Presentation Manager API… only then developers of operation systems realized that keeping ABI stable would make or break them (and even after that moment Linux developers denied the obvious for decades, but that's another thing).

I would bet that no one expected to see sockaddr, casually introduced in 1983 just for the “large-scale experiment” (as Vint Cerf attests IPv4 was envisioned as large-scale experiment, no one expected it to be used outside of academy and there were expectation that another thing or yet another thing would be used by everyone else) – would still be in use 40 years later and would affect hundreds of billions devices… if they have heard about it they would have assumed you have lost all your marbles.

Why oh why …?

smurf — Thu, 07 Nov 2024 20:19:10 +0000

> places in the kernel where struct sockaddr is embedded within another structure, usually not at the end

Well since the subtype of "struct sockaddr" that's actually used in these places is not and never was 14+2 bytes long, why was the option to replace these with the sockaddr type that's actually used there not considered?

Seems like a no-brainer to me.

BSD didn't even save a size_t by doing this

NYKevin — Thu, 07 Nov 2024 19:37:11 +0000

As I read this, it looks like this comes down to a poor design decision in the initial API, namely the choice to use a statically-sized array instead of a dynamic array. It's easy to claim that this would have made sense at the time because it saves a size_t... but that's not actually true:

> The sa_family field describes which address family is in use — AF_INET for an IPv4 address, for example. sa_data holds the actual address, the format of which will vary depending on the family. The implementation notes say that: "the size of the data field, 14 bytes, was selected based on a study of current address formats". In other words, 14 bytes — much longer than the four needed for an IPv4 address — should really be enough for anybody.

A size_t (or the moral equivalent in pre-standards C) has never been 10+ bytes on any real platform that people have used for serious purposes, with the possible exception of a small handful of natively 128-bit supercomputers. If you're willing to waste 10 bytes in the extremely common case of IPv4, you should be willing to spend 4 bytes on a size_t/long/whatnot (or however many bytes a size_t-ish was in 1983), since that would net out to 6 bytes saved in the case of IPv4. I think the more plausible explanation is that somebody didn't feel like writing the extra code to initialize the size field.

(And no, using something other than a size_t due to its lack of standardization would not have caused problems today. Unless there's some weird networking protocol I don't know about that requires addresses bigger than 64K, a 16-bit size field would still be more than adequate even if it would be a bit nonstandard.)