That's silly. You can have strlcat() and strlcpy(), yet be compatible [LWN.net]

That's silly. You can have strlcat() and strlcpy(), yet be compatible

Posted May 20, 2009 17:44 UTC (Wed) by sbishop (guest, #33061) [Link] (15 responses)

I suspect that they're worried about application compatibility. Any application written to expect strlcat() and strlcpy() would be incompatible with glibc--making it a Debian-only app, in effect.

That's a non-problem

Posted May 20, 2009 18:14 UTC (Wed) by smurf (subscriber, #17840) [Link] (12 responses)

Since you can't take a Debian app and run it on other distributions without recompiling anyway, and since anybody who uses strlcpy and friends will still be required to ship their own versions if libc doesn't supply them, that is not a problem in any real sense of the word.

IMHO, including those functions should be no-brainers. Seriously. Anybody care to summarize any technical arguments against them?

That's a non-problem

Posted May 20, 2009 18:25 UTC (Wed) by dlang (guest, #313) [Link]

you actually _can_ take most apps from debian and run them on other distros (and go the other way as well).

there are a handful of apps that don't work, but _very_ few.

That's a non-problem

Posted May 20, 2009 18:38 UTC (Wed) by elanthis (guest, #6227) [Link] (10 responses)

And by that argument, including strlcpy() is pointless because any app wanting to use it already includes a copy of it.

Including strlcpy() in eglibc will do Linux application developers no more good than its inclusion into BSD libc. You still won't be able to use it everywhere and you'll still be required to include copies of it. If and when glibc proper supports it (which seems unlikely) then the Linux ecosystem on a larger whole will have it and people can start using it.

It's worth noting that the reasoning against strlcpy() is relatively valid. It isn't actually safe. It protects against buffer overflows, but what you really want to do is check before hand anyway and either error out or allocate a larger buffer, because truncated text in many contexts is just as much of a security issue as a buffer overrun, especially in cases where some kind of validation is done on the string before the copy is made. Even if it's not a security issue, in most cases the user is far better served by an error or warning about the truncated text than he is by just having his data silently cut off. Many people (including Drepper) believe that giving users strlcpy() is going to leave them just as lazy and will just shift software from one problem to another, instead of teaching them to Do It Right(tm) or even better just doing it right in the first place.

It is for that reason that people should consider making use of the glibc asprintf() and the standard strdup() functions instead of the BSD l-style functions.

That's a non-problem

Posted May 20, 2009 18:56 UTC (Wed) by dlang (guest, #313) [Link] (5 responses)

aren't the strlcpy() family of calls included in the latest posix spec?

That's a non-problem

Posted May 20, 2009 22:15 UTC (Wed) by msbrown (guest, #38262) [Link] (3 responses)

No, the strl* family is not in POSIX 2008. They were proposed but rejected since they did not really solve the problem (as pointed out above).

and what would they standardize?

Posted May 21, 2009 4:27 UTC (Thu) by JoeBuck (subscriber, #2330) [Link]

The OpenBSD strlcat has different behavior than the Solaris strlcat, which makes it a problem to use the function in a portable application.

That's a non-problem

Posted May 21, 2009 11:11 UTC (Thu) by ncm (guest, #165) [Link] (1 responses)

We can point to strtok() and gets(), already in POSIX, as interfaces even more stupid than strlcpy(), but that hardly seems like a compelling argument for inclusion.

That's a non-problem

Posted May 21, 2009 17:05 UTC (Thu) by madscientist (subscriber, #16861) [Link]

gets() and strtok() are in POSIX because they're in the ISO C standard runtime library, and POSIX includes the ISO C standard library in its entirety by reference.

That's a non-problem

Posted May 21, 2009 0:11 UTC (Thu) by dododge (guest, #2870) [Link]

ISO/IEC TR 24731-1 adds bounds-checking functions to the Standard C library, but it's still just a draft. The proposed functions use a "_s" suffix with existing function names. The group explicitly rejected the strlcpy name because they want to provide a large consistent suite of these functions in one shot -- not just basic string operations but also things like tmpfile_s, vfprintf_s, asctime_s, wmemcpy_s, and many others. According to the associated Rationale, strcpy_s also performs more checks and has more elaborate error handling than strlcpy.

That's a non-problem

Posted May 21, 2009 10:09 UTC (Thu) by nix (subscriber, #2304) [Link]

Yes indeed. Of course what people then do is use strdup() and forget to check for NULL returns: you'd think 'oh that only leads to a coredump, DoS only', but thanks to structures and arrays it's quite easy to convert a pointer to NULL into a pointer to *anywhere*, so NULL dereferences are often security holes.

C is a nice low-level language but should never be allowed near user data, IMHO. (Now, back to work for me, writing financial stuff in C which spends all its time getting near user data.)

That's a non-problem

Posted May 21, 2009 17:16 UTC (Thu) by spitzak (guest, #4593) [Link] (2 responses)

<i>strlcpy protects against buffer overflows, but what you really want to do is check before hand anyway and either error out or allocate a larger buffer,</i>

You seem to have missed the fact that strlcpy returns the size of the buffer that would be needed. You have a free test to see if the buffer needs to be reallocated. And if not, how about that, the copy is done, and you don't have to do another call!

The arguments againsts strl* are just stupid now. It is pretty obvious that it is what everybody wants.

That's a non-problem

Posted May 22, 2009 11:29 UTC (Fri) by liljencrantz (guest, #28458) [Link] (1 responses)

Speaking as somebody who does occasionally use strlcpy, I'm not sure I agree with that everyone wants strlcpy. The counterargument is that extremely few projects use them, so they would waste memory in the shared library, since they'd likelyu be paged in all the time because they're on the same page as actually _used_ functions. And for no benefit, since very few people use C for direct string manipulation, and those that do perform heavy duty string manipulations use a container library, and not raw C strings.

I don't know if that argument is true, but I did a quick grep of my s?bin-folders (Plain Ubuntu Handy Hardon system), and though there are a few dozen instances of programs that use strlcpy, none of them are running right now on my system, so for me personally, including them _would_ waste memory. Gimp, totem, and sftp uses it, though, and maybe the users of those applications are a majority. I honestly don't know.

That's a non-problem

Posted May 25, 2009 13:41 UTC (Mon) by wawjohn (guest, #509) [Link]

Good thing I had already finished my cup of coffee when I read your OS
version description:

> Plain Ubuntu Handy Hardon system

Got any more 'corrected' names for other Ubuntu releases?

Isn't that what autoconf is for?

Posted May 21, 2009 10:50 UTC (Thu) by alex (subscriber, #1355) [Link] (1 responses)

Doesn't autoconf check for these sort of things? I would assume packages that compile on BSD and GNU systems already have #ifdef HAS_STRLCAT or something similar.

Isn't that what autoconf is for?

Posted May 21, 2009 17:10 UTC (Thu) by jschrod (subscriber, #1646) [Link]

This is about binary compatibility, not source compatibility. `Drop-in replacement' means that one doesn't have to compile applications anew when one exchanges one glibc for the other.

That's silly. You can have strlcat() and strlcpy(), yet be compatible

Posted May 20, 2009 18:01 UTC (Wed) by xilun (guest, #50638) [Link] (1 responses)

I guess they want two-way ABI compatibility on systems where glibc already works fine.

That's silly. You can have strlcat() and strlcpy(), yet be compatible

Posted May 21, 2009 4:30 UTC (Thu) by JoeBuck (subscriber, #2330) [Link]

Yes, exactly. A binary that works with eglibc should work with glibc, so eglibc has to be a proper subset. It's supposed to be glibc with the option to strip it down, hence the "e".

That's silly. You can have strlcat() and strlcpy(), yet be compatible

Posted May 20, 2009 18:53 UTC (Wed) by MisterIO (guest, #36192) [Link] (18 responses)

But what's the problem about using libbsd anyway, if you want those functions?

That's silly. You can have strlcat() and strlcpy(), yet be compatible

Posted May 21, 2009 9:23 UTC (Thu) by danpb (subscriber, #4831) [Link] (17 responses)

Or if someone is really soooo desperate to use these 2 functions, just copy the source for them into the application codebase, so they can be guaranteed available on every possible platform. Many apps already do this kind of thing when using GNULIB, which includes source for a great many standard POSIX/libc functions, to be used on OS where the normal impl is broken / incomplete. It seems to me that most people are just using strlcat() / strlcpy() as an excuse for bashing glibc. Back in the real world there are many trivial solutions, and the lack of inclusion of these APIs in glibc really isn't blocking their use in any app that wants them.

That's silly. You can have strlcat() and strlcpy(), yet be compatible

Posted May 21, 2009 10:12 UTC (Thu) by nix (subscriber, #2304) [Link] (16 responses)

The lack of proper randomness of strfry(), though, that really is a major problem.

One complaint I really can't have about Ulrich is his commitment to a stable ABI. Yes, he's canned stuff in the past that has broken things like the JDK, but those were internal symbols exposed because the linker had no way to hide them at the time. The public interface really has maintained stability, right down to legacies of glibc's embryonic days like strfry()/memfrob() which have probably been used by about one person in the entire history of the world.

That's silly. You can have strlcat() and strlcpy(), yet be compatible

Posted May 21, 2009 12:50 UTC (Thu) by liljencrantz (guest, #28458) [Link] (15 responses)

Chain of events, probably not chronological, but illustrative:

Drepper co-creates a much quicker, Posix-compliant threading implementation, which significantly helps Linux performance of e.g. MySQL and increases adoption of LAMP stack.
Somebody with way too much time on their hands thinks it would be hillarious to add a funciton to make a random anagram of a string to glibc as a joke and gets it snuck into the codebase, where it becomes part of the stable API and has to be supported by somebody else. Forever.
Drepper significantly optimizes the glibc footprint and helps out with defining the posix libc specification.
Sombody else with even more time on their hands figures the useless anagram function isn't completely random and sends a broken patch.
Drepper writes the excellent «What every programmer should know about memory», published on LWN and then given away for free.
Sombody else with even more time on their hands figures the useless patch to the useless anagram function isn't completely random and sends yet another patch.
Drepper doesn't want to include the useless anagram patches because he feels other tasks (like, you know... anything that isn't completely useless navel-gazing wankery) are more pressing because nobody uses strfry. On the account of it being a joke. Not a single use of strfry on my Ubuntu system.
People decide Drepper is a pain to work with because he drops important patches and should preferably be kicked out of the open source community.

Yeah...

That's silly. You can have strlcat() and strlcpy(), yet be compatible

Posted May 21, 2009 16:36 UTC (Thu) by nevyn (guest, #33129) [Link] (11 responses)

> [snip strfry strawman]
> People decide Drepper is a pain to work with because he drops important
> patches and should preferably be kicked out of the open source community.

I've written patches for glibc that fixed bugs in functions pretty much every application uses, and have been accepted. I've spoken to people who have contributed even more.

I've yet to speak to _anyone_ who has anything better to say about working with Drepper than "he's actually an ok human being, if you meet him in real life".

To say I've contributed to glibc less because of Drepper, would be a vast understatement. It's been a running joke for a long time, that if you don't need to contribute to glibc don't ... and if you do, open a Red Hat Bugzilla so you just have to deal with Jakub.

The biggest problem I see with eglibc is that due to the compatibility requirements there are a large amount of things that can't be done. They really seem to be trying to stay very close to glibc, so the only thing it changes is that you can now s/Jakub/eglibc/ in the above.

That's silly. You can have strlcat() and strlcpy(), yet be compatible

Posted May 22, 2009 11:14 UTC (Fri) by liljencrantz (guest, #28458) [Link] (9 responses)

The comment I replied to specifically brought the strfry function up. So does the web page explaining why Debian has switched to eglibc. Different people have at multiple times stated that this is one of the important reasons for switching away from glibc, and I was making a direct response to one of them. Please explain to me how I am making a strawman argument.

Other then that, I agree with your comment. Drepper is by default terse to the point of approaching continuous rudeness. When people try to waste his time, he will flame them. He even perceives people as wasting his time when they are actually making valid questions sometimes. He is hard to work with. Glibc would be better of if he wasn't.

But much like Theo, Drepper is an exceptional project leader when it comes to technical aspects. The most relevant question is, would glibc be better of without him? Would it be better if an average coder with good people skills was managing the project? My guess would be no, that all the combined lost contributions from other people are smaller in value than his contributions to Glibc. Same thing with Theo. On the other hand, my guess would be that Jörg Shilling and the XFree86-people fall on the other side of that line. But that's just me guessing.

That's silly. You can have strlcat() and strlcpy(), yet be compatible

Posted May 23, 2009 9:07 UTC (Sat) by nix (subscriber, #2304) [Link] (2 responses)

Er, the comment you replied to was a tongue-in-cheek joke, pointing at
aspects of Ulrich's maintainership that I really don't have any sort of
problem with (hell, I don't care much about strfry() either: if only
glibc's objects were sorted so strfry() wasn't pulled into memory on the
majority of systems... and now with the paging-prefers-executables stuff
it won't even get pushed out again.)

Did you really not see that?

That's silly. You can have strlcat() and strlcpy(), yet be compatible

Posted May 23, 2009 10:04 UTC (Sat) by liljencrantz (guest, #28458) [Link] (1 responses)

I am sorry, I did not. My bad. On the other hand, I do not think the same argument was ironic when used to motivate Debians use of eglibc.

That's silly. You can have strlcat() and strlcpy(), yet be compatible

Posted May 23, 2009 11:37 UTC (Sat) by nix (subscriber, #2304) [Link]

The point isn't strfry() et al. It's that Ulrich's behaviour towards
someone trying to fix a bug in that exemplifies his behaviour towards
*anyone* trying to contribute *anything*, except if he's known them a long
time (and sometimes not even then).

I could dig up dozens of examples but it's too depressing and it's a bank
holiday here and I have two flashy new Nehalems to bring up (die, old
ten-year-outdated PIIIs! ... only thing is I'm not sure 36Gb RAM in the
two combined will be enough ;} )

That's silly. You can have strlcat() and strlcpy(), yet be compatible

Posted May 23, 2009 9:22 UTC (Sat) by nix (subscriber, #2304) [Link] (5 responses)

Drepper is an exceptional project leader when it comes to technical aspects

For the record I consider this incorrect. He's an excellent hacker and virtually everything he implements is a sort of glittering jewel, perfectly designed (and perfectly undocumented and he has a habit of ignoring contributions of documentation). He'd make an excellent contributor. But unfortunately that is not his role in glibc. Maintainers' roles are higher-level, and there is where Ulrich's bad side comes out.

He drops subsystems that he thinks are rarely used or that relate to architectures he merely finds unpleasant (e.g. MIPS, ARM) without listening to anyone pointing out that he might be wrong and without saying why; more than once he's rejected contributions, implemented them nearly identically himself some time later and credited himself alone in NEWS (I suspect anyone working in academe decided never to go near glibc again when that happened for the first time); he mostly refuses to give rationales for technical rejections so that people can fix whatever was wrong with their patches. Whether this is because he's scared of being convinced otherwise or just arrogant I have no idea... but can you imagine what would happen to the kernel, say, if it was run that way? 90% of the useful stuff we now have, maybe more, would simply not be there.

He's such a hard maintainer to deal with that an entire mailing list was created whose sole reason for existence was that it was somewhere where people could talk about glibc without Ulrich growling at them every five minutes for 'abusing' libc-alpha@ by, uh, discussing alpha releases of glibc on it. I have never seen *that* in any other project.

Developers can be in a project to scratch their own itch: but I think glibc is proof that large free software project maintainers need other goals too. First and foremost I think they have to be driven by a desire to make software that others can hack on easily and that does what users want, and in both of these areas Ulrich falls down like a lead balloon (especially when you note that glibc's users are random developers, and we've seen how he treats them when they dare to ask for help with one of his glibc-only totally-undocumented public APIs).

That's silly. You can have strlcat() and strlcpy(), yet be compatible

Posted May 23, 2009 10:15 UTC (Sat) by liljencrantz (guest, #28458) [Link] (3 responses)

I think it's a pretty sane model to make the mainline version of a project only handle a small number of architectures, and force people who use rare architectures to create a «portable version». This means the maintainer doesn't have to be an expert on 20 different wonky architectures, and cleanly separates the responsibility of portability from the responsibility of development. This is what e.g. the OpenSSH people do.

As to dropping good patches and then reimplementing them himself while taking full credit, I had not heard such accusations before. Do you have any specific patch set in mind, something to validate this?

That's silly. You can have strlcat() and strlcpy(), yet be compatible

Posted May 23, 2009 11:51 UTC (Sat) by nix (subscriber, #2304) [Link]

I'll have to dig it up. It didn't happen often and it was years and years
ago: I dithered about including it. It's not the real problem, to me:
maybe he changed his mind and forgot the original contribution had ever
existed, which is fine. The real problem is his attitude to contributions.

Rare architectures

Posted May 24, 2009 10:06 UTC (Sun) by man_ls (guest, #15091) [Link] (1 responses)

I'm sorry, but ARM is not a "rare" architecture. It is #4 for Debian, and would rate #3 if arm and armel were counted together.

Rare architectures

Posted May 24, 2009 11:45 UTC (Sun) by nix (subscriber, #2304) [Link]

More generally, there are more ARMs made than any other processor by a
considerable margin. It's only 'rare' if you ignore the embedded space
completely.

That's silly. You can have strlcat() and strlcpy(), yet be compatible

Posted May 23, 2009 12:06 UTC (Sat) by nix (subscriber, #2304) [Link]

Lest this seem pointlessly nasty I think the emerging de-facto split is
pretty much ideal: it leaves Ulrich and a few others to do technical
wizardly in the alleged 'upstream' where they don't have to do anything
hard like interact with human beings, and everyone else can go to eglibc,
where all the contributions from everyone else and all the stuff people
need to actually build a glibc for the distributions (other than RH) can
be added on.

This is in effect the split that has long existed anyway: it's only that
the distro stuff was split across dozens of repositories, which made it
very hard for people building their own glibc to do so. Now it's made much
easier, and everyone is in their ideal positions and can be happy :) and
Ulrich doesn't have to talk to anyone he doesn't want to, ever again,
which I suspect will make him happy. Or at least less grumpy. :)

bindresvport is another good case

Posted May 25, 2009 14:00 UTC (Mon) by saffroy (guest, #43999) [Link]

And even dealing with Jakub does not save you from having sane suggested improvements blocked forever by Ulrich: in the following case, where the glibc implementation of bindresvport is shown to be causing serious problems in very legitimate use cases, and while other distros and OSes (eg. SuSE, FreeBSD, Solaris) offer better solutions, our respected maintainer prefers some kludgy workaround (ie. portreserve):
https://bugzilla.redhat.com/show_bug.cgi?id=103401

That's silly. You can have strlcat() and strlcpy(), yet be compatible

Posted May 21, 2009 21:19 UTC (Thu) by nix (subscriber, #2304) [Link] (2 responses)

Your timeline is trivially false from inspection of the source, simply
because strfry() and memfrob() were implemented over a decade *before*
NPTL, in the very early days of glibc, not by some random nobody but by
Roland McGrath, the original author of glibc and still its co-maintainer.

It was implemented at the same time as a heap of K&R-related horrors and a
fix to strstr(): I suspect Roland was very bored after that and wanted to
do something fun to flush his brain out. Hence the glorious uselessness
that is strfry() and memfrob(). Their negative consequence is principally
that they sort lexicographically next to heavily used functions, and there
is no attempt to sort objects in libc.so according to frequency of use: so
these two functions reside permanently in memory on most Linux systems,
dragged in because the other str*() and mem*() functions are next to them
and in essentially continuous use.

That's silly. You can have strlcat() and strlcpy(), yet be compatible

Posted May 22, 2009 10:59 UTC (Fri) by liljencrantz (guest, #28458) [Link] (1 responses)

I specifically stated that the chain of events was unlikely to be chronological. :)

I didn't know it was McGrath that implemented them, but it doesn't change the fact that Drepper is (IMO) well within his right to say that he does not wish to waste his time by partaking in the joke, and when people again and again try to force him to do so, he is well within his right be grumpy.

The guys that keep pushing the patch are like some nerds I know, who keep making jokes about something very nerdish, e.g. phasers or VAX systems, in a group of people completely indifferent to the subject. Nobody gets the joke, nobody laughs, but they keep pushing the joke, trying to explain why it's funny instead of just dropping it.

That's silly. You can have strlcat() and strlcpy(), yet be compatible

Posted May 23, 2009 9:04 UTC (Sat) by nix (subscriber, #2304) [Link]

Er, they're perfectionists. They want strfry(), as long as it's there, to
be as good as it can be. Having a function that's both a joke *and* broken
would be worse yet. Ulrich is a perfectionist too, but a curiously
selective one: he only cares that the parts of glibc that he uses or that
someone's paying him for are perfect. The rest (unusual ports, locales,
POSIX functions he dislikes) can go hang.

Personally I think that's a rather discreditable attitude in a maintainer,
but maybe that's just me.

That's silly. You can have strlcat() and strlcpy(), yet be compatible

Posted Jun 2, 2009 15:46 UTC (Tue) by rmh (guest, #58387) [Link]

What's the problem with adding -lbsd to access those? They're in libbsd.