GNU C library 2.33 released

Posted Feb 4, 2021 10:00 UTC (Thu) by anton (subscriber, #25547)
In reply to: GNU C library 2.33 released by arekm
Parent article: GNU C library 2.33 released

glibc has alternative implementations for some functions already before (e.g., glibc-2.28 contains names such as __memmove_ssse3, __memmove_avx512_unaligned, __memmove_avx512_no_vzeroupper and more). So this seems to be a mechanism to help select between such versions.

to post comments

GNU C library 2.33 released

Posted Feb 4, 2021 10:03 UTC (Thu) by arekm (guest, #4846) [Link] (5 responses)

Actually I'm asking about optimized variants loaded from "under the glibc-hwcaps directory. I don't see any like this in 2.33.

GNU C library 2.33 released

Posted Feb 4, 2021 12:24 UTC (Thu) by fweimer (guest, #111581) [Link] (4 responses)

Distributions can build glibc multiple times with different build flags, and install the shared objects into those subdirectories. Some distributions already did this with a POWER9-optimized glibc in the legacy power9 AT_PLATFORM subdirectory.

GNU C library 2.33 released

Posted Feb 4, 2021 12:54 UTC (Thu) by arekm (guest, #4846) [Link]

That way, ah! Thanks.

GNU C library 2.33 released

Posted Feb 4, 2021 21:03 UTC (Thu) by joib (subscriber, #8541) [Link] (2 responses)

But there is no interaction between the glibc-hwcaps and function multiversioning, no?

So the x86-64-v4 libc.so will still contain __memmove_ssse3 etc. though it will never be used?

GNU C library 2.33 released

Posted Feb 6, 2021 10:40 UTC (Sat) by anton (subscriber, #25547) [Link] (1 responses)

That could be the case, unless somebody adjust the source code to not compile it unless the level is v2.

Anyway, reading this message makes the whole idea much clearer to me: The architecture levels are something that you set the compiler to compile for and for the dynamic linking mechanism to use in searching for libraries (not sure how it will work out for finding executables; do we have to set the PATH depending on which machine we are working on?). So the idea is that people will compile a library for, say, the v3 level, and if the auto-vectorizer finds something to vectorize, it will use AVX and AVX2 rather than, at most, SSE2.

So this will be more automatic and require less development effort to make use of instruction set extensions than what glibc does for its memmove variants.

I wonder whether the benefits are enough to make this a success. It reminds me of the x32 ABI; it had some demonstrated performance advantages over the i386 and x86-64 ABIs, but in the end it was not enough to make it fly.

GNU C library 2.33 released

Posted Feb 6, 2021 12:04 UTC (Sat) by joib (subscriber, #8541) [Link]

> The architecture levels are something that you set the compiler to compile for and for the dynamic linking mechanism to use in searching for libraries (not sure how it will work out for finding executables; do we have to set the PATH depending on which machine we are working on?). So the idea is that people will compile a library for, say, the v3 level, and if the auto-vectorizer finds something to vectorize, it will use AVX and AVX2 rather than, at most, SSE2.

Yes, that is my understanding as well. I don't think there is an equivalent mechanism for executables (like you say it could be done with $PATH, but, ugh).

> So this will be more automatic and require less development effort to make use of instruction set extensions than what glibc does for its memmove variants.

Yes. Though they are separate, there is a (minor) bloat reduction potential in not generating function versions "below" the microarchitecture level. But I guess the implementation could be somewhat hairy, and it's just a little bit of extra space on disk as it won't be loaded into memory, so maybe not worth worrying about.

> I wonder whether the benefits are enough to make this a success. It reminds me of the x32 ABI; it had some demonstrated performance advantages over the i386 and x86-64 ABIs, but in the end it was not enough to make it fly.

I think the big difference here is that it's not a new ABI, requiring the recompilation of everything. Rather this can be opt-in on a per-library basis, with the levels used chosen by the developers or distro packagers depending on what performance impact it might have. Say, glibc could provide all of x86-64{,-v2,-v3,-v4}, libfoo could provide x86-64{,-v3} and libbar could provide only the base x86-64, and depending on what hardware you run on the dynamic loader would use the best available variant of each library, mixing them freely.