GNU C library 2.33 released

Posted Feb 4, 2021 9:45 UTC (Thu) by anton (subscriber, #25547)
In reply to: GNU C library 2.33 released by dvdeug
Parent article: GNU C library 2.33 released

This mess is probably the reason for the new scheme. So yes, glibc may not use all the features of the CPU at hand, but then, does it make a difference? And if it makes a difference, I guess the shared object for, say, x86-64-v4, can still do a CPUID check for the additional feature and then put the appropriate function pointer in the dispatch table, but at least the code can rely on all the v4 features and does not have to worry about whether these are available.

to post comments

GNU C library 2.33 released

Posted Feb 5, 2021 19:14 UTC (Fri) by nix (subscriber, #2304) [Link] (7 responses)

Yeah, most of these things use runtime checks anyway. The hwcaps scheme that this replaces was meant for things like cmov which had pervasive impact on compiler output, so that rebuilding whole shared libraries for different subarchitectures was worthwhile -- so really the question isn't "what are x86 CPUs capable of" which is indeed a bloody awful mess, but rather "what things that x86 CPUs are capable of does GCC generate noticeably better code for while not being backwardly-compatible to the older models". Since GCC's code generation for different x86 processors is completely under the control of the GNU toolchain people (and, indeed, mostly the exact same GNU toolchain people who specified this feature), I think we can say they probably won't get torpedoed in the future, since they'd literally be torpedoing themselves.

GNU C library 2.33 released

Posted Feb 5, 2021 20:17 UTC (Fri) by joib (subscriber, #8541) [Link]

LLVM has also added support for these microarchitecture levels: https://github.com/llvm/llvm-project/commit/012dd42e027e . Though I don't know if they have any input into the design, or whether they're just copying what the GNU toolchain is doing.

GNU C library 2.33 released

Posted Feb 6, 2021 6:08 UTC (Sat) by dvdeug (subscriber, #10998) [Link] (5 responses)

Looking at the PDFs joib so helpfully linked to, the latest level is AVX-512, which AMD has apparently issued no public plans to support. It seems not impossible that AMD will never support AVX-512, and will add its own extensions that GCC wants to support. H. J. Lu, the lead author on the PDF, works for Intel, and none of the authors list amd.com emails; it seems likely they're more tied into Intel's plans than AMD's plans.

From another angle, https://www.extremetech.com/computing/312673-linus-torval... mentions that Atom CPUs don't even support AVX; what happens when Atom CPUs come out with new instructions specialized for low-power ChromeBooks? Or even some specific instructions, like AES enhancements, make their way all the way down the line to Atom CPUs?

Either way, AMD adding its own instructions without supporting AVX-512 or newer instructions making their way down to cheap CPUs with limited AVX support, it seems entirely out of the control of the GNU toolchain people that there are no new instructions they need to support on CPUs that don't support AVX-512.

GNU C library 2.33 released

Posted Feb 6, 2021 10:20 UTC (Sat) by anton (subscriber, #25547) [Link] (4 responses)

It is unlikely that AMD will add their own extensions, at least as long as they don't dominate the market for AMD64 architectures. They tried that with 3DNow!, it did not become popular, and since about 2011 AMD CPUs have dropped 3DNow! (but they had to support it for quite a while, in K8 and K10). FMA4 is a similar story.

But if AMD should choose to not implement AVX-512 (I expect they will support it at some time), AMD users run the code compiled for v3; I don't see a problem here. However, looking at Bug 23249, one can get the impression that Intel succeeds in hampering AMD CPUs not just in code compiled with the Intel compiler and using Intel-supplied libraries, but also in free software; and if they succeeded in the hwcaps case, why should they fail in the instruction-set-level case?

As for Intel CPUs not supporting instruction set extensions, it's not just the Atom product line. Lots of Intel CPUs disable this instruction set feature or that, even on silicon where it is implemented (e.g., Skylake Pentiums have AVX disabled); apparently Intel's marketing thinks that they can perform market segmentation in that way.

The only thing it achieves is that almost nobody uses the more advanced features. This instruction-set level idea has a certain potential at improving this situation, but I wonder: Even if it's as easy as invoking gcc with some architecture-level option, I guess that most upstream developers will set up their build infrastructure to just compile for the base level by default (and the builder configure to have it compiled with specific options). Will distributions compile every packet for four different levels and install the one for the particular machine?

GNU C library 2.33 released

Posted Feb 6, 2021 12:23 UTC (Sat) by joib (subscriber, #8541) [Link] (3 responses)

> However, looking at Bug 23249, one can get the impression that Intel succeeds in hampering AMD CPUs not just in code compiled with the Intel compiler and using Intel-supplied libraries, but also in free software; and if they succeeded in the hwcaps case, why should they fail in the instruction-set-level case?

Well, it seems that story got a happy ending, in that the -hwcaps is the ostensibly vendor-neutral solution to the same problem the old vendor-specific "haswell" subdirectory tried to solve?

> Lots of Intel CPUs disable this instruction set feature or that, even on silicon where it is implemented (e.g., Skylake Pentiums have AVX disabled); apparently Intel's marketing thinks that they can perform market segmentation in that way.

Stupid indeed. One can understand not wanting to spend chip area on large vector units, but fusing out functional units for product segmentation reason? Argh.

> Will distributions compile every packet for four different levels and install the one for the particular machine?

I would guess that the binary package for libfoo would contain the .so's for all 4 microarchitecture levels (or if the developers or distributors do benchmarking, only those levels where it makes a difference). Then again, I recall Debian/Ubuntu used to have libc6-i686 which was glibc compiled for the i686 target rather than the default i386 or i486, so maybe they will do separate packages. Sounds tedious, but oh well.

GNU C library 2.33 released

Posted Feb 6, 2021 16:54 UTC (Sat) by Wol (subscriber, #4433) [Link] (1 responses)

> Stupid indeed. One can understand not wanting to spend chip area on large vector units, but fusing out functional units for product segmentation reason? Argh.

Or is this a bit of complex real-estate that has a habit of not working? Fusing out stuff that fails QA and segmenting the product has some logic behind it.

Cheers,
Wol

GNU C library 2.33 released

Posted Feb 6, 2021 17:49 UTC (Sat) by anton (subscriber, #25547) [Link]

At least Skylake (and its variants up to Comet Lake) can do AVX/AVX2 work with the upper 128 bits of the unit powered down, at lower performance (and it works that way until the upper 128 bits have been powered up). So a better way to deal with both market segmentation and partially failing SIMD units would be to support AVX but on the Pentiums run it without the upper 128 bits of the functional unit. If you wonder about non-working lower 128 bits: then SSE won't work and they cannot sell the die even with disabled AVX.

GNU C library 2.33 released

Posted Feb 6, 2021 17:06 UTC (Sat) by anton (subscriber, #25547) [Link]

Well, it seems that story got a happy ending, in that the -hwcaps is the ostensibly vendor-neutral solution to the same problem the old vendor-specific "haswell" subdirectory tried to solve?

Sorry for mentioning hwcaps, which played no role in this. As the bug report explains, "haswell" is documented as vendor-neutral (despite its name), with the same meaning as "x86-64-v3", but the implementation worked only on Intel CPUs as documented, by explicitly checking for Intel CPUs (like Intel does in the MKL) and using the instruction set extensions only for them. The bug was not fixed for two years, and it's not clear that it is fixed now; and the reason given does not sound convincing at all. So, can we expect that "x86-64-v3" will fare any better than "haswell"?

Concerning chip area, AMD has demonstrated that AVX can be implemented with 128-bit wide functional units, including the low-power low-cost Jaguar (competition for Intel's Atom family).

Concerning packaging, at least with Debian's multiarch support, treating them as separate architectures and compiling all libraries for every level might be less work. But distributions that only have lib32/lib64 to deal with multiple architectures cannot go there.