GNU C library 2.33 released

Posted Feb 2, 2021 16:52 UTC (Tue) by josh (subscriber, #17465)
Parent article: GNU C library 2.33 released

This is the headline feature:

> The dynamic linker loads optimized implementations of shared objects from subdirectories under the glibc-hwcaps directory on the library search path if the system's capabilities meet the requirements for that subdirectory. Initially supported subdirectories include "power9" and "power10" for the powerpc64le-linux-gnu architecture, "z13", "z14", "z15" for s390x-linux-gnu, and "x86-64-v2", "x86-64-v3", "x86-64-v4" for x86_64-linux-gnu. In the x86_64-linux-gnu case, the subdirectory names correspond to the vendor-independent x86-64 microarchitecture levels defined in the x86-64 psABI supplement.

This is a replacement for the old hardware capabilities mechanism, which caused a combinatorial explosion of checks for each capability. This new mechanism groups capabilities together into logical architecture levels that are increasing supersets, so it can just look for one of them at a time.

to post comments

GNU C library 2.33 released

Posted Feb 4, 2021 6:39 UTC (Thu) by dvdeug (subscriber, #10998) [Link] (10 responses)

Except that looking at https://en.wikipedia.org/wiki/Advanced_Vector_Extensions#... :
For various AVX-512 features--see Wikipedia for the descriptions--
Knights Landing supports ER.
Knights Mill supports ER, 4FMAPS and VPOPCNTDQ.
Skylake-SP supports VL (and none of the above).
Cannon Lake supports VL and IFMA.
Cascade Lake-SP supports VL and VNNI.
Ice Lake supports VL, IFMA, VBMI2, VPOPCNTDQ and VNNI.

That is, Knights Mill is a superset of Knights Landing; Skylake is a subset of Cannon Lake and Cascade Lake (neither of which are a subset of each other), and Ice Lake is a superset of Cannon Lake and Cascade Lake (but not Knights Landing).

Likewise, FMA4 can never be supported in this scheme, since more recent processors don't support it.

Basically AMD64 is a mismash of supported features, across all chips and even within just Intel chips. Even if you're not concerned about getting optimal performance out of older chips, nobody could have known when FMA4 came out if and how it was going to be supported in the future. Looking at the full AVX-512 table, I don't know what features in Ice Lake are going to be standard in the future; it seems like either they're going to be running far behind the leading edge, or end up defining the top "logical architecture level" as something that it turns out no new processors are supporting, and thus being forced to break the architecture levels.

I've got "System V Application Binary Interface: AMD64 Architecture Processor Supplement (With LP64 and ILP32 Programming Models) Version 1.0", but I don't see any details about the microarchitecture levels; if that's the right document, the newer versions only come as TeX files.

GNU C library 2.33 released

Posted Feb 4, 2021 9:45 UTC (Thu) by anton (subscriber, #25547) [Link] (8 responses)

This mess is probably the reason for the new scheme. So yes, glibc may not use all the features of the CPU at hand, but then, does it make a difference? And if it makes a difference, I guess the shared object for, say, x86-64-v4, can still do a CPUID check for the additional feature and then put the appropriate function pointer in the dispatch table, but at least the code can rely on all the v4 features and does not have to worry about whether these are available.

GNU C library 2.33 released

Posted Feb 5, 2021 19:14 UTC (Fri) by nix (subscriber, #2304) [Link] (7 responses)

Yeah, most of these things use runtime checks anyway. The hwcaps scheme that this replaces was meant for things like cmov which had pervasive impact on compiler output, so that rebuilding whole shared libraries for different subarchitectures was worthwhile -- so really the question isn't "what are x86 CPUs capable of" which is indeed a bloody awful mess, but rather "what things that x86 CPUs are capable of does GCC generate noticeably better code for while not being backwardly-compatible to the older models". Since GCC's code generation for different x86 processors is completely under the control of the GNU toolchain people (and, indeed, mostly the exact same GNU toolchain people who specified this feature), I think we can say they probably won't get torpedoed in the future, since they'd literally be torpedoing themselves.

GNU C library 2.33 released

Posted Feb 5, 2021 20:17 UTC (Fri) by joib (subscriber, #8541) [Link]

LLVM has also added support for these microarchitecture levels: https://github.com/llvm/llvm-project/commit/012dd42e027e . Though I don't know if they have any input into the design, or whether they're just copying what the GNU toolchain is doing.

GNU C library 2.33 released

Posted Feb 6, 2021 6:08 UTC (Sat) by dvdeug (subscriber, #10998) [Link] (5 responses)

Looking at the PDFs joib so helpfully linked to, the latest level is AVX-512, which AMD has apparently issued no public plans to support. It seems not impossible that AMD will never support AVX-512, and will add its own extensions that GCC wants to support. H. J. Lu, the lead author on the PDF, works for Intel, and none of the authors list amd.com emails; it seems likely they're more tied into Intel's plans than AMD's plans.

From another angle, https://www.extremetech.com/computing/312673-linus-torval... mentions that Atom CPUs don't even support AVX; what happens when Atom CPUs come out with new instructions specialized for low-power ChromeBooks? Or even some specific instructions, like AES enhancements, make their way all the way down the line to Atom CPUs?

Either way, AMD adding its own instructions without supporting AVX-512 or newer instructions making their way down to cheap CPUs with limited AVX support, it seems entirely out of the control of the GNU toolchain people that there are no new instructions they need to support on CPUs that don't support AVX-512.

GNU C library 2.33 released

Posted Feb 6, 2021 10:20 UTC (Sat) by anton (subscriber, #25547) [Link] (4 responses)

It is unlikely that AMD will add their own extensions, at least as long as they don't dominate the market for AMD64 architectures. They tried that with 3DNow!, it did not become popular, and since about 2011 AMD CPUs have dropped 3DNow! (but they had to support it for quite a while, in K8 and K10). FMA4 is a similar story.

But if AMD should choose to not implement AVX-512 (I expect they will support it at some time), AMD users run the code compiled for v3; I don't see a problem here. However, looking at Bug 23249, one can get the impression that Intel succeeds in hampering AMD CPUs not just in code compiled with the Intel compiler and using Intel-supplied libraries, but also in free software; and if they succeeded in the hwcaps case, why should they fail in the instruction-set-level case?

As for Intel CPUs not supporting instruction set extensions, it's not just the Atom product line. Lots of Intel CPUs disable this instruction set feature or that, even on silicon where it is implemented (e.g., Skylake Pentiums have AVX disabled); apparently Intel's marketing thinks that they can perform market segmentation in that way.

The only thing it achieves is that almost nobody uses the more advanced features. This instruction-set level idea has a certain potential at improving this situation, but I wonder: Even if it's as easy as invoking gcc with some architecture-level option, I guess that most upstream developers will set up their build infrastructure to just compile for the base level by default (and the builder configure to have it compiled with specific options). Will distributions compile every packet for four different levels and install the one for the particular machine?

GNU C library 2.33 released

Posted Feb 6, 2021 12:23 UTC (Sat) by joib (subscriber, #8541) [Link] (3 responses)

> However, looking at Bug 23249, one can get the impression that Intel succeeds in hampering AMD CPUs not just in code compiled with the Intel compiler and using Intel-supplied libraries, but also in free software; and if they succeeded in the hwcaps case, why should they fail in the instruction-set-level case?

Well, it seems that story got a happy ending, in that the -hwcaps is the ostensibly vendor-neutral solution to the same problem the old vendor-specific "haswell" subdirectory tried to solve?

> Lots of Intel CPUs disable this instruction set feature or that, even on silicon where it is implemented (e.g., Skylake Pentiums have AVX disabled); apparently Intel's marketing thinks that they can perform market segmentation in that way.

Stupid indeed. One can understand not wanting to spend chip area on large vector units, but fusing out functional units for product segmentation reason? Argh.

> Will distributions compile every packet for four different levels and install the one for the particular machine?

I would guess that the binary package for libfoo would contain the .so's for all 4 microarchitecture levels (or if the developers or distributors do benchmarking, only those levels where it makes a difference). Then again, I recall Debian/Ubuntu used to have libc6-i686 which was glibc compiled for the i686 target rather than the default i386 or i486, so maybe they will do separate packages. Sounds tedious, but oh well.

GNU C library 2.33 released

Posted Feb 6, 2021 16:54 UTC (Sat) by Wol (subscriber, #4433) [Link] (1 responses)

> Stupid indeed. One can understand not wanting to spend chip area on large vector units, but fusing out functional units for product segmentation reason? Argh.

Or is this a bit of complex real-estate that has a habit of not working? Fusing out stuff that fails QA and segmenting the product has some logic behind it.

Cheers,
Wol

GNU C library 2.33 released

Posted Feb 6, 2021 17:49 UTC (Sat) by anton (subscriber, #25547) [Link]

At least Skylake (and its variants up to Comet Lake) can do AVX/AVX2 work with the upper 128 bits of the unit powered down, at lower performance (and it works that way until the upper 128 bits have been powered up). So a better way to deal with both market segmentation and partially failing SIMD units would be to support AVX but on the Pentiums run it without the upper 128 bits of the functional unit. If you wonder about non-working lower 128 bits: then SSE won't work and they cannot sell the die even with disabled AVX.

GNU C library 2.33 released

Posted Feb 6, 2021 17:06 UTC (Sat) by anton (subscriber, #25547) [Link]

Well, it seems that story got a happy ending, in that the -hwcaps is the ostensibly vendor-neutral solution to the same problem the old vendor-specific "haswell" subdirectory tried to solve?

Sorry for mentioning hwcaps, which played no role in this. As the bug report explains, "haswell" is documented as vendor-neutral (despite its name), with the same meaning as "x86-64-v3", but the implementation worked only on Intel CPUs as documented, by explicitly checking for Intel CPUs (like Intel does in the MKL) and using the instruction set extensions only for them. The bug was not fixed for two years, and it's not clear that it is fixed now; and the reason given does not sound convincing at all. So, can we expect that "x86-64-v3" will fare any better than "haswell"?

Concerning chip area, AMD has demonstrated that AVX can be implemented with 128-bit wide functional units, including the low-power low-cost Jaguar (competition for Intel's Atom family).

Concerning packaging, at least with Debian's multiarch support, treating them as separate architectures and compiling all libraries for every level might be less work. But distributions that only have lib32/lib64 to deal with multiple architectures cannot go there.

GNU C library 2.33 released

Posted Feb 4, 2021 20:53 UTC (Thu) by joib (subscriber, #8541) [Link]

> Basically AMD64 is a mismash of supported features, across all chips and even within just Intel chips.

AFAIU the glibc-hwcaps and the microarchitecture levels are meant to provide somewhat decent baselines that apply to many CPU's out there and are expected to be "standard" features in x86 CPU's going forwards. It's up to the psABI developers to not be too aggressive in adding stuff there that might disappear from the market or are of interest only to a small subset of the market (both of these applying to the Xeon Phi's you mentioned).

> I've got "System V Application Binary Interface: AMD64 Architecture Processor Supplement (With LP64 and ILP32 Programming Models) Version 1.0", but I don't see any details about the microarchitecture levels; if that's the right document, the newer versions only come as TeX files.

For some reason the PDF's are excruciatingly hard to find. If you click on the "Wiki" section you get a placeholder page saying that the project has no wiki pages. However, the PDF's are there, the URL is

https://gitlab.com/x86-psABIs/x86-64-ABI/-/wikis/x86-64-p...

(The microarchitecture levels are in Table 3.1)