LFCS: The GNU C library

The GNU C library (glibc) is at the heart of most Linux systems. Developers at all levels of the system end up using the library, but there were some well-known barriers to actually contributing to glibc—up until a year or so ago. As part of that move into a more contributor-friendly mode, Carlos O'Donell—one of the glibc maintainers—updated attendees of the Linux Foundation Collaboration Summit (LFCS) on the status of the project, and how developers could help to improve the library.

Whether you are a kernel developer or an application developer, you end up working with glibc—unless, of course, you are working with Android. It has its own C library (bionic), but most Linux systems use glibc, O'Donell said.

There are several recurring complaints heard about glibc: it is too slow, has incorrect behavior, or it is missing features. "We kind of like it this way", O'Donell said, because it means that people are using it and are paying attention to it. Part of the mandate of the project is to be a high-performance C library, so complaints about performance, in particular, are looked at carefully. The project got "excellent feedback" at last year's LFCS presentation, and "we did listen", he said.

Status

O'Donell began with an overview of the status of the community. There have been two "time boxed" releases in the last year, which means the project has been through the release process twice more. In addition, two dedicated reviewers (O'Donell and Andreas Jaeger) have been added for changes to the x86 and x86-64 code. There is a renewed focus on locales, adding new ones and fixing issues with existing ones.

There is a still a need for an effective bug triage process, he said. There are also more patches coming in than can be reviewed by the core developers. In addition, the testing infrastructure is less than optimal, but O'Donell is "excited" by those things. It means that the glibc community is "not horribly broken"—it is just like every other free software project.

In a "short digression", O'Donell mentioned that there are plans to run a logo contest for glibc. He put up a slide with several suggestions of imagery. In a straw poll of the audience the "bridge troll" was the overwhelming favorite. Anyone "with a shred of artistic talent" will be able to submit their logo proposals once the contest gets going. More details will be available in an LWN announcement soon, he said.

There is only a short amount of time until the next glibc 2.18 release in June, so the time is ripe to start thinking about features that will go into the release. There is already a long list of proposed features including hardware lock elision for Intel Haswell cores, library dependency handling fixes, math routine cleanups, IPv4/v6 dual-stack issues (especially in the resolver), header cleanups to mirror the UAPI split that was done in the kernel, and so on. 54 key bugs were squashed, including two CVEs, O'Donell said. It is important to remember that changes going into 2.18 won't show up in distributions for six months or a year.

New features

One of the biggest complaints last year was the existence of "second-class architectures", but that has largely been addressed. All of the supported architectures are now in one tree, and people have stepped up to maintain the architectures that formerly lived in glibc-ports. There is still more work to do, but putting them all into one tree has been working well.

For glibc 2.17, the minimum Linux kernel version supported is 2.6.16. By updating the minimum kernel, some older code can be removed from glibc. Future glibc versions will continue to raise those minimum versions, so those who need older kernels should stick with older glibc versions as well. In general, the toolchain, glibc, and kernel versions should all "line up temporally", O'Donell said. The cross support has been improved for glibc, including cross-compilation and testing, as well as bootstrap support (where no previous glibc has been built), though there is still more to do there.

There is a new micro-benchmark suite for glibc. The project had been getting performance-related patches, but didn't really know how to decide whether to accept them or not. The benchmark makes that easier, though it is not the "be all and end all" of testing for acceptance as humans are still required to review the code and make the final decision. But the benchmark does provide a data point and it was used heavily for the math library (libm) performance improvements.

O'Donell wanted to alert ARM developers that the dynamic loader name for the hard-float ABI has changed. That requires making a link to the new loader name for existing root filesystems. There are also some changes from 2.18 that may need to be backported to 2.16 and 2.17 for mixed soft/hard-float environments (which is uncommon for ARM), O'Donell said.

New architecture support has been added to glibc over the last year. As of 2.16, x32 and Tilera support were added. 64-bit ARM (AArch64) support came in 2.17. For 2.18, Xilinx Microblaze support is in review. He suggested that anyone interested in adding support for a new architecture talk to the project as soon as possible to avoid the release delays that have come from late architecture additions in the past (e.g. AArch64).

Review

O'Donell then went through the goals he listed in last year's talk, giving a brief update on how well those goals were fulfilled. Most of the short-term goals, such as getting more involvement from distributions and the community, getting the word out, and making releases, were all successful, but there were a few that were missed. In particular, better bug triage and cooperation with the TI-RPC (transport independent remote procedure calls, which is used by NFSv4) developers did not come about because no one from the community stepped up to take them on.

A bug triage process is not something that is interesting for most developers, O'Donell said, but it is important to have something in place, especially for critical bugs and CVEs. Typically, the distributions have handled CVEs, but he would like to see them fixed in the mainline first, with backports to stable glibc releases.

The existing SunRPC code in glibc does not have IPv6 support, so a transition to TI-RPC, which does, is needed. But the glibc project wants to ensure that users' programs continue to build and run throughout the transition. Some discussion with audience members (particularly someone from the NFS-GANESHA user-space NFS server project) about the state and licensing of the existing TI-RPC libraries led O'Donell to note that RPC is not a good fit with glibc's mission. It would be nice to move it into a separate library so that new architectures would not have to maintain that piece going forward, he said. Unfortunately, that didn't sound like it could happen before 2.18, so it is likely that Microblaze will need to carry the RPC symbols. Based on the discussion, though, some collaboration between O'Donell and the NFS-GANESHA project should result in a cleanup of the SunRPC/TI-RPC issue before too long.

On the medium-term goals, some work has been done on most of the items, though upgrading the testing framework did not get any attention. There has been some work on merging EGLIBC changes back into glibc and some work on testing (including the micro-benchmark mentioned earlier). For documentation, progress has been made as well. There is now POSIX threads documentation in the glibc manual, not just the man pages as it had been in the past. In addition, each glibc function is being annotated in the documentation to show whether it is thread-safe, asynchronous-signal-safe, POSIX compliant, etc.

For the long-term goals, O'Donell was surprised that the project had actually made some progress on two of them in the last year. Auto-generating math library routines has begun, as has integrating some tracing into glibc. There are patches to add SystemTap tracepoints into various pieces of glibc, including the dynamic loader, memory allocation, threading, and math library slow paths. Other tracing solutions could be added as well, but people like SystemTap tracing much better than using ptrace(), and the tracepoints have effectively zero impact when they are not enabled, he said. Both "power awareness" and "exception-less system calls [PDF]" (which are a form of asynchronous system calls) are still on the long-term list, but have yet to see any progress.

Help wanted

There are various places where the glibc project could use some help, starting with variants for the math library. Complaints about the math library are typically that it is imprecise or that it is too slow. There are also those who don't particularly care about either the precision or speed of the library. That argues for three variants: high precision, default (for those who don't care), and constant runtime. The constant runtime variant will not be as precise, may not conform to the IEEE standards, and have other limitations.

The default math library is likely to continue using the IBM multi-precision code that is used today. For the other two variants, the routines can be auto-generated using polynomials to approximate the result. Those polynomials can take a higher number of bits to produce more precise solutions, or fewer bits for less precision (but more speed). A compiler flag would be used to choose the right namespace under the covers, he said.

There is work to be done to expand the micro-benchmark coverage of the math library. The generated code does not have the "millions of hours" of testing that the existing code has, so "math geeks" are needed to design tests and help ensure correctness. In answer to a question, O'Donell said that he knew of no demand for runtime selection of the math library. That leads to complications supporting libraries that have different math library needs (one needs precision, one constant runtime) in the same executable. It is not a per-process choice, but more of a "per-context" choice, where context is a block of code or a critical section, he said.

The project would like to move beyond just micro-benchmarking to get to whole system benchmarking. The idea would be that users would run the benchmark with their workloads, which would generate a description of the performance characteristics of that workload that could be shared with the glibc project. New releases of the library could then rerun the tests to compare how it does with many different workloads.

Doing more than micro-benchmarking is important for ARM systems, partly because there are power implications. For example, turning on the NEON co-processor for ARM means that context switches need to save and restore the NEON registers, which has performance and power implications for the rest of the system. So, a naïve optimization in glibc that turned on the NEON processor would impact every process in the system. It is those kinds of system-wide issues that need to be measured.

The runtime tunable parameters for glibc need some attention. The defaults don't match all workloads. It would be better to choose them based on some kind of average workload, or to do auto-tuning in the library. In addition, help is wanted in the networking, locales, and documentation areas for the project.

Overall, O'Donell painted a picture of an active, vibrant project that is making good progress. There is still plenty to do, of course, but the general outline of those needs has been fleshed out. After a period of semi-stagnation, it is good to see that glibc appears to be back on the right track.

[ Thanks to Elena Zannoni for her notes from the session. ]

---

(Log in to post comments)