Leading items

Adding strlcpy() to glibc

One of the longest-running requests for the GNU C Library (glibc) is the addition of the strlcpy() family of string functions. These functions, which have their origins in the BSD world, were written to address the longstanding security problems associated with strcpy() and its related functions. Despite years of requests, however, the glibc maintainers have never allowed these functions to be added. But that situation might just be about to change.

The problem with functions like strcpy(), of course, is that they perform no length checking on the strings passed to them. The result has been a long history of buffer-overrun issues and security problems. In response, functions like strncpy() were created, but they have problems of their own; in particular, strncpy() can create strings without the terminating null byte, encouraging other types of buffer overruns. So strlcpy() was created to ensure that all strings would be null-terminated. Problem solved, or so one might think.

Back in 2000, one Christoph Hellwig posted a patch adding strlcpy() and strlcat() to glibc. The glibc maintainer at that time, Ulrich Drepper, rejected the patch in classic style:

Christoph gave up after a token protest, but other users did not. Over the years, there have been many requests to add these functions to glibc, but the project's position has never changed. Fourteen years after Christoph's patch was posted, there is still no strlcpy() in glibc.

The primary complaint about strlcpy() is that it gives no indication of whether the copied string was truncated or not. So careless code using strlcpy() risks replacing a buffer-overrun error with a different problem resulting from corrupted strings. In the minds of many developers, strlcpy() just replaces one issue with another and, thus, does not solve the problem of safe string handling in C programs. They believe it should not be used, and, since it should not be used, it also should not be provided by the library.

The argument on the other side is simple enough: like it or not, plenty of programs use strlcpy() for string manipulation. If the system library does not provide an implementation, they will provide their own, and that implementation, beyond being duplicated code, is likely to be slower and buggier than a standard library implementation. Failure to support strlcpy() does not cause those programs to be changed; instead, it just makes them use inferior alternatives.

Florian Weimer pointed out that some 60 packages in Fedora use strlcpy(); those packages, he said, will not go away. Among other problems, the implementations of strlcpy() found in those packages do not take advantage of the FORTIFY_SOURCE option provided by GCC. With fortification turned on, a number of buffer overruns can be detected at run time, causing the program to crash but avoiding a potential security hole. The recent, remotely exploitable Samba vulnerability, Florian said, was caused by an erroneous use of strlcpy() that would have been caught if fortification were in use.

This argument has gone back and forth many times over the years (and was covered here in 2012). One might think it would go on forever, except for one thing: the management of the glibc project changed significantly in early 2012. Under the new regime, the project has been more open to the addition of new features. It took a couple years for this particular subject to come back, but, when David A. Wheeler recently asked if a strlcpy() implementation might now be accepted, glibc developer Joseph Myers responded that "it would be reasonable to consider".

Florian wasted little time in putting together a patch adding the functions to glibc. The first version ran into some criticism (it didn't behave like the BSD version), but the second iteration has been better received. Which is not to say that there is a consensus that these functions should go into the library; some developers see it as encouraging their use. But the prevalent attitude would seem to be one of resigned acceptance; as David Miller put it:

The reasons are simple enough: replace a bunch of hand-rolled strlcpy() implementations with one high-quality library implementation. The good news is that, since most programs use autotools for configuration, those programs would switch over to the glibc implementation on systems where it is available with no intervention required.

So the strlcpy() issue may finally be put to rest. Of course, that does not solve the bigger problem: what the glibc developers would recommend for safe, fast, and simple string handling for C programs. The C language does not lend itself to providing all three of "safe," "simple," and "fast" in the same package. In many cases, the right answer is to use a different language anyway. But there will be a lot of C code out there for a long time, so there will be a lot of string-handling bugs as well, regardless of whether strlcpy() is used.

Comments (92 posted)

X and SteamOS

In a talk entitled "SteamOS Magic", longtime X developer Keith Packard looked at the new Linux "distribution" and the effort to turn the Linux desktop into a gaming console. It turns out that, with a fairly small amount of code, Steam and SteamOS creator, Valve, was able to take the existing X-based desktop and turn it into a "living-room experience". He gave the talk at LinuxCon North America in Chicago on August 21.

Packard's employer, Intel, has been "working closely with Valve" on graphics issues for some time now. Valve's interest is to see more Linux gaming and, to that end, the Steam game-management client was ported to Linux with Intel's help. Packard said that Valve was "happy to work with us", since Intel has a "nice, compliant OpenGL" running on its GPUs.

Steam and SteamOS

From the perspective of a game developer, there is not much difference between Steam and SteamOS. The latter is a way to create a new console, without developing new hardware. Instead, a standard PC platform is used, which might sound familiar. The architecture is similar to that of the Xbox, he said. SteamOS is meant to support the existing catalog of Steam games, many of which now run on Linux.

There are three basic elements to the Steam experience. First, there is the "enormous library of games" that have been ported to Steam and are tested to ensure they run well on Steam. There is a store to sell hats and other merchandise. Finally, there is a social network in Steam that uses both voice and text integrated into the game. "Even gamers have friends", Packard said with a chuckle.

The user interface revolves around the "Steam overlay", which is an in-game interface that "sits on top of your game". It is "amazing" that the overlay works at all, he said. To make it work, it redirects the internals of OpenGL to show the overlay. He is not really a video gamer (instead preferring a different game that "involves Emacs and typing"), though his children are, so he has seen the overlay in action both in testing and with them.

SteamOS is based on Debian Linux, with the standard X Window System on top of it. There are a number of customizations, as well as the SteamOS compositing manager (steamcompmgr). There is also "a long list of kluges" that make X work for SteamOS, he said.

Steam and SteamOS use essentially the same software, but provide a different experience for users, which embodies the difference between the desktop and console worlds. On the desktop, there may be many applications running on multiple monitors, with desktop widgets, icons, menus, and so forth. On a console, there is just one full-screen application. Any system UI in a console is either an overlay, as in Steam, or simply replaces the game entirely.

The ecosystems for the desktop and console are quite different as well. Developers focus on the newest CPU/GPU combinations for the desktop, typically developing on pre-release hardware and targeting future hardware configurations. Desktop games will tolerate a wide range of systems, with configurable rendering options and output sizes. The console ecosystem is more homogeneous, with the same CPU, GPU, monitor size, and basic input devices on every system. It also features limited configuration, no screen-size changes, and few rendering options.

So the "SteamOS challenge" is to take a desktop operating system and window system, add in desktop-oriented games, and run it all on desktop hardware—all while providing a living-room experience, Packard said.

Enter X

X makes for a good user experience (UX) toolkit because it focuses on "mechanism, not policy", he said. There is no built-in UX and there are hooks that can be used to design any UX, such as external window management and external compositing. In addition, of course, the source is available to be hacked up in whatever way is needed.

The SteamOS UX has two modes. "Desktop mode" is a standard GNOME desktop running the Steam client. "Big picture mode" uses the SteamOS Console UX, which is what the rest of the talk would be covering, he said.

Desktop computers have keyboards, but consoles have controllers. In addition, desktop games expect to get their input from a keyboard and mouse. So Valve built a custom, programmable controller that looks like a keyboard and mouse to the game. The user sees and interacts with a controller, though, that maps various buttons and other controls to keyboard and mouse events.

It turns out that for the living room, "mode switching is bad". A TV can go black for five or ten seconds if you switch modes. That is only getting worse with modern video links, Packard said. Also, the monitor may not support the mode that the game wants, especially if it wants a large mode. To combat that problem, SteamOS never changes the video mode, it simply scales the game to fit the size of the output display.

UI components

SteamOS has several different UI components, starting with an X server that has some "custom modifications". There is also the steamcompmgr, which is a custom compositing and window manager. It talks to the X server, as does the Steam UI, which appears to be the same as the desktop version, though there is no source available for that piece. The Steam UI is a proprietary piece of code that is "watching everything you do", he said, which might be a bit worrisome for some.

The games also talk to the X server, but there is an LD_PRELOAD library that intercepts mode-switching calls. A game might make an RandR call to change the mode, and the intercepting library just "says yes and does nothing". It records the information for use in calculating the appropriate scaling, however.

Steamcompmgr is based on the original X compositing manager, which is some "really bad X code", he said. It is GL-based, which ensures VBLANK synchronization and provides the fastest drawing path. Window management is integrated into the compositing manager, which is common for X environments. There is just one full-screen application window that is scaled to fit the screen (with letterboxing or pillarboxing to preserve the aspect ratio), with the Steam UI overlay optionally on top of that. In many ways, SteamOS is simply using the X Window System to "do what it supposed to do", he said.

Some implementation details

Many games use Simple DirectMedia Layer (SDL), which creates two windows, one application window centered inside the other, full-screen window. Steamcompmgr digs the second window out from within the first and uses that directly.

In order to fake mode switches, the LD_PRELOAD library hooks a bunch of X calls, which simply pretend that they worked. It also drops certain "fullscreen" input events so that steamcompmgr does not get confused. It uses the "zoom lock" in the XF86VidMode extension to prevent applications that somehow evade the LD_PRELOAD from causing mode switches.

The application output is scaled as needed in the compositor to make it fill the screen. The overlay is not scaled, however, so it will still be high-resolution even if the game itself has been scaled.

The cursor may also need scaling. To do that, the X server cursor is hidden and the compositor fetches the cursor image from the server. It then paints a scaled version into the display. Cursor position scaling also has to be done. The position will get "warped" if it ever goes outside the display boundaries. In addition, applications are prevented from grabbing the cursor themselves.

Future

There are two separate features that are likely to lead to future work in X. Handling the cursor scaling in the server, rather than having to do it in the compositor is one. Another is handling applications that are larger than the screen. Packard presented several possibilities for supporting these features in X.

"Hack number one" would be to add a "mouse proxy window" that would redirect and scale mouse events from one window to another. That would work for both windowed and full-screen mode, and have no impact on the output path, but it might be hard to position menus and other pop-up windows correctly.

Another option would be to run the game within a nested X server instance. It would be easy to implement with Xephyr, he said, and not require any server changes. But inter-client communication (to, say, cut and paste between client applications) would be hard to do and there is some potential for added latency on input and output.

Finally, there is an RandR-based screen scaling option. It has several advantages, including that there would be no issues with window and input positions, but there are some problems as well. For one thing, the code to do the output scaling would need to be changed to allow the compositing manager to do it, rather than being done only by the X server, as it is today. Also, the existing code does an unnecessary mode switch even when the mode doesn't need to change.

In answer to a question, Packard said that Valve's code has been released, but not proposed for upstream inclusion. "Their requirements are useful", but not the implementation.

Packard wrapped up by noting that SteamOS shows "how easy it was" to take an existing Linux desktop, add a bit of code, and turn it into a living-room, console experience. We will have to wait and see how successful it is in the market, however.

[I would like to thank the Linux Foundation for travel assistance to Chicago for LinuxCon North America.]

Comments (54 posted)

An open Q&A with the KDE e.V. board

The yearly KDE gathering Akademy is more than just a two-day conference. The core talk days are followed by several days of workshops and are preceded by the annual general assembly meeting of KDE e.V., the non-profit organization that represents the KDE community. The organization is led by an elected, five-member board of directors, and that board held an "Ask us anything" session on the first day of Akademy 2014. Attendees asked questions about the KDE e.V. board, its inner workings, and its priorities.

Three board positions were up for election this year, which were filled in a vote at the general assembly the day before the session. Marta Rybczyńska was elected as treasurer, Aleix Pol was elected as a new member, and Lydia Pintscher was re-elected to her seat, and was then chosen by the board to be the president. Joining these three for the question-and-answer session was board member Albert Astals Cid; the fifth board member, Pradeepto Bhattacharya, was not present.

A new president and new leadership

The first question was directed at Pintscher, congratulating her on her election as president and asking if, as president, she had any sort of special powers. Pintscher answered that all of the board members have equal power to sign contracts on behalf of KDE, access KDE's financial accounts, sign contributor licensing agreements, and to represent both KDE e.V. and the KDE project. Pintscher said that "those are some powers, and they come with responsibilities," but that she wouldn't call them extraordinary.

The next question was whether or not the new board has had a chance to talk about the first things they want to do. Pintscher responded that they have not yet had a chance to do so, but added that there are several challenges ahead, such as improvements to fundraising, better accounting processes, and things like that. Rybczyńska continued, explaining that accounting is currently done in a "manual way," and consequently it takes a lot of effort to do correctly. She said the board will be moving to an online accounting system, and hopes it will save a lot of time and reduce complications. The new system will let the board have "more or less" real-time access to KDE e.V.'s current financial situation, which will make things easier to handle. Astals added that one of the first things the new board did was to try to find another date on which to meet in person, and that they had settled on some time in December. He elaborated that the board will still have daily conference calls and will still work in the meantime; it will just have a longer board meeting then.

The next question was about the paid position of KDE e.V. executive director; the executive director is independent of the board, and the role is currently unfilled. An audience member asked when the recruitment process for filling that job would begin. Cornelius Schumacher, the outgoing board president, responded from the audience that he assumes it will require another month or two to take care of the rest of the process. The scope and the requirements for the position have already been decided upon, and there is a group of people working on the process. Unfortunately, he said, not everyone involved could be present at Akademy, so the work will have to be done online.

The audience was suddenly without ideas for a moment but, after taking a short and lighthearted break, someone asked for more details on the inner workings of the board—specifically, who handles requests for travel support. Pintscher answered that the board does it together, as the requests are delivered to the board mailing list. At least three people have to agree on accepting a request, she said, with no one objecting. Once that happens, one of them sends the "okay" to requester. Rybczyńska added that in the second part of the process, after the board receives the bill for the travel, one of the board members checks to ensure that the bill is within the budget.

Workload handling

The next question was a hypothetical: if someone wants to prepare a batch of KDE-branded t-shirts and sell them at events, can the e.V. pre-pay for them, and is that something that the board decides? Pintscher answered that the person can ask, but that this sort of activity is something the board tries to avoid doing. The question then arose, in response, whether or not the board was even the correct place to ask. Pintscher admitted that it is probably not the right place for such a question, but that there is, currently, no alternative. It would be nice, she said, if the board could get help from outside, perhaps even employees, to handle those questions.

On a related note, the next question was how the e.V. board decides if it should send KDE people to a particular event. Astals explained that this is not currently a problem, as there are not "so many requests," and the requests that it gets are "very obvious," like sending someone to Qt Developer Days.

An audience member then commented that the board has had a lot of work since Claudia Rauch—who had worked as the business manager of KDE e.V.—left her position in March. The audience member asked how other members of the organization could help, and how the board is handling the delegation of tasks. Pintscher answered that they have several ways of dealing with the workload. One is asking a large group, such as as the e.V. membership or the KDE community mailing list, if there is someone who can help. If that doesn't work, the board tries to think of people who have done similar tasks before and approaches them directly, which, she added, is a more successful approach most of the time. In the next stage, if the task is crucial, someone on the board will have to do it, or else it will get deferred or just not get done.

Pintscher agreed that the board needs more help. Astals confirmed that it is good to recruit people who have done similar work before—but added that for some stuff, they cannot get outside help, such as financial matters. Pintscher also added that delegating itself takes time, and that factor is something to keep in mind as well.

Community questions

The next question was for Pintscher in particular: she has been working for the Wikimedia Foundation for a long time now, so has she learned anything there that can be used to improve KDE e.V.? Pintscher replied that there are many such things, some of which she is already bringing in. The biggest difference, in her eyes, is the amount of money and paid personnel that Wikimedia has; more than 350 people are being paid to work on the Wikipedia software and its sister programs. But, she said, the large size also creates some problems, and she had considered whether it isn't better to be "smaller and cozy." But she concluded that there are drawbacks and benefits to both situations.

A follow-up question was whether KDE e.V. would have the same issues if it grew to that size. Pintscher said that she thinks some issues could be avoided, but some are inherent in growth and in the friction between a large paid staff and volunteers.

The question then arose about whether the relatively new community mailing list is a success: specifically, has enough traffic been moved from the board's closed mailing list to the open one, and is that migration still a goal? Pintscher confirmed it is still a goal, and that she felt they have moved several important topics there, but she added that the community list is even more important as a central place to discuss topics that are not development related but affect everyone. Rybczyńska noted that most people active on the community mailing list are also e.V. board members, which would be nice to change.

In a follow-up question, someone asked if KDE working groups should post reports to the community list, which they are not doing currently. Pintscher agreed that it was a good idea and the board will try to encourage it. Quarterly reports for the e.V. were also mentioned, and Pintscher asked for volunteers to help with them, as they are behind schedule. Someone then suggested that "help wanted" requests should be made in the web discussion forums, which have many more users than the mailing list. Pintscher said that the board hasn't tried the forums yet, and that maybe it should.

The blueberry pancake

The discussion then returned to Claudia's departure as business manager; an audience member made a memorable and well-received comparison to a blueberry pancake, where the 200-odd KDE e.V. members are the pancake and the board is a blueberry on top. The question was, is the blueberry enough people to organize everyone else? Or should a "syrup" (another management layer) be created?

Pol replied that the analogy isn't strictly true, because the various working groups serve as a way to address these organizational concerns. But the questioner disagreed, saying that the working groups are just "a little bit of syrup." Pol replied with a counter-question: why aren't there more working groups, then, and why aren't more people joining them? Astals also mentioned the importance of working groups, adding that the board knows that all 200 KDE e.V. members aren't going to contribute simultaneously, but that isn't the problem. The e.V. needs all of those people to be there, he said, and to have 50 different people working every month, so that it can get ideas from everyone over time.

Continuing the debate, the same audience member replied that if "these people are not in the organization to do work to support its aims, they shouldn't be in there." He suggested that the community should come up with a way to enable these people to contribute, adding that other projects' boards are more proactive about getting volunteers to contribute. The e.V. board does not do that, which he considers a cultural problem. Pintscher agreed completely, adding that the current system is not sustainable. She admitted there should be "more syrup and more blueberries".

The discussion then moved on to the lack of a management structure. The e.V. organization is not free software, an audience member noted, so it can have management. Delegation had been called a hard job for the board, they continued, so that means it is necessary to add one more layer of delegation. Pintscher responded that the board really needs to think about this and find ways to solve the current issues. She said she doesn't know if the solution is another layer of management or something else, but she acknowledged that board members need to work on the problem in the coming weeks and months.

The board members were then asked about their own management experience, and it was suggested that if their skills are lacking, they should take management courses. Astals mentioned that during the last board meeting, the board had a fundraising expert on hand for a day, and that it was helpful. He agreed that it might be a good idea to do a management workshop.

At that point, the session was brought to an abrupt halt when the allocated time was exceeded. Although not all audience questions were answered, the talk seemed to have been productive for both the board, which received various suggestions and gained a valuable insight into what KDE e.V. members think is important, and for the audience, which had an opportunity to learn more about inner workings of the board and its priorities over the coming year.

Comments (3 posted)