LWN.net Weekly Edition for September 26, 2013

Free drivers for ARM graphics

At the 2013 Linux Plumbers Conference in New Orleans, Rob Clark presented an update of the progress toward free software graphic drivers on ARM systems. Free drivers for desktop display hardware are more actively developed, with many users simply resigning themselves to the use of binary-only drivers in the ARM system-on-chip (SoC) world. But as Clark explained, the effort to provide free drivers has gained significant momentum in the past year alone.

A few years ago, Clark said, the situation looked very bleak, and there were no projects attempting to write free drivers for ARM SoCs. Then Luc Verhaegen started his Lima project, an attempt to reverse engineer the Mali GPU. That project "motivated the rest of us," Clark said, and now there are four active projects, each targeting a different GPU.

The first that Clark described is the Etnaviv project for GPUs made by Vivante. The Vivante GPUs are on the low-end, usually found in mobile phones and other small devices like the nettop-class CuBox PC. These GPUs have a fairly straightforward design that is similar to many desktop GPUs. They use a unified shader architecture, meaning that there are not separate shader units designated for separate purposes (e.g., vertex shaders, pixel shaders, and geometry shaders). Most Vivante models have supported OpenGL ES 2.0, while the latest revision has moved to OpenGL ES 3.0 and OpenCL. The instruction set is all vector-based, and historically offered only floating-point data—although here, too, there is a recent change: the newest models now offer integer support, too.

Etnaviv is making very rapid progress, Clark said: work only started in late 2012. As of now, the project has produced a working Gallium3D driver, which is capable of playing some games. But it only supports the Linux framebuffer (fbdev) backend. The X backend needs a lot of help, he said: Direct Rendering Manager (DRM), Direct Rendering Infrastructure (DRI), and 2D X (DDX) support are all missing.

For NVIDIA's Tegra SoC series, there is the Grate project. Tegra is a widely-adopted SoC that can be found in a number of well-known products, such as Samsung Galaxy tablets, the (first-generation) Google Nexus 7, and Trim-Slice nettops. The GPU architecture is more sophisticated than Vivante's, with separate vertex and fragment shaders. The instruction set is "minimalist," Clark said, not even providing loops, but the GPUs tend to offer good performance by incorporating a massive number of cores. The devices support OpenGL ES 2.0.

Grate is still in the very early stages, he said, and is not yet usable. Grate can capture and replay a command stream, the basic GL state is well understood, and the project has reverse engineered the vertex shader instruction set architecture. But the vertex shader is straightforward; the challenge is the fragment shader, which "is more weird," he said. It uses three separate instruction streams: one for texture lookup, one for varying variable interpolation and the "Special Function Unit" or SFU (which implements out-of-the-ordinary functions like reciprocal square roots), and one for the standard arithmetic logic unit (ALU). But even the ALU is unusual; it accepts packets of three to four instructions and offers just four opcodes.

Clark next discussed the Lima project, which was the first such ARM GPU driver project to get started. Most of the Lima effort is directed at the Mali 200 and 400 series, although work has recently started on the Mali 600 series. The 200 and 400 are similar: both support OpenGL ES 2.0 and offer separate vertex and fragment shaders, and both support 2D and cubemap textures. The 600 series diverges considerably: it supports OpenGL ES 3.0 and OpenCL 1.1, uses a unified shader architecture (with different models using varying numbers of cores and ALUs of varying register widths), and supports 3D textures in addition to 2D and cubemap textures. Mali 200/400 chips are found in high-end phones, Allwinner devices, and some Samsung tablets, while the Mali 600 series is found in Google's Chromebooks and Nexus 10 tablet.

The 200/400 series vertex shader is a bit unusual, he said. It is single-threaded but deeply pipelined. It uses very long instruction words (VLIW), each of which can include two additions, two multiplications, one complex ALU operation, one pass-through ALU operation, one attribute load, one register load, one uniform load, and one varying store. In addition, there are no explicit output registers: the outputs from previous instructions are routed into the current instruction, a job which the shader compiler is responsible for sequencing correctly. The fragment shader is not quite as strange, he said, although it uses variable-length VLIW rather than the fixed-length VLIW design of the vertex shader.

As of now, the Lima driver is starting to reach usable status on Mali 200 and 400 series GPUs. There is a Mesa-based DRI driver that runs es2gears (the OpenGL ES version of the famous "glxgears" demo) as well as some other 3D demos. Connor Abbott has been developing a shader compiler, although it has not been hooked up to Lima yet. For the Mali 600 series, work has only recently begun, and so there is little progress to report.

Clark himself is the developer behind Freedreno, the driver project for Adreno GPUs. Adreno GPUs are found in Qualcomm Snapdragon SoCs, HP TouchPads, and several high-end Android phones. The GPU is available in two generations. The 200 series chips support OpenGL ES 2.0, while the 300 series chips support OpenGL ES 3.0 and Open CL 1.1. Both use a unified shader architecture, although there are differences between them. The 200 series uses VLIW instructions on vectors with the ability to co-dispatch scalars, while the 300 series uses explicitly pipelined scalar instructions.

As of now, the Freedreno driver is the furthest along of the free driver projects. There is an initial DRM/KMS driver in for kernel 3.12, and there is a working Gallium3D driver that supports both 200 and 300 series GPUs. The Gallium3D driver also works on the Kernel Graphics Support Layer (KGSL)/fbdev backend for Android. There is an X driver, too, which can utilize Adreno Z180 2D vector graphics cores. Freedreno currently implements support for OpenGL ES 1.0 and 2.0 and offers OpenGL 1.4 support on what Clark called "a best effort basis." That best effort is good enough to run GNOME Shell, XBMC, and several well-known 3D games like Xonotic and OpenArena. The DRM/KMS backend also supports the Weston compositor for Wayland.

Binary graphics drivers are a thorny issue: many free software supporters decry them, but they are a common sight—particularly on mobile Linux devices. Considering the rapid progress that has been made on free drivers for desktop GPUs in recent years, it can be easy to forget just how recently binary drivers were considered a necessary evil for that hardware as well. As Clark's talk illustrated, free drivers for ARM SoC systems still have a ways to go, but they are also making rapid progress, which should give hope to those who feel stuck with proprietary GPUs in their pockets.

Comments (21 posted)

A gathering of kernel developers

The now-traditional kernel panel, with yet another new set of participants, was held at LinuxCon North America (LCNA) in New Orleans, Louisiana on September 18. The questions ranged from the kernel development process to more personal queries about the kernel hackers on stage. It is a popular session (Linux Foundation Executive Director Jim Zemlin called it his favorite in the introduction) that helps give the audience a glimpse into the personalities of some of those who create and maintain the kernel that underlies their businesses and, perhaps, other parts of their lives as well.

Red Hat's Ric Wheeler moderated the panel, asking his own questions as well as some from the audience. The panel consisted of Sarah Sharp from Intel, who works on USB 3; Tejun Heo from Red Hat, who is mostly working on control groups and resource management these days, he said, but has also done work on workqueues and per-CPU data structures; and Greg Kroah-Hartman of the Linux Foundation who maintains several subsystems and is the stable kernel maintainer. Linus Torvalds rounded out the panel, noting that he no longer did any real work as he had "turned to the dark side": management. He just merges other people's patches these days, he said with a grin.

With much of the focus on embedded devices today, has the kernel moved too far in that direction and away from servers, Wheeler asked. Torvalds said that there was a good balance in the last merge window; while there were lots of commits for "wild wacky devices", there was also a lot of scalability work that was added. Heo mostly agreed, saying that the companies doing the kernel work are allocating their resources to generally maintain the balance. No one seemed to think there was a real problem or concern that servers would be ignored in favor of the latest smartphone.

Getting involved

Wheeler turned to the question of diversity within the community, asking if it had gotten easier for new developers to get involved. He note that Sharp has been leading an Outreach Program for Women (OPW) effort for the kernel and turned to her first. Sharp said that the OPW kernel effort came out of an interest in helping women find a bigger project, with a mentor to assist them in getting up to speed. Seven internships were awarded, and several of the participants were presenting at the conference, she said.

Kroah-Hartman said that he was an OPW mentor this year and was pleased to see some 60 patches from participants in the last merge window, including some to the TTY drivers. He also noted that there is a professor in the Czech Republic that is making "get a patch into the kernel" an assignment for his students. While it was difficult for some, a handful (3-5) said it was "easy" and would continue working on the kernel.

Sharp also pointed out that the documentation for getting started has improved, partly because of the tutorial that she wrote for OPW on creating a first patch and interacting with the community to get it merged. The participants in the program "are doing real work", she said, including speeding up the x86 boot process by parallelizing it.

While the kernel can be difficult to get involved with because of its complexity, it can also be easier to do so because of all of the different kinds of contributions that can be made, Torvalds said. People can contribute drivers, bug fixes, documentation, and so on, which gives more opportunities to contribute than some other open source projects. "Just look at the numbers", he said, since there are patches from more than a thousand developers every release, "it can't be that hard" to contribute.

It really just takes two hours to go into a file and look at it, Heo said, you will likely find things that are "stupid" in the code. That provides plenty of opportunities for new developers. He also noted the rise in the numbers of Chinese developers, which has been noticeable recently and is good to see. Wheeler said that there are few places in the world that don't submit patches to the kernel these days. Kroah-Hartman chimed in that Antarctica should be included in the list as patches had come from there in the past.

The first audience question was a bit whimsical. "Apparently there is an opening for a CEO at Microsoft", Wheeler said to laughter from both audience and panel, was there any interest in the position? The silence (or chuckles) from the panel members made it pretty clear what they thought of the idea.

The airplane seat-back entertainment system seems to be a popular choice for the "most embarrassing place you have seen Linux used". Normally, you only know that it runs Linux because it has crashed, as Sharp pointed out. Torvalds said that he hates seeing the kernel version numbers (like 2.2.18) that sometimes pop up when embedded Linux crashes.

What do you see for Linux beyond your lifetime was the next question up. Kroah-Hartman said that he wanted to see it continue to succeed, which means that it needs to continue to change. "If we stop our rate of change, we will die", he said, because hardware keeps on changing and Linux needs to keep adapting to it. "I can't argue with that", Torvalds said; he hopes that hardware innovation continues its pace.

On the other hand, Heo said that he has no long-term view because he "can't predict reality". He can't see much beyond getting the user-space interface to control groups into a usable form, "after that, who knows?". Sharp said she "would like to see a community that is welcoming to all" to applause.

Kernel and user space

Linux is an ecosystem, Wheeler said, so are there important user-space issues that concern the panel? "That's why we have the Plumbers conference", Kroah-Hartman said. That conference and LCNA overlapped on the day of the panel and Plumbers is an opportunity to put the kernel and user-space developers together to resolve any issues. Torvalds would like to see more kernel developers work in user space. He wouldn't necessarily like to lose them, but it would be worth it to see some of the kernel culture spread to user space. In particular, he complained about APIs breaking with every release; "we know how to do these things" correctly, he said.

But Heo said that the kernel developers could learn from user space as well. There is a need for more communication "between those using our features and us", he said. It would be "beneficial if we talked more". Sharp pointed to power management as one place where collaboration could be better. Linux could have the best power management of any operating system, but it will take cooperation between the kernel and user space.

Another audience question was mostly targeted at Torvalds. It is known that he is a diver and likes conferences that are near good diving locations, so what would be good locations for upcoming conferences? He suggested that "more conferences in the Caribbean" would be desirable. Heo suggested Hawaii as a destination as well. Both were unsurprisingly met with widespread applause.

The only one who responded directly (most just laughed) to the question of "have any of you been approached by the US for a backdoor?" was Torvalds. After a bit of a pause, he bobbed his head up and down in assent, while saying "NO!", which brought another laugh from the packed house.

How the panel members ended up involved in kernel programming was the next question on the agenda. Kroah-Hartman's girlfriend (now his wife) told him about sitting in on a talk with a "strange bearded guy" (Richard Stallman) about free software, which was all new to him. Some years went by and he eventually learned about Linux and got bored when his wife and daughter went away on a trip—so he wrote a driver. In that same vein, Sharp's boyfriend (now husband) was involved with open source rocketry, so she got involved with Linux to work on those rockets.

Torvalds said that lack of money is what drove him to kernel development—creation really. He couldn't afford to buy Unix, nor to buy games (he had to type them in), so he turned to kernel development out of necessity. Heo said that he was unsure why but always wanted to do operating system programming. He is from Korea and universities there did not offer operating system development without pursuing a Masters degree. That led him to Linux, where no one cares what degree you have: "if you can do it, you can do it".

The difficulties of being a maintainer was next up. Kroah-Hartman said that he had a whole talk on what makes his life hard, but it could be summed up as "read the documentation". One of Torvalds's major sources of stress is last minute pull requests—"and I am looking at you James [Bottomley]". Those make him hurry, which he hates to do. If the code isn't ready, just wait until the next release rather than making him hurry, he said.

How to interact with Torvalds (and others, like Kroah-Hartman) is not really documented, Heo said, and it takes six months or a year to come up to speed on that. Beyond that, maintaining a piece of the kernel is "not that hard", he said. For Sharp, getting patches without a justification is one of the hardest problems she deals with. Submitters need to convince her that the patch is actually needed; "why should I care?" If it fixes a bug or adds a feature, the patch should say that. Torvalds emphatically agreed with Sharp; submitters should not just send patches in a "drive-by" fashion, but be ready to answer questions and justify their patch.

Combining two questions, Wheeler asked about what the panel members did in their non-software time and what, if anything, might draw them away from Linux development eventually. Kroah-Hartman joked that Linux was once his hobby, but then he got a Linux job and lost his hobby. Lately he has been building a kayak. Sharp listed several things she likes to do outside of kernel hacking including bicycling, gardening, and fantasy gaming (e.g. Magic: The Gathering and Dungeons & Dragons).

Diving and having a "regular life" occupy Torvalds. He doesn't see anything "coming along that would be more interesting than Linux". Nothing else would "fill that void in my life". Heo is also happy in his job and "could not imagine" doing anything else for work. He has recently moved to New York City, so he spends time "hanging out in the city" and "trying to have a life", he said with a grin.

Working with the community

The final question came out of what is sometimes called "The Greg and Jim show", which is when Kroah-Hartman and Zemlin travel to different companies to talk with them about how to get better at working with the kernel community. Zemlin retook the stage to ask the panel if they had advice for those companies' engineers. Getting involved with the community early in the hardware design phase is important, Kroah-Hartman said. Some companies understand that, to the point where code for an Intel chip that never shipped had to be ripped out of the kernel a few cycles ago, he said.

Torvalds said it goes beyond just looking at the problems that come from your company's new hardware, you need to look at the bigger picture. The "perfect solution" for a specific problem may not be useful for others with similar problems. Features need to be added "in a way that makes sense for other people".

Heo said that it is important for companies to budget extra time and resources to work with upstream. Some think they can "send it and forget about it", but it doesn't work that way. Sharp said that companies need to design their changes more openly; don't design a new API behind closed doors. Instead, working with the kernel developers on the design will help build up the trust, which eases work with the community.

As a closing note, Zemlin not only thanked the panel and moderator, but also reported on a conversation that he recently had. He has spoken with the airline entertainment system company and reassured everyone that it would be upgrading its kernel "soon". With luck, perhaps, that means it will crash less often too.

[ I would like to thank LWN subscribers for travel assistance to New Orleans for LinuxCon North America. ]

Comments (3 posted)

A SPDX case study

At LinuxCon North America in New Orleans, Samsung's Young-taek Kim described his company's experience rolling out support for the Software Package Data Exchange (SPDX) standard in its product development tools. SPDX, of course, is a data format for tracking software components, licenses, and copyrights. The company was able to improve its efficiency regarding license compliance, but that was not the only benefit to the program. The implementation team also came away from the experience with feedback for several ways to improve the SPDX specification itself.

Why SPDX

Kim is an engineer in Samsung's Open Source Initiative (OSI) team. Like the open-source groups inside many large corporations, the team is charged (in addition to its development duties) with educating and guiding other units in the company about open source principles. Kim gave a quick overview of SPDX before describing the OSI team's task and where SPDX fit into Samsung's workflow.

The SPDX specification is designed to produce a standardized "bill of materials" for an open source software package, he said. It communicates the licenses and copyrights that make up a package—including, importantly, packages that are derived from multiple sources. A constant problem in business scenarios is making sure that one's company gets good information about these factors from software suppliers and subcontractors. It is common, he said, for a supplier to say simply "this is open source" and provide no further information. The package could be MIT-licensed or under the GPL, but if one does not know which of those licenses it is, one does not know how to comply with it.

In practice, Kim said, he has often manually vetted a package by looking through the source. He does not mind this process, but it clearly results in duplication of effort when multiple project teams in multiple divisions repeat the vetting for a package that is already in use elsewhere. Standardizing the license and copyright information with SPDX lets the company create a central database to unambiguously keep track of the packages it has already vetted, and it helps resolve complex compliance questions that arise from combining multiple packages. Both benefits were of interest to Samsung.

Samsung's pilot program

Kim explained that Samsung wanted to reduce the overhead of license compliance, so it charged the OSI team with deploying SPDX data interchange in a pilot program inside the company. He then described Samsung's existing open source compliance process. The company breaks the process into four steps: discovering an open source package, developing a product with the package, verifying the obligations imposed by the open source license, and releasing the appropriate material to satisfy that obligation.

The SPDX pilot program was charged with improving those final two steps. Before the program, the verification stage meant confirming the license on a package by having a human read through the source, which is time-consuming, often redundant (such as when the same package has already been verified by a different product team), and prone to error. Human beings, he said, can reach different conclusions when reading the same code. The obligation-satisfaction stage was also largely manual (e.g., a person having to post source code on a public Samsung web site, make it available to customers, or insert a copyright statement onto a product screen) and could be expensive (especially when printing a source code offer in a user manual was involved—and even more expensive when re-printing is necessary).

The pilot program's first goal was to reduce the time lost to re-verification. The OSI team developed a tool called AIRS to identify software packages and verify their license and copyright in SPDX format. AIRS started out with a command-line interface, but is also usable as a Java library. It uses the Protex code verifier from Black Duck to scan a package and pick out license and copyright information. It then exports this information as SPDX data, including the licenses and copyrights of all components and (perhaps most importantly) the "concluded license" that applies to the combined work as a whole. It identifies files by SHA1 checksum, which helps catch duplicates—meaning that files which have already been scanned and analyzed once do not need to be re-scanned even when directory structures have been rearranged.

The eventual design is for AIRS to store this SPDX data in a central, company-wide database, which can then be queried whenever a new (or a duplicate) package is imported for testing. Right now, teams within the company exchange SPDX information internally using other tools. However, the chief benefit of AIRS is that it can identify the correct license and copyright of a package automatically. Even for a small development team, that demonstrably saves time.

The second goal of the pilot program was to simplify the obligation-satisfaction step, Kim said. For this, the OSI team developed a web tool (tied in to AIRS) that can automatically publish the appropriate license notice for a package on the company's web site. It generates the page for each package based on the stored SPDX data, and even generates a QR Code containing a link to the license page URL. Samsung intends to start putting these URLs on physical product packaging, perhaps as soon as October.

SPDX in the future

Overall, the company was quite happy with the pilot program, Kim said, so work is continuing. The AIRS centralized SPDX database is the first order of business, but there are several other to-do list items. One is support for verification engines other than Protex; another is the ability to identify the same code snippet even when the file checksum changes. The OSI team also wrote its own SPDX parser when developing AIRS, which Kim said he hopes to release as an open source project in its own right.

In reply to an audience question, Kim said that the company may start requiring external software suppliers to provide SPDX data on the packages that they supply. What makes that request tricky is that Samsung is still responsible for verifying that the information is correct, so it will probably have to use AIRS to process the suppliers' code anyway.

Despite its general satisfaction, Samsung ran into several problems with SPDX itself when running its pilot program. First was the "Artifact of Project" property (defined in sections 6.8 to 6.10 of the SPDX specification [PDF]), which is meant to indicate that the file in question belongs to a specific project. In the specification, the cardinality of this property is "one," so a given file can only be associated with a single project. Samsung found that insufficient to record projects that constitute combined works, and had to modify its SPDX output to list every project that a file belongs to.

The property also requires parent projects to be described with the Description of a Project (DOAP) format, which duplicates the same RDF/XML data for every file in a project—a simple database reference would save space. In addition, Kim said, Samsung found it problematic that SPDX does not account for sub-projects within a project, which is a common situation when creating large products. It also ran into problems caused by the fact that SPDX does not enforce a common rule for the formatting of file paths; packages can reference files with relative path names, which makes it difficult to match them up for the purpose of determining the concluded license. Requiring the file paths be normalized would simplify things.

SPDX is often touted for its ability to ensure correctness in license-compliance efforts, so it is interesting to see that it can enable other benefits, too, such as reducing the amount of duplicated work undertaken by developers. Samsung is an enormous company, so even saving a small amount of time on a per-project basis can add up to a lot.

[The author would like to thank the Linux Foundation for assistance with travel to New Orleans.]

Comments (3 posted)

Page editor: Jonathan Corbet

Inside this week's LWN.net Weekly Edition

• Security: Encouraging a wider view; New vulnerabilities in chromium, kernel, policykit, tiff, ...
• Kernel: Split PMD locks; A perf ABI fix.
• Distributions: Fedora 20 takes shape; openSUSE, SteamOS, Tails.
• Development: OpenGL debugging; Lightning 2.6; GStreamer 1.2; GNOME 3.10; ...
• Announcements: Studio Storti joins TDF, events.

Next page: Security>>