LWN.net Logo

Updates from Linaro

By Jake Edge
May 25, 2011

Over the last month or so, I have sat in on a few different talks about Linaro and the work that it is doing to improve the ARM Linux landscape. While Linaro is a consortium of six major ARM companies, its work is meant to be available to all ARM companies, developers, or users. The focus may be on the needs of its member companies, but Linaro's influence and efforts are likely to spread well beyond just those needs. It is an organization that wants to try to keep the best interests of the entire ARM ecosystem in mind—perhaps that's not so surprising with both ARM, Ltd. and major ARM silicon fabricator IBM on board.

In the year since Linaro began to take shape, and roughly eleven months since it was announced to the world, the organization has expanded its scope, changed to a monthly release cycle, and stepped in to to try to help head off a crisis in the ARM tree. It has also made progress in many of the problem areas (kernel, toolchain, graphics, ...) that it set out for itself. But there is, of course, lots more to do.

Linaro Developer Summit

[George Grey]

Linaro CEO George Grey spoke briefly at the opening of the Linaro Developer Summit (LDS), which was co-located with Ubuntu Developer Summit in Budapest, and described some industry trends and things he sees coming for Linaro. Grey noted that the last twelve months have shown some "extraordinary changes" in the use of open source software in real world products. "Android in particular has startled a lot of people", he said. The Android share of the smartphone market has risen from 5% to 25% in that time span, he said, which is something that has never happened before.

Device manufacturers are no longer happy to get a board support package (BSP) that is out of date and requires a BSP-specific toolchain. Instead, they are looking for product-quality open source platforms, which is something that Linaro is delivering. The Linaro automated validation architecture (LAVA)—a test and validation platform—will be very important to that effort as it will help increase the quality of the software that Linaro delivers.

Getting development platforms into the hands of open source developers is another area where Linaro can make a difference, Grey said. In the past, it has been difficult for smaller players to get their hands on these development platforms because it is hard to get the attention of the system-on-chip (SoC) vendors. But, these days there are development platforms at reasonable prices for ARM SoCs from Texas Instruments, Freescale, and ST-Ericsson (all Linaro members), with more coming.

That solves the hardware problem, and Linaro will be providing software to run on those boards. That means that companies or developers with limited budgets can get complete hardware with multiple I/O devices and a product quality software stack. Grey is excited to see what the community can do with all of that.

Grey also announced that there would be no Linaro 11.11 (it had been making releases based on Ubuntu's schedule, though delayed by one month) in favor of monthly (or more frequent) code drops. The various Linaro working groups will be continuously pushing their code upstream, while there will be daily releases of some code, mostly for internal use. There will also be monthly supported releases of the kernel and toolchain for external use.

Looking to the future, Grey noted that work was progressing on support for the Cortex A15 processor. The intent is to have the processor supported by Linux when it releases, rather than a year or two later as has been the case in the past. He also said that there "clearly is going to be a market for ARM-based servers", and Linaro is doing some early work on that market segment as well.

[Christian Reis]

Linaro VP of Engineering Christian "Kiko" Reis spoke after Grey, mostly about the nuts and bolts of how LDS would operate, and how attendees could get the most out of it. He also looked back over Linaro's first year, noting that a lot of progress had been made, particularly in the areas of communication and collaboration between various ARM vendors. Starting Linaro wasn't easy, he said, but there are harder things still to be done, including bringing the whole ARM community together.

Specifically, Reis mentioned the ARM kernel consolidation work that still needs to be done. Linaro "spent a year not really doing this", and now is the time to get it done. Determining the right course for memory management for embedded graphics devices is also high on the list. Delivering that will take more than just a year, but planning for that is one of the priorities for the week.

The organization of Linaro

David Rusling, Linaro CTO, gave a talk at the Embedded Linux Conference back in April to give an overview of the organization, highlight some of the accomplishments in the first ten months, and look ahead to some future plans. Rusling started off by clarifying that Linaro is meant to be an engineering organization and not a "tea and cakes in Honolulu standards thing". He also pointed out that "Linaro" was the "least hated name" among the candidates, which mirrored Dirk Hohndel's explanation earlier in the day of the choice of the name "Yocto" for that project.

Rusling said that he recognized that the fates of ARM and Linux were intertwined in 2009 or so, when he was at ARM, Ltd., but there were lots of companies "running around" and not collaborating. There was a clear need for more collaboration on the things that were common between various SoCs, but there were not enough engineers working on those problems. Linaro was born out of that need.

Linaro started out with around 20 engineers and was envisioned as an "upstream engineering" organization. The "genius move" was to do all of that in the open, he said. Everything is on the wiki, though some things may be hard to find, and "upstream is our mantra".

Much of what Linaro does is "social engineering", Rusling said. There are a number of misconceptions about Linux and open source that need to be dispelled, including the idea that open source is difficult to deal with. There are gatekeepers in Linux and other projects that have strong views, but interested organizations and vendors simply need to "engage with them". The "really bad false statement" is that open source is cheaper. That's not really true as working in the open source communities requires a deeper involvement because it's all about influencing development direction; there is no control, he said.

The six member companies want to drive the technical agenda for Linaro, which does its work through the working groups that have been established. Those working groups are "very independent" and the member companies aren't trying to run the projects directly, but are instead allowing the groups to work upstream on solving problems in their areas.

There is also a platform group that builds, tests, and benchmarks the work that is done by the working groups (and upstream projects). The idea is to prove that new kernel features work or that tool changes make things go faster by creating and testing evaluation builds. "Any time you do any changes you have to measure" the impact of those changes, he said. There are also landing teams for each of the member SoC vendors (Samsung, Texas Instruments, ST-Ericsson, and Freescale) that take the outputs from the platform team and turn it into usable builds for their customers. The landing teams are the only teams in Linaro that are closed to community participation.

It is not just kernel work that lands upstream, as the toolchain working group is doing a lot of work on GCC and other tools. Support for ARMv7a, Thumb 2, Neon, and SMP are being added to GCC 4.7, which won't be released until April 2012, and won't get into distributions until October 2012 or so, sometime after the 4.7.1 release. In the interim, the toolchain group will be making "consolidation builds" that can be used by ARM developers prior to the GCC release. In addition to work on GCC, the group is also adding functionality to tools like gdb, QEMU, and valgrind, he said.

After the first release in November, two new working groups were added to address graphics and multimedia issues. In addition, the other working groups started looking at long-term problems, Rusling said. The kernel group started adding device tree support for all of the Linaro members' hardware. Work on vectorizing support for GCC was one focus of the toolchain group, and the power management group started tackling segmented memory so that portions of the memory can be powered down. All of those things are "tricky areas that require a lot of coordination within the ARM space, but also upstream", he said.

For multimedia, much of the work involves testing, benchmarking, and tuning various codecs for ARM. Standardizing on the OpenMax media libraries and the GStreamer framework is the direction that Linaro is going. Android has gone its own way in terms of multimedia support, he said.

Rusling, like Grey, also pointed to the work being done on LAVA as something that is very important to Linaro, but also to the community. It is a "completely open" test and validation system that could be used by others in the ARM community or beyond.

There were some hard lessons learned in the first year or so of Linaro's existence, Rusling said. It is difficult to build a new engineering organization from scratch with engineers donated from multiple companies. On the other hand, people thought that the ARM community couldn't collaborate but Linaro has shown that not to be the case. Everything takes longer than he would like, and there is still a lot to learn, he said. "Open source is wonderful", but there are challenges to using it.

It is clear that ARM has become a very important architecture for Linux, and is completely dominating the low-power mobile device market. That doesn't look likely to change anytime soon, and it may be that ARM's efforts to move into the server space will bear fruit in the next few years. Certainly power consumption (and the associated heat produced) are very important not just in pockets, but in data centers as well. Linaro has, so far, been making many of the right moves to ensure that ARM is well-supported—and maintainable—in Linux. It will be interesting to see what the next year (and beyond) hold.


(Log in to post comments)

Updates from Linaro

Posted May 26, 2011 4:18 UTC (Thu) by aryonoco (subscriber, #55563) [Link]

It's interesting to note the absence of Qualcomm and Nvidia in this effort. While TI and Samsung are important ARM SoC makers, Qualcomm's Snapdragon (in various forms and guises) still dominates the Android space, and in the tablet space, Nvidia has had a good headstart over others.

Historically, both Qualcomm and Nvidia have been very reluctant participants in the Linux ecosystem, both keeping their drivers proprietary to the heart. Working upstream has been anathema to Qualcomm. If members of Linaro can show a demonstrable benefit from their involvement in this project, in an open and transparent manner, it will be interesting to observe Qualcomm and Nvidia's reaction.

Updates from Linaro

Posted May 30, 2011 15:40 UTC (Mon) by kiko (subscriber, #69905) [Link]

It's very insightful to highlight that one way to get Qualcomm and Nvidia to join Linaro is by showing them the benefit of working in a more transparent way. It's rather unfortunate that Android has shown them the benefit of doing exactly the opposite -- or maybe less sarcastically, that sales volume trumps being open.

I am currently spending a lot of braintime thinking about ways to highlight how SoC vendors benefit (or survive) in the long run by openly collaborating upstream, following trunk closely and engaging with distribution vendors to support their hardware. I'd definitely appreciate help with ideas that support or exemplify those claims.

Updates from Linaro

Posted May 26, 2011 6:18 UTC (Thu) by Cato (subscriber, #7643) [Link]

The thing I like the most about this is the test and validation work - if this can improve the quality of Linux on ARM, it may catch cross-platform issues such as KMS regressions (compared to pre-KMS), making X actually work reliably again, and power management regressions (http://www.phoronix.com/scan.php?page=article&item=li...)

Updates from Linaro

Posted May 28, 2011 1:50 UTC (Sat) by plars (subscriber, #7736) [Link]

We have a validation tools team at Linaro working on the infrastructure to do automated testing. We currently do daily automated testing and benchmarking on several of the boards we support, with more boards on the way over the coming weeks and months. If you're interested in finding out more about this, or anything else Linaro is working on, I encourage you to attend the public plan reviews next week (https://wiki.linaro.org/Cycles/1111/PublicPlanReview)

Updates from Linaro

Posted May 26, 2011 6:57 UTC (Thu) by grahame (subscriber, #5823) [Link]

I set up Linaro on my i.MX53 development board last week. Set up went quite well, but there's annoying problems once you get it going. Some of these are probably down to inexperience and can be hacked around. The most annoying is that the uboot update script from Ubuntu doesn't understand Linaro's kernel versioning scheme, so when you try to modify your uboot setup or upgrade your kernel you get 'not updating, wrong subarchitecture' (not the exact message.)

I wish it was based on Debian rather than Ubuntu, but I suppose Ubuntu's more frequent release cycle is useful to them.

Updates from Linaro

Posted May 26, 2011 18:46 UTC (Thu) by rsalveti (subscriber, #42934) [Link]

This was caused by bug https://bugs.launchpad.net/bugs/721147, that we already integrated on our own repositories (PPAs). Fix in Ubuntu itself should come in the following weeks.

If you grab current image this issue should be gone.

Updates from Linaro

Posted May 27, 2011 6:45 UTC (Fri) by grahame (subscriber, #5823) [Link]

Great, thanks!

Updates from Linaro

Posted May 27, 2011 13:53 UTC (Fri) by grahame (subscriber, #5823) [Link]

This is probably the entirely wrong place to ask about this but I thought I'd mention; I just installed these images and flash-kernel is still broken:
hwpack_linaro-lt-mx53loco_20110527-0_armel_supported.tar.gz
linaro-natty-alip-tar-20110527-0.tar.gz

I commented out the 'exit 0' in the check and everything works OK now.

Updates from Linaro

Posted May 26, 2011 7:25 UTC (Thu) by mjr (guest, #6979) [Link]

Don't know how feasible it is, but would be nice if Linaro could exert some pressure to open up the ARM-bundled GPUs a bit...

Updates from Linaro

Posted May 26, 2011 18:46 UTC (Thu) by kiko (subscriber, #69905) [Link]

We definitely are trying. I've had multiple meetings with directors from both ARM SoC vendors and GPU IP vendors, but so far it's been meetings and not much else. I've discussed sharing of specifications, independent driver development and the marketing (and quality) advantage of a fully open source solution, but they still don't seem to understand the integration challenges open source presents, and are paranoid about IP lawsuits which might be well-founded given the amount of GPU-related patents approved and the amount of IP that's gone around in mergers acquisitions.

What sort of argument should we present them, and are there other things we could do to improve the GPU situation on ARM?

Note that David Airlie and I ran a session on this at Southern Plumbers; there's a video of it at http://blip.tv/linuxconfau/embedded-gpus-for-arm-open-dis...

Updates from Linaro

Posted May 27, 2011 12:21 UTC (Fri) by mjr (guest, #6979) [Link]

Thanks, glad to hear that it's at least pursued. Sadly, I don't have any magic bullet arguments to suggest.

Updates from Linaro

Posted Jun 7, 2011 7:19 UTC (Tue) by alison (✭ supporter ✭, #63752) [Link]

Arnd Bergmann of Linaro/IBM said at Linux Collaboration Summit that he expects to see an open GPU driver released in 2011. He doesn't strike me as a wild prognosticator, so he must know that one of the major vendors is considering a release their driver code. Once one vendor has done so, if they aren't sued to bits by others, perhaps we'll see real progress with GPUs.

I wonder how releasing the drivers would affect the GPU designers' business. On the one hand, they can monopolize the support and tools business around their closed products. On the other hand, the closed driver model may limit adoption of their products and will prevent community contribution of patches and other valuable ideas.

I read somewhere that Nvidia was actively collaborating with the developers of the Nouveau open-source drivers for their products. And not too long ago, Broadcom joined the Linux Foundation. TI is all in on open-source, so they may be pressuring Imagination Technologies about their drivers. So I think there's a glimmer of hope on the GPU front if no good news just yet.

Updates from Linaro

Posted May 26, 2011 23:06 UTC (Thu) by wookey (subscriber, #5501) [Link]

People are exerting pressure. Engineers understand what a PITA non-free drivers are, and what the vendors are missing out on by only providing an API, not instruction-level info, and I (and others) agitate within ARM every chance we get, so the people with the ability to do something about it are hearing the message regularly. But none of them have changed their minds yet. It'll take a while, same as it did on the desktop, and change is not inevitable.

Some people are scared about patents and loss of 'IP' control. Some believe their software is better than everyone else's so they don't want people copying it (failing to understand the difference between driver code and instruction set specs there). Several don't believe that outsiders could write drivers that were any good.

Updates from Linaro

Posted May 27, 2011 1:29 UTC (Fri) by aryonoco (subscriber, #55563) [Link]

You could point out to them that if we've been able to write a kernel that's good enough for them, we should be able to write drivers that are good enough as well.

As for patents, you could point out that whether their product has open source drivers or not has no bearing on whether it's infringing any patents. They can keep their drivers as close as they want and still be hit with patent lawsuits.

In the desktop and server space, there are many huge open source programs these days, programs that are essential in the enterprise market, and none of them have been hit with patent lawsuits any more than proprietary software of similar nature has been. I fail to understand the logic that assumes opening up code suddenly makes one more vulnerable to patent lawsuits.

Updates from Linaro

Posted May 27, 2011 11:00 UTC (Fri) by stevem (subscriber, #1512) [Link]

The usual argument is that a lot of the patented stuff is not immediately visible with closed drivers, whereas if people give out source showing how various features of a GPU work then it makes it easier for others to spot infringements.

The GPU area is a total mess because of the patent bollocks: as always, the only people benefiting are the lawyers.

Updates from Linaro

Posted May 27, 2011 11:53 UTC (Fri) by wookey (subscriber, #5501) [Link]

Indeed - that is excactly the argument given. Clearly code licencing has exactly zero effect on infringement status, but nevertheless people feel that at least if they only ever ship binaries it's harder to point the finger at any particular infringement. In practice there is probably some truth in that, but it seems a pretty ropey scheme to base a major plank of your business on.

Of course whilst we'd like open drivers we're not actually asking for them to hand over the source - just some specs so we can write our own. I think the fear there is that writing down enough detail to program the device also shows fairly clearly which patents might be being infringed. I have no idea of the truth or otherwise of that as my expertise in not in GPUs and their architecture/instruction sets.

Copyright © 2011, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds