LFCS: Linux penetration, Yocto, and hardware success stories

The first day of the Linux Foundation Collaboration Summit (LFCS) tends to have a wide variety of talks and panels that range all over the Linux ecosystem. This year's edition didn't disappoint, though the topics seemed a bit more focused toward the embedded space and Linux hardware support. There was also the always interesting kernel panel, which we cover elsewhere. LF executive director Jim Zemlin kicked things off with a look at Linux penetration along with some thoughts he has on where Linux (and the LF itself) is headed.

While he didn't reprise his "it's like kicking a puppy" comment regarding Microsoft bashing in his LFCS keynote, he did mention it earlier in the week at a Camp KDE talk. Kicking anything is not in Zemlin's immediate future, however, as he gave his talks from a wheelchair—"on heavy narcotics"—due to an ugly looking break in his leg from a skiing mishap. He showed before and after x-rays that made it very clear why no one will want to "get behind me at airport security".

Zemlin went through some examples of eye-opening places where Linux is running, including air-traffic control systems, supercomputing, nuclear submarines, and so on. He noted that 72% of the world's equities are traded using Linux (and that may be low as some exchanges are not releasing any information about their platform). Linux first appeared on the list of the top 500 supercomputers in 1998, by 2004 Linux was running over half of the raw computing power on that list; now that number is 95%.

Embedded and mobile devices are also an arena where Linux has made huge strides. The mobile space, in particular, is "massively hedged" because there are multiple big bets on Linux in that area (i.e. Android, MeeGo, WebOS, and others). He also said that there is "no declared winner" in mobile and that while he has no real predictions what the mobile landscape will look like in three to five years—they would "inevitably be wrong"—he is sure of one thing: the leaders will be running Linux.

Other trends worth watching in the future according to Zemlin are things like pre-integrated, minimal configuration computing based on Linux. Those systems will be for both the enterprise and consumer markets. They will come with pre-configured applications that are ready to use out of the box. Another trend is toward specialized high-performance computing systems as typified by "Watson", which is IBM's Linux-based supercomputer targeted at "solving" the Jeopardy game show. He did note that he thought the humans could have won if one of the answers (in the form of a question, of course) contained the commands used to format Linux disks.

"Don't believe a lot of the FUD [fear, uncertainty, and doubt]" that is being spread by Linux competitors, Zemlin said. There is an ongoing effort to discredit Linux and various Linux-using companies on the basis of copyright and patent issues. It is not very important in terms of Linux adoption, but more than that, it is not open-source-specific as any platform can be attacked in this way; it's based on how successful the platform is. It will lead to efforts to reform the patent system and increase the quality of patents that are issued, he said, but it won't slow down the adoption of Linux.

There are a number of changes at the LF, which Zemlin passed along, including the merger with the CE Linux Forum (CELF), which was announced in October, the first annual Android Builders Summit to be held the week following LFCS, and a new LinuxCon for Europe, which will be held in Prague in October. There is also a new high availability workgroup within the LF that was announced as part of LFCS as well as the release of Carrier Grade Linux 5.0.

Yocto project and panel

The release of Yocto 1.0 was another announcement that was made. The Yocto project is a collaborative effort to simplify the process of creating an embedded Linux distribution under the LF umbrella. There are multiple companies participating, "many of them are fierce competitors", but they see the advantage of working together on these tools, Zemlin said. The next breakthrough device or gadget will very likely be built on Linux, because no other platform has the breadth of architecture support. The role of the LF is not to find that product or the person who is building it, but help the community provide the best tools to create those products. "That's what's going on in the Yocto project", he said, and it is "all going on in the open".

The update on the project was a transition to a panel discussion that Zemlin led with three embedded industry veterans: William Mills of Texas Instruments, Mark Hatle of Wind River, and Steve Sakoman of Sakoman, Inc., which is an embedded consulting company. The overall theme of the discussion was collaboration—not surprising given the name of the conference—but collaboration is not something the embedded industry, or embedded Linux, is known for.

According to Mills, it is "far too difficult" to create embedded Linux distributions currently, but that Yocto changes that. It is important that Yocto is based on OpenEmbedded as there are a number of companies that already use those tools, and Yocto is not trying to start from scratch. Sakoman agreed that building atop OpenEmbedded is the right way to go, but that Yocto adds some credibility that was lacking because customers didn't "see any big names behind OpenEmbedded". Hatle pointed out that having each company create and maintain its own build tools doesn't make sense and is a waste of money, "let's do it once and share the results", he said.

The companies involved are still going to compete Hatle said, they will "just compete on a higher level, not on the version of grep" that one is shipping. Vendors can add better integration or better debugging, he added. Sakoman concurred, saying that there will always be opportunities available for companies that can assist in embedded development, particularly because the time to market for products is so short.

Embedded Linux is definitely important these days and only getting more so. Zemlin noted that a while back he was getting a lot of calls from companies looking to hire talented embedded Linux developers. More recently those calls have changed to: "Do you know any companies I can acquire to get Linux talent?"

When asked about their "dream participant" in the project, both Hatle and Mills pointed to the smaller players, who can't afford to hire companies like Wind River, as the ones they would like to see get involved. Mills said that there were more silicon vendors and operating system companies involved than he expected, but that he would like to see more community projects, like OpenWRT for example, switching to use Yocto. It is a highly usable system, Hatle said, that "works right out of the box", which is something that he has never seen for embedded Linux before.

Hardware success stories for Linux

Later in the day, Greg Kroah-Hartman moderated a panel targeted at hardware success stories for Linux. It featured Jason Kridner of Texas Instruments (TI), Mark Charlebois of the Qualcomm Innovation Center (QuIC), and Dirk Hohndel of Intel, who discussed their companies' reasons for supporting Linux, along with a number of other hardware-related topics.

Kroah-Hartman started things off by asking why it is that each company decided to contribute to Linux. Charlebois said that "Linux is a core part of the mobile ecosystem", which made it important to Qualcomm's customers. QuIC had worked out of tree for a long time and "felt the pain of doing that", so it needed to start working upstream. Hohndel noted that Intel has a "pretty long track record" of working with open source and that is because "open source is the shortest path to innovation". Kridner said that TI wanted to make its "huge investment in mobile processors more easily available" to its customers. Linux enables that and "a whole lot more things than just mobile phones", he said.

How the panelists make the case for working upstream was the next topic tackled. It is "easy when customers are asking for it", Charlebois said. Building up a set of out-of-tree patches is "too painful" because integrating them with the upstream kernel takes too much time. Hohndel said that it is the financial argument that "really resonates with upper management". If you look at the costs of porting things forward, backward, and sidewards, he said, you will find that working upstream makes the most sense. "Fundamentally it is in your own best financial interests". Kroah-Hartman noted that he was happy to hear that, as he had always hoped it was true.

From the audience, Jonathan Corbet asked about Charlebois's statement that "the mobile space is about proprietary drivers", which he had made in a talk earlier in the day. Corbet noted that having proprietary drivers goes against all of the points described above. Charlebois said that it is "not always possible to open source" drivers due to third party intellectual property (IP). Kroah-Hartman pointed out that by doing so, QuIC and others were "weighing two legal risks" as some kernel developers would claim that distributing binary-only drivers violates their copyright.

While Charlebois didn't seem to have much more to say about that, Hohndel described the situation as a "rock and a hard place". When a vendor buys IP blocks, the third parties they purchase them from often have "strange ideas of GPL compliance". There are lots of constraints that go into building SoCs, he said, and vendors have to deal with the real world, not the one they wish they lived in. He noted that Intel had done a lot of work to get acceptable drivers for PowerVR-based graphics devices into the kernel.

Hohndel said he is pretty happy with how the staging tree is working out, in response to a question from Kroah-Hartman. He would rather have solid drivers developed, but that doesn't always happen, so the "halfway house" provided by the staging tree is valuable. It allows the community to help improve the drivers, which has been happening, "and it's awesome", he said. But the staging tree is "not a panacea", has failed in some places, and doesn't solve all the problems.

Making more open documentation available to the community was the topic of another question from the audience. Kridner said that documentation is "fantastic" and that TI is always improving theirs, but that personal contact with the engineers involved with a particular feature is even more important. Hohndel said that documentation is "incredibly expensive to produce" and is legally difficult if third parties are involved. Getting the driver is usually easier because there needs to be a belief that documentation will bring in more customers before it will be produced.

Grant Likely asked whether hardware companies felt that there was a lack of input from Linux developers, which was something that came up (in the context of disks) at the recently concluded Linux Filesystem, Storage, and Memory Management Summit. Hohndel said that Intel gets a "ton of direct feedback from community members" and was "very happy that we get it". Kridner said that TI gets input on device architectures from a lot of different places, while Charlebois said that QuIC is focused on customer requirements. Kroah-Hartman noted that both Intel and IBM have gotten kernel developers together with their employees at various times and the kernel hackers will give "unabashed input" on hardware architectures.

There were, of course, several other talks and panels on the first day, and one cannot help but be struck be the number of competing companies that shared the stage. Obviously, Linux and open source are providing a platform not just for those companies to build products with, but also a place for them to collaborate, something which shouldn't be much of a surprise at a summit that is targeted at accelerating that process.