A conversation with Linus at LinuxCon Japan

Linus Torvalds rather famously does not like speaking in public, so his conference appearances tend to be relatively rare and unscripted. At LinuxCon Japan, Linus took the stage for a question-and-answer session led by Greg Kroah-Hartman. The result was a wide-ranging discussion covering many issues of interest to the kernel development and user communities.

3.0

The opening topic was somewhat predictable: what was the reasoning behind the switch to 3.0 for the next kernel release? The problem, said Linus, was too many numbers which were getting too big. A mainline kernel release would have a 2.6.x number, which would become 2.6.x.y once Greg put out a stable release. Once distributors add their build number, the result is an awkward five-number string. Even so, we have been using that numbering scheme for about eight years; at this point, the "2.6" part of a kernel version number is pretty meaningless.

Once upon a time, Linus said, making a change in the major number would be an acknowledgment of some sort of major milestone. The 1.0 kernel was the first to have networking, 1.2 added support for non-x86 architectures, 2.0 added "kind of working" SMP support, and so on. We used to think that incrementing the major version number required this kind of major new feature, but, in the 2.6.x time frame, we have stopped doing feature-based releases. The current development process works wonderfully, but it has caused the 2.6.x numbering scheme to stick around indefinitely. As we approach the 20th anniversary of the Linux kernel, we had a good opportunity to say "enough," so that is what Linus did.

"3.x" will not stay around forever - or even until the kernel is 30 years old; Linus said he expects to move on around 3.20 or so.

Linus noted that some people were thinking that 3.0 meant it was time to start with big new features (or removal of old code), but that's not what is meant to happen. It is just a number change, no more. Trying to have the kernel be stable all the time has, he said, worked very well; that is not going to change. Greg was clearly happy with this change; he presented Linus with the bottle of whiskey he had promised as a token of his appreciation. After debating opening it on the spot (Greg brought paper cups too, just in case), they decided it might be best to finish the discussion first.

Greg asked: what recent changes did he like the most? Linus responded that he tends to like the boring features, things that people don't notice. Performance improvements, for example; he called out the dcache scalability work as one example. There is no new interface for users, it just makes the same old stuff go faster.

Features and bloat

Is the kernel, as Linus famously said on a 2009 panel, bloated? Linus acknowledged that it is still pretty big; it couldn't possibly run on the machine that he was using to develop it 20 years ago. But even phones are far more powerful than that old machine now, so nobody really cares. The kernel has been growing, but that growth is generally necessary to meet the needs of current hardware and users.

What about the addition of features just for the heck of it - new stuff that is not strictly driven by new hardware? Is that something that we can still do? Linus said that there are certainly developers working on features with no current users, thinking five years or so into the future. Sometimes that work succeeds, sometimes we end up with code that we regret adding. Linus said that he is increasingly insisting on evidence that real users of a feature exist before he is willing to merge that feature.

Greg asked about control groups, noting that a lot of kernel developers really object to them. Linus responded that control groups were a feature that did not initially have a whole lot of users, but they do now. Control groups were initially added for certain specific server setups; few others had any interest in them. Developers were unhappy because control groups complicate the core infrastructure. But control groups have begun to find a lot of users outside of the original target audience; they are, in the end, a successful feature.

Symmetric multiprocessing (SMP) was also, at the beginning, a feature with few users; it was a "big iron" feature. Now we see SMP support used across the board, even in phones. That illustrates, Linus said, one of the core strengths of Linux: we use the same kernel across a wide range of platforms. Nobody else, he said, does things that way; they tend to have different small-system and large-system kernels - iOS and Mac OS, for example. Linux has never done that; as a result, for example, there has never been a distinct cut-down kernel for embedded systems. Since the full kernel is available even at that level, Linux has been a real success in the embedded world.

Embedded systems, world domination, and the next 20 years

Continuing with the topic of embedded systems, Greg asked about the current furor over the state of the ARM tree. Linus responded that developers in that area have been a bit insular, solving their own problems and nothing more. That has resulted in a bit of a mess, but he is happy with how things are working out now. As a result of pushback from Linus and others, the ARM community is beginning to react; the 3.0 kernel, Linus thinks, will be the first in history where the ARM subtree actually shrinks. The embedded world has a history of just thinking about whatever small platform it's working on at the moment and not thinking about the larger ecosystem, but that is changing; this community is growing up.

Many years ago, Greg said, Linus had talked about the goal of "total world domination" and how more applications were the key to getting there. Is that still true? Linus responded that it's less true than it used to be. We now have the applications to a large degree. He also no longer jokes about world domination; it was only funny when it was obviously meant in jest.

At this point, Linus said, we are doing really well everywhere except on the traditional desktop. That is kind of ironic, since the desktop is what Linus started the whole thing for in the first place - he wanted to run it on his own desktop system. We have most of what we should need at this point, including a lot of applications, but the desktop is simply a hard market to get into. It's hard to get people to change their habits. That said, we'll get there someday.

Can we do anything in the kernel to further that goal? Linus responded that he'd been thinking about that question, but he didn't really know. A lot of work has been done to get the kernel to support the desktop use case; kernel developers, after all, tend to use Linux as their desktop, so they are well aware of how well it works. But it's up to the distributors to target that market and create a complete product.

Greg noted that 20 years is a long time to be working on one project; has Linus ever thought about moving on? Linus responded that he really likes concentrating on one thing; he is not a multi-tasker. He's really happy to have one thing that he is doing well; that said, he never had expected to be doing it for this long. When asked if he would continue for another 20 years, Linus said that he'd be fairly old by then. Someday somebody young and energetic will show up and prove that he's really good at this work. That will be Linus's cue to step aside: when somebody better comes along.

What do we need to do to keep the kernel relevant? Linus said that relevance is just not a problem; the Unix architecture is now 40 years old, and it is just as relevant today as ever. Another 20 years will not make a big difference. But we will continue to evolve; he never wants to see the kernel go into a maintenance mode where we no longer make significant changes.

Moments, challenges, and licenses

A member of the audience asked Linus to describe his single most memorable moment from the last 20 years. Linus responded that he didn't really have one; the kernel is the result of lots of small ideas contributed by lots of people over a long time. There has been no big "ah ha!" moment. He went on to describe a pet peeve of his with regard to the technology industry: there is a great deal of talk about "innovation" and "vision." People want to hear about the one big idea that changes the world, but that's not how the world works. It's not about visionary ideas; it's about lots of good ideas which do not seem world-changing at the time, but which turn out to be great after lots of sweat and work have been applied.

He did acknowledge that there have been some interesting moments, though, going back to nearly 20 years ago when Linux went from a personal project to something where he no longer knew all of the people who were involved in it. At that point, he realized, Linux wasn't just his toy anymore. There have been exciting developments; the day Oracle announced that it would support Linux was one of those. But what it really comes down to is persistence and hard work by thousands of people.

Another person asked whether the increasing success of web applications would mean the end of Linux. Linus responded that the move toward the browser has, instead, been helpful to Linux. There used to be a whole lot of specialized, Windows-only applications for tasks like dealing with banks; those are now all gone. When applications run in the browser, the details of the underlying operating system don't matter, at which point it comes down to technology, licensing, and price - all of which are areas in which Linux excels.

The next question was: are you happy with Ubuntu? Linus suggested that Greg should answer that question for a more amusing result. He went on to say that Ubuntu is taking a different approach and is getting some interesting results. It is helpful to have a distributor working with a less technical, more user-centric approach. Ubuntu has been successful with that approach, showing the other distributors a part of the market that they were missing. Greg added that his main concern is that he wants to see the kernel community grow. Things are, Greg said, getting better.

What is the toughest technical problem that Linus has ever had to deal with? Linus answered that the biggest problems he faces are not technical. In the end, we can solve technical issues; we make bad decisions sometimes, but, over time, those can be fixed. When we have serious problems, they are usually in the area of documentation and help from hardware manufacturers. Some manufacturers not only refuse to help us support their hardware; they actively try to make support hard. That, Linus said, irritates him, but that problem is slowly going away.

What is really hard, though, is the problem of mixing the agendas of thousands of developers and hundreds of companies. That leads to occasional big disagreements over features and which code to merge. If Linus loses sleep, it tends to be over people and politics, not technical issues; the interactions between people can sometimes frustrate him. We usually solve these problems too, but the solution can involve bad blood for months at a time.

The linux-kernel mailing list, Linus said, is somewhat famous for its outspoken nature; it is seen as a barrier to participation sometimes. But it's important to be able to clear the air; people have to be able to be honest and let others know what they are thinking. If you try to be subtle on the net, people don't get it; that can lead to developers putting years of work into features that others simply hate. In the long run, Linus said, it can be much healthier to say "hell no" at the outset and be sure that people understand. Of course, that only works if we can then admit it when it turns out that we were wrong.

The final question was about the GPL: is he still happy with the license? Linus said that, indeed, he is still very happy with GPLv2. He had started with a license of his own creation which prohibited commercial use; it took very little time at all before it became clear that it was making life hard for distributors and others. So he has always been happy with the switch to the GPL which, he said, is a fair and successful license. He feels no need to extend it (or move to GPLv3); the license, he said, has clearly worked well. Why change it?

[Your editor would like to thank the Linux Foundation for assisting with his travel to Japan.]

---

(Log in to post comments)