ALS: The open source talent war

Linux Foundation executive director Jim Zemlin kicked off the Spring (at least in the northern hemisphere) edition of the Automotive Linux Summit with some advice for the automotive industry executives and managers in the audience. There is battle going on for developers and other open-source-knowledgeable employees, and it is an important battle to win. There is so much open source development going on that talented people are being snapped up by the competition—not just competitors in the automotive world, but mobile, web, cloud, and other areas as well.

He started off by noting that a typical car now has more than 100 million lines of code in it. Customer expectations are changing all aspects of computing and automotive is just part of that trend. What people see in their mobile phones, tablets, and other consumer electronic devices is creating a demand that leads to a "huge increase in the amount of software in vehicles". That's why the industry is using open source: because it is an essential tool to help meet those expectations.

But just using open source is not enough. In addition, the automotive industry needs more software developers. More software means more developers, he said. In Silicon Valley right now, the demand for developers in the consumer electronics, web, mobile, and cloud areas is huge; there is a war for talent going on. In fact, top developers now have agents, much like Hollywood actors.

Zemlin has learned some lessons about hiring the best programmers in the world. The reason he knows something about it is because he is Linus Torvalds's boss. Torvalds is an incredible programmer who shares some traits with someone else that Zemlin is the boss of: his daughter. Both are adorable, geniuses, and, most importantly, neither of them "listen to anything I have to say", he said to a round of laughter.

More seriously, in working with Torvalds he has learned some things about how to work with the best programmers. There is a big difference between the best programmer and an average one; Zemlin likened it to the difference between a house painter and Picasso. The best programmers can do the work of 100 average programmers, he said.

Five lessons

There are five lessons he has learned about hiring great programmers. The first is to hire developers who play at work; programmers "who goof off". That may sound crazy, but if you look back at Torvalds's original email about Linux it says (paraphrased) "not doing anything big, just something for fun". Torvalds created Linux because it was fun and he still does it today because it continues to be fun. All of the best programmers do their work because they find it fun.

Zemlin mentioned a study from a book called Drive (by Daniel Pink) that looked at what motivates creative people. Since software is a creative profession, it gives insights into our industry, he said. In the study, people were divided into three groups and paid at three different levels (a lot, an average amount, and very little) for doing various tasks. For manual tasks, like factory work, those who were paid the best did the best work. But for creative tasks it was reversed, those who were paid the most did the worst. The study was originally run with students at MIT, but in case the students were atypical, they ran the study again in India: same result.

The "best creative people are not motivated by money alone", Zemlin said. Money is important, but it's not the only thing. What motivates great programmers is to be in an environment where they have the opportunity to "master their craft". Employers should be looking for people who are driven to master the skill of software development, he said.

Hire people who want to give their software away was lesson number two. If you really love what you do, you don't mind giving it away, he said. That leads to an obvious question: how do you make money if you give your software away? There are companies who have figured out how to make money in new ways, which is different from the old way of "keeping everything inside and selling it". He put up a comparative stock chart from 2008 to 2012 for three companies: Red Hat, IBM, and Microsoft. In that time, Red Hat has doubled, IBM is up 85% and Microsoft is flat, so the one who barely gives away any of its software is the one that has seen no gain in its share price. Automotive companies are a lot like IBM, Zemlin said, they don't need to make money on the software, they can make it on products, services, and so on.

Lesson three is to hire developers who don't stick to a plan. He asked: is planning important to the automotive industry? It is, of course, so he is not saying to have no overall plan, but companies shouldn't try to centrally plan everything, he said. Torvalds controls what goes into Linux, but he doesn't have a plan for what comes next. Without a plan, though, Linux seems to do well enough, with 1.3 million Linux-based phones activated daily, 92% of the high performance computing market running Linux, 700,000 Linux-based televisions sold daily, nearly all of the major stock exchanges and commodities markets running on Linux, and on and on.

In software development, it is good to let organic ideas grow, rather than to try to plan everything out in advance. For example, an organic community formed that cared about Linux battery life. That community worked on fixing the power performance of Linux, which turned out to help the high performance computing (HPC) community because most of the cost of HPC is power. So, without any kind of central planning, a problem was fixed that helped more than just those originally interested in it.

"Hire jerks" is Zemlin's fourth lesson. Linux developers "can be kind of difficult", he said, they will engage in flame wars over the code that is submitted to the linux-kernel mailing list. That public criticism was a problem for Japanese and other Asian developers when they first started getting involved. But public criticism actually helps create better ideas, he said.

A 2003 study done at the University of California, Berkeley looked at how people create the best ideas. The participants were split into two groups, and one was told to brainstorm about ideas. That meant that all ideas were considered good ideas and that criticism was not allowed because it might stop the flow of ideas. The other group was told to be critical, to debate and argue about the ideas as they were raised. The group that used criticism was eight times better than the other group; it created more ideas, better ideas, and was far more successful than the brainstorming group. That means "it's OK to be a jerk", Zemlin said, but don't go overboard. The conclusion is that it is important to be critical of others' ideas.

Zemlin's last lesson is that automotive companies should hire a person to manage their external research and development. They should borrow an idea from the consumer electronics companies, Intel, IBM, Red Hat, and others to have someone that helps determine their open source strategy. That person would help decide which projects to participate in, what efforts to fund, and so on. It would be a person who is familiar with open source licenses, who knows how to hire open source developers, and is knowledgeable about how open source works.

"Talent is what is going to make the difference", Zemlin said. Open source is going to "help you compete", but automotive companies have to hire the best software developers. The good news for the automotive industry is that "everyone loves cars". The industry has a reputation for being "sexy" and "exotic", so auto companies can "leverage that position to hire cool developers". He concluded with an admonishment: software development is a talent war, hire the best and you will succeed, but if you don't, your competition certainly will.

[ I would like to thank the Linux Foundation for travel assistance so that I could attend the Automotive Linux Summit Spring and LinuxCon Japan. ]