|
|
Log in / Subscribe / Register

Rusty's reminiscences

By Jonathan Corbet
February 1, 2019

LCA
Rusty Russell was one of the first developers paid to work on the Linux kernel and the founder of the conference now known as linux.conf.au (LCA); he is one of the most highly respected figures in the Australian free-software community. The 2019 LCA was the 20th edition of this long-lived event; the organizers felt that it was an appropriate time to invite Russell to deliver the closing keynote talk. He used the opportunity to review his path into free software and the creation of LCA, but first a change of clothing was required.

Russell formally left the kernel community in 2017 to pursue the blockchain dream; his entrance at LCA 2019 reflected the wealth that resulted. Or that was maybe supposed to result; the chart of the value of Bitcoin he put up reflected the real-world experience. After a show of dismay, Russell stripped down to the obligatory LCA uniform [Rusty Russell] (shorts and an LCA T-shirt) to get into his talk where he would, he said, "misremember history into a narrative that reinforces my personal biases".

He found his way into the Unix world in 1992, working on an X terminal connected to a SunOS server. SunOS was becoming the dominant Unix variant at that time, and there were a number of "legendary hackers" working at Sun to make that happen. But then Russell discovered another, different operating system: Emacs. This system was unique in that it was packaged with a manifesto describing a different way to create software. The idea of writing an entire operating system and giving it away for free seemed fantastical at the time, but the existence of Emacs meant that it couldn't be dismissed.

Even so, he took the normal path for a few more years, working on other, proprietary Unix systems; toward the end he ended up leading a research project developed in C++. The proprietary compilers were too expensive, so he was naturally using GCC instead. He did some digging in preparation for this talk and found his first free-software contribution, which was a patch to GCC in 1995. The experience of collaborating to build better software for everybody was exhilarating, but even with as much fun as he was having there was another level to aim for.

Encountering Linux

In 1997, he went to a USENIX conference which featured a special "USELINUX" track. Developers like Ted Ts'o, Alan Cox, Dave Miller, and Stephen Tweedie were all there; Russell found himself standing nervously among them as they were talking about implementing SMP. Miller grabbed a napkin and jotted down some x86 assembly code on it; Cox then optimized it on the spot. The highlight of the event, Russell said, was a talk by Miller and Miguel de Icaza on the SPARC port of the Linux kernel; the talk was not recorded but Russell is confident that it was the greatest technical talk ever given. It started with some lmbench benchmark results showing Linux performing rather poorly compared to Solaris. Miller went into a great many details regarding the SPARC architecture and how Linux could be made to perform better on it; Russell "understood some of the words". By the end of the talk, Miller showed that Linux now outperformed Solaris on every benchmark.

At one point, Russell talked with Jon "maddog" Hall, who said that it was hard to describe the pre-Unix experience to those who had not been there. Something similar holds with regard to the world before and after this USENIX event. Russell had walked into the room not thinking that a group of students could cobble together a system, a couple more could then hack on it, and the result would beat what the professionals — those legendary hackers — were producing. Afterward, it was obvious that things could work that way.

Russell's immediate conclusion was that he wanted to work with these people. Later in 1997, he recruited in Michael Neuling to work on a firewall implementation, which he had anticipated would be merged alongside the existing code in the kernel. Miller looked at the code and not only applied it; he replaced the old code entirely. Russell woke up one day and found that he had become the kernel's firewall maintainer.

Also in 1998, he returned to USENIX and ran into somebody who was looking to hire an ipchains expert. Russell explained his plans for an ipchains successor, saying "give me the money and I can do it in six months". The pitch worked, amazingly. The implementation, though, being software, still took twelve months to come around.

The genesis of LCA

These experiences convinced him of the need for an Australian Linux conference, so he decided to organize one. Part of the process involved visiting a number of Australian user groups to recruit attendees; the most common question he got was something like "why would I go to a Linux conference?" At the time, most people working with Linux were students and hobbyists who were developing free software in their spare time. It was hard to explain to them why a Linux-specific gathering would be cool; he could mention speakers and talks and such, but that is missing the forest for the trees. The job is even harder, he said, when the conference in question doesn't actually exist yet.

Beyond attendees, he also needed speakers. That required paying for long-distance travel and more — a challenge. The first speaker he invited was, of course, Miller. In the end, Hall, Federico Mena Quintero, and Carsten "Rasterman" Haitzler also agreed to come. The old story that he [Rusty Russell] funded the whole thing on his credit card turns out to be true; at one point he got a call from American Express and had to fax them a copy of his bank statement to show that he could pay the bill.

The actual conference had no WiFi network (this was 1999, after all), so somebody printed the Slashdot front page every day and posted it at the venue. The last time slot was kept empty so that the best three talks could be chosen to be presented again — a tradition that LCA kept for some years. There was, naturally, no video. The conference did include some tutorial sessions, for which the materials had been requested from the speakers two weeks before the event. That didn't happen, so a lot of last-minute copying had to be done; that was a problem, since the school term was starting and all of the copy shops were fully booked. So 400 book copies were made by a group of volunteers feeding money into a set of coin-operated copy machines.

Andrew Tridgell brought a big machine with three CD burners, which were used to crank out copies of the conference proceedings — a process that took about two days. As the event began, somebody complained about the lack of a conference T-shirt, to which Russell answered something like "I am so tired right now" and the person went away. But then somebody bought a bunch of white shirts and found a silkscreen printer at the university; they then proceeded to crank out a set of shirts featuring the front page of the conference web site.

Tridgell collected donations from the speakers to buy a gift for Russell, which he found touching. That tradition remains with LCA until this day. Russell said that he learned well the most important lesson from having organized a conference: "never again". Others have stepped up since, though, with the result that LCA has become one of the premier Linux events; he has given 19 LCA talks since then.

After 1999

LCA, as his Conference of Australian Linux Users came to be called, has shaped his life. He met Tridgell, which led to his moving to Canberra and working at IBM's OzLabs, which he said was the greatest concentration of Linux kernel hackers to ever work in a single location. He had found a tribe of hackers who made him a happier and better person. All great projects, he said, come down to a small group of people — people who are smart enough to complete the job, and who are dumb enough to try. He has been lucky enough to be a part of this kind of project.

Russell's takeaway from this experience is that, when somebody has something that they want to try, and it's not actively harmful, the best responses are "sounds great, tell me more", and "what can I do to help?" You can make people far more likely to succeed by actively collaborating with them. Collaboration is "a superpower", but it is not always easy. Something he learned far too late (and is still working on) is that getting along with people is a skill in its own right. But it's a skill that is well worth picking up.

There are, he said, "headwinds to collaboration" all over our community. One of those is impostor syndrome, which causes people to shy away from roles in our community. As he was working on his exit from the kernel community, he found a developer who, he thought, would be a fine person to take over the maintainership of the kernel's module loader, but she did not feel up to the job. Six months later, though, he tried again and she accepted. With enough encouragement, people can be convinced to step up and take on responsibilities within our community.

Another headwind is a general attitude that tells people that "you don't belong here". People, he said, are well tuned to "go away vibes" that are aimed at them, but they are also good at not even seeing them otherwise. The recipients of such messages don't get to collaborate in our community; there are no superpowers for them. That is hurtful for the people involved, but also for our community as a whole.

Russell concluded with a statement that people with the skills needed to work within our community are rare, but they are wonderful and he wants to experience them as much as possible. If you want to work on the projects that he is involved with, and you have the skills to do so, he wants to work with you.

A video of this talk is available (also on YouTube).

[Thanks to linux.conf.au and the Linux Foundation for supporting my travel to the event.]

Index entries for this article
Conferencelinux.conf.au/2019


to post comments

Rusty's reminiscences

Posted Feb 2, 2019 15:08 UTC (Sat) by NightMonkey (subscriber, #23051) [Link]

Wonderful article.

Rusty's reminiscences

Posted Feb 4, 2019 12:36 UTC (Mon) by ncultra (guest, #121511) [Link] (2 responses)

Rusty's early Linux code is the finest 'C' code I've worked with in 30 years of coding.

Rusty's reminiscences

Posted Feb 4, 2019 15:41 UTC (Mon) by imphil (subscriber, #62487) [Link] (1 responses)

Would you mind posting an example?

Rusty's reminiscences

Posted Feb 4, 2019 16:18 UTC (Mon) by ncultra (guest, #121511) [Link]

Download linux 2.6.39 from github: https://github.com/torvalds/linux/releases/tag/v2.6.39

Once you have untarred or unzipped it, cd to linux-2.6.39 and grep -rn rusty@rustcorp.com.au. I most like the kallsyms code and lguest. But the most important code, imo is netfilter.

[linux-2.6.39]$ grep -rn rusty@rustcorp.com.au
...
arch/x86/kernel/apm_32.c:125: * (Paul "Rusty" Russell <rusty@rustcorp.com.au>).
arch/x86/lguest/boot.c:29: * Copyright (C) 2006, Rusty Russell <rusty@rustcorp.com.au> IBM Corporation.
drivers/lguest/core.c:376:MODULE_AUTHOR("Rusty Russell <rusty@rustcorp.com.au>");
drivers/lguest/x86/core.c:2: * Copyright (C) 2006, Rusty Russell <rusty@rustcorp.com.au> IBM Corporation.
drivers/net/virtio_net.c:3: * Copyright 2007 Rusty Russell <rusty@rustcorp.com.au> IBM Corporation
drivers/net/xen-netback/xenbus.c:4: * Copyright (C) 2005 Rusty Russell <rusty@rustcorp.com.au>
drivers/platform/x86/thinkpad_acpi.c:44: * thanks to Rusty Russell <rusty@rustcorp.com.au>
include/linux/kallsyms.h:3: * Copyright 2002 Rusty Russell <rusty@rustcorp.com.au> IBM Corporation
kernel/kallsyms.c:6: * Copyright 2002 Rusty Russell <rusty@rustcorp.com.au> IBM Corporation
kernel/kmod.c:19: Rusty Russell <rusty@rustcorp.com.au> Jan 2003
kernel/stop_machine.c:5: * Copyright (C) 2008, 2005 Rusty Russell rusty@rustcorp.com.au
lib/find_last_bit.c:4: * Written by Rusty Russell <rusty@rustcorp.com.au>
net/ipv4/netfilter/nf_nat_ftp.c:24:MODULE_AUTHOR("Rusty Russell <rusty@rustcorp.com.au>");
net/ipv4/netfilter/nf_nat_irc.c:4: * (C) 2004 Rusty Russell <rusty@rustcorp.com.au> IBM Corporation
net/netfilter/nf_conntrack_ftp.c:30:MODULE_AUTHOR("Rusty Russell <rusty@rustcorp.com.au>");
net/netfilter/xt_connlimit.c:12: * GPL (C) 1999 Rusty Russell (rusty@rustcorp.com.au).
net/netfilter/xt_state.c:18:MODULE_AUTHOR("Rusty Russell <rusty@rustcorp.com.au>");
tools/perf/bench/sched-messaging.c:7: * Based on hackbench by Rusty Russell <rusty@rustcorp.com.au>

Rusty's reminiscences

Posted Feb 8, 2019 14:45 UTC (Fri) by wurtel (guest, #7155) [Link]

I attended a half-day course on iptables that Rusty gave together with Harald Welte at Linux Kongress 2001 in Enschede, the Netherlands (http://www.linux-kongress.org/2001/abstracts.html#russell...). That was the most enjoyable, worthwhile and even entertaining training I've ever had in the past 30 years.


Copyright © 2019, Eklektix, Inc.
This article may be redistributed under the terms of the Creative Commons CC BY-SA 4.0 license
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds