Andrew Chatham came up the peninsula to San Francisco from Google to talk
to the 2013 Embedded
Linux Conference about the self-driving car project. Chatham has
worked on the project since 2009 and seen it make great strides. It is by
no means a finished product, but the project has done 400,000 miles of
driving so far.
"Cars are a miracle", he said. The 45-mile drive he did to Mountain View
yesterday would have taken our ancestors all day to do on a horse. But,
cars are also problematic, with more than 30,000 annual deaths in the US
due to car accidents. That number has "finally started dropping", likely
due to more seat belt usage, but it is still too high. Even if there
are no fatalities, accidents cost time, money, and more. We have done a
pretty good job figuring out how to survive accidents, he said, but it is time
to stop having them.
In the mid-2000s, the US Defense Advanced Research Projects Agency (DARPA) ran several
challenges for self-driving cars on a 150-mile course in
Desert. The first year, the winning team's vehicle went only seven
miles. But the next year, five teams actually completed the course, which
was eye-opening progress. In 2007, DARPA moved the challenge to a
simulated urban environment that featured a limited set of traffic
interactions (four-way stops, but no traffic lights, for example). After
that event, "DARPA declared victory" and moved on to other challenges,
In 2009, Google stepped in to solve the problem "for real". Chatham noted
that people have asked why Google would get involved since driving cars doesn't
involve searching the internet or serving ads. The company thinks it is an
important problem that needs to be solved, he said. Google is qualified to
attack the problem even though it has never made cars because it is mostly
a software problem. Also, one major component of the problem involves maps,
which is an area where Google does have some expertise.
Broadly, there are two categories for self-driving cars: one for cars with
smarts in the car itself and one where the smarts are in the road.
For cars that are self-contained, they need to be ready for anything and
cannot make assumptions about the obstacles they will face. That tends to
lead to cars that move slowly and drive cautiously, much differently than
humans. Smart roads allow for dumb cars, but there are some serious
obstacles to overcome. Infrastructure is expensive, so there is a
chicken-and-egg problem: who will build expensive smart roads (or even
non-existent dumb cars that can use them?
The Google approach is something of a hybrid. There are no actual
infrastructure changes, but the system creates a "virtual infrastructure".
That virtual infrastructure, which is built up from sensor readings and map
information, can be used by the car to make assumptions much like a human
does about what to expect, and what to do.
Sensors and such
The car's most obvious sensor is the laser rangefinder that lives in a
bubble on top of the car. It spins ten times per second and produces
100,000 3D points on each spin. Each of those points have 5cm accuracy.
The laser can only see so far, though, and can be degraded in conditions that
affect photons, such as rain.
The car also has radar, which is not as precise as the laser, but it can
see further. It can also see behind cars and other solid objects. Using
the doppler effect, speed information for other objects can be derived.
There are also cameras on the car. The general "computer vision problem"
is hard, and still unsolved, but it isn't needed for the car's usage of the
camera. The camera is used for things that humans use as well, which means
they are generally rather obvious and are of known shapes, sizes, and
likely positions (e.g. traffic lights). Beyond that are the expected
sensors like gyroscope, accelerometer, GPS, compass, and so on.
There are two main computers in the car. One is a very simple "drive by
wire system" that has no operating system and is just in a tight loop
controlling the brakes, steering, and accelerator. The second is a
"workstation class system running FreeBSD", Chatham joked. In reality it
is running a lightly customized Ubuntu 12.04 LTS. It is not running the
realtime kernel, but uses SCHED_FIFO and control groups to provide
There are several classes of processes that run on the system, with the
least critical being put into control groups with strict resource limits.
If any of the critical processes miss their deadlines, it is a red flag
event which gets logged and fixed. In the 400,000 miles the cars have
traveled (so far always with a human on board), those kinds of problems
have been largely eliminated. All of the data for those journeys has been
stored, so it can be played back whenever the code
is changed to try to find any regressions.
From the "blank slate" state of the car, GPS data is added so that it knows
its location. That data is "awful really", with 5m accuracy "on a good
day" and 30m accuracy at other times. The car's sensors will allow it to
accurately know which way it is pointing and how fast it is going. From
there, it adds a logical description of the roads in that location derived
from the Google Maps data. It uses maps with 10cm resolution plus altitude
data, on top of which the logical information, like road locations, is layered.
All of that information is used to build a model of the surroundings. The
altitude data is used to recognize things like trees alongside the road, as
well as to determine the acceleration profile when climbing hills. The
goal is to stay close to the center of the lane in which the car is
traveling, but not if something is blocking the lane (or part of it). You
also don't want to hit the guy in front of you, so don't go faster than he
does. Once the model is built, driving is largely a matter of following
those two rules, he said.
Problems and solutions
In California (unlike many other places) it is legal for motorcycles to
travel between the lanes, moving between the cars that are in each of those
That was a difficult problem to solve because that situation could fool the
sensors to some extent.
One of the tests they ran was a race track that was set up to see how the
versus Google employees. It reliably beat them in that test, though
Chatham believes the employees would have eventually beaten the car. It's
"a Prius, not a sports car", so there were limits to the kinds of
maneuvering that can be done, but the test really showed the "precision
with which we can
reliably drive the car", he said.
Lots about driving is "social", Chatham said. For example, there is a set
of rules that
are supposed to be followed at a four-way stop, but no one follows
them. The car had to learn to start to edge out to see when the others
would let it go through. Similarly, merging is social, and they have spent
a lot of time getting that right. Unlike human drivers, the car can't make
eye contact, so it is a matter of getting the speed and timing right for
what is expected.
The "bug on the window problem" is another difficult one. For the car, anything
that messes up its sensors needs to be handled gracefully. In those cases,
handing control back to the human in a sensible fashion is the right thing
Many people ask about how the car does in snow, but it hasn't been tried
yet. Currently, Chatham thinks it wouldn't do all that well, but thinks it
"could do OK eventually". One problem there is that snowbanks appear to be
giant walls of water to the lasers.
"People do stupid things", he said. If you drive 400K miles, you are going
to experience some of them. Normally the expectation is that other people
value their lives; if you didn't believe that, you would never leave home.
But there are exceptions, so
a self-driving car, like a regularly driving human, needs to be prepared
for some of that craziness.
The video of a blind
the self-driving car is the kind of story that shows where this
technology could lead, Chatham said. There are a lot of people who can't
drive for one reason or another, so a self-driving car has the potential to
change their lives. "I wish it were done now, but it's not", he said.
Chatham answered a few questions after the talk. They have done very
little work on "evasive maneuvers", he said. Everyone overestimates their
ability in that area and the advice from police and others is to just use
the brakes. There are no plans as yet to release any of the source code,
nor are there any plans for a product at this point. Three states have
"legalized" self-driving cars, California, Nevada, and Florida. It is
furthest along in California where the Department of Motor Vehicles is
currently drafting rules to govern their use.
[ I would like to thank the Linux Foundation for travel assistance to attend ELC. ]
to post comments)