Parallel computing is at least part of the future of
computing. When even fairly low-end phones have multiple cores, it is
clear where things are headed. In the future, hardware with even more
parallelism will be available, but parallel programming is not keeping
up. Part of the problem there is the lack of availability of inexpensive, highly
parallel hardware—which is exactly what Andreas Olofsson and his team
at Adapteva have set out to address.
In his keynote at the 2013 Linux
Foundation Collaboration Summit (LFCS), Olofsson described the
$99 supercomputer that Adapteva ran a Kickstarter project to design and
build. He began by noting that his keynote followed one from Samsung,
which is the world's largest electronics company while he was representing "perhaps
electronics company. Olofsson is not only the CEO, he is also one-third of
the engineering team, he said with a chuckle. Despite being so small, the
company was able to build a 64-core 28nm chip that runs 100 GigaFLOPS at 2
Adapteva created that chip two years ago and went around the world trying
to sell it, but that effort was met with "crickets", he said. The world
just wasn't ready yet, so that's where the Kickstarter idea came from.
Adapteva is turning into a systems company because "people want computers,
they don't just want chips".
Energy efficiency has not kept up with Moore's law, which creates an
"efficiency gap" that impacts both battery powered and plugged-in systems,
The gap has only started to widen in the last few years, which is why most
people don't care—yet.
But it is a
problem that is getting worse and worse and will be quite severe by 2020 or
2025. We have to start thinking differently, he said.
The architecture that everyone wants is an infinitely fast single-threaded
CPU with infinite memory and infinite bandwidth to that memory, which is,
of course, impossible to have. We keep trying new things when we hit
various limitations, scaling up the processor frequency, issuing multiple
instructions per cycle, adding SIMD (single instruction,
multiple data), and eventually adding more cores. "Now what?", he asked.
We are "running out of tricks". We have seen this play out in servers and
desktops, and are seeing it play out in mobile "as we speak".
When he started the company in 2008,
there were a number of chip-design trends that he believed would shape the
future of computing: things like power consumption, memory bottlenecks,
thermal density, yield issues, software complexity, and Amdahl's law. But
we have a very different example of computing that is close to hand: our
Whenever you look at the brain, you realize that what we have been
designing is so primitive in comparison to what nature has already
accomplished, he said. It is massively parallel (billions of neurons),
energy efficient (30 watts),
heterogeneous (different parts of the brain handle different functions),
and robust (losing a small part generally does not shut the whole thing
The "practical vision" for today is heterogeneous, Olofsson said. We have
system-on-chips (SoCs) that can combine different kinds of functional
units, such as "big" CPUs, GPUs, and FPGAs. But, we won't see 1000 ARM
cores on a single SoC, he said. What Adapteva has done is to add hundreds,
or even thousands, of small RISC CPUs into a SoC with ARM or x86 cores as
the main processor.
The current state of parallel programming is not up to the task of writing
the code that will be needed for massively parallel systems of the future.
The challenge is to make parallel programming as productive as Java or
Python is today, Olofsson said. We should get to the point where GitHub
has thousands of projects that are using parallel programming techniques
that run on parallel hardware.
To get to this world of parallel ubiquity,
the challenges are "immense". The computer ecosystem needs to be rebuilt.
Billions of lines of code need to be rewritten, and millions of programmers
need to be re-educated. In addition, the computer education curriculum
needs to change so that people learn parallel programming in the first year
of college—or even high school.
In his mind, there is no question that the future of computing is parallel;
"how else are you going to scale?". Where else can you get the next
million-times speedup, he asked. But there is a question of "when?" and
it is "going to hurt before we get there". There is no reason not to start
now, though, which is where some of the ideas behind Parallella come from.
Inspired by the Raspberry Pi, Parallella is a $99 credit-card-sized
parallel computer. Adapteva is trying to create a market for its chip, but
it is also "trying to do good at the same time". The only way to build up
a good-sized community around a device like Parallella is to make it open,
"which is hard for a hardware guy", he said. But that makes the platform
accessible to interested hackers.
Parallella is cheap and easy to use, Olofsson said. He wishes Adapteva
could sell a million of them (like Raspberry Pi), but thinks that's
unlikely. If it sold 10,000 and people took them and did innovative things
using Parallella, that would be a big success.
The Kickstarter tried to raise $750,000 in 30 days and was successful in
doing so. He and his team have spent the last six months trying to deliver
what had been promised. The goal was a 5 watt computer that could do
100GFLOPS, which was achieved. There are many different applications for
such a device including software defined radio, ray tracing, image
processing, robotics, cryptography, gaming, and more.
The system has a dual-core Cortex-A9 processor that runs Linux. It has
most of the peripherals you would expect, but it also has a built-in FPGA.
That device can be configured to "do anything" and is meant to augment the
main processor. In addition, there is Adapteva's Epiphany
co-processor, which brings many small RISC cores to the system.
At the time of the talk, it had only been five days since Adapteva had
received the boards. Both 16-core
and 64-core Epiphany versions were delivered (Olofsson is holding one
of each in the photo above). Each consumes around 5 watts and he is
"excited to get them in the hands of the right people". By the second day,
the team could run "hello world" on the main CPU. Earlier in the
day of the talk (day 5), he heard from another member of the team that it
could talk to the Epiphany and run longer programs on the A9. Six months
ago, he didn't know if you could actually build this type of system in
credit card form factor with a $99 price point, but it can be done.
Now Adapteva needs to ship 6300 systems to people who donated to the
Kickstarter, which is no easy task. There are some serious logistics to be
worked out, because "we want people to be happy" with their boards.
Adapteva also wants to give away free kits to universities. Building a
sustainable distribution model with less than five people in the company
will be challenging. It is running a little late, he said, but will be
shipping all of the boards in the (northern hemisphere) summer.
Olofsson concluded with Adapteva's plans after all the shipping and
handling: next up is "massive parallelism" with 1024-core Epiphany
co-processors. It will be interesting to see what folks will do with
that system when it becomes available.
to post comments)