LWN.net Logo

Asteroid "mining" with Linux and FOSS

By Jake Edge
September 18, 2013
LinuxCon North America

Planetary Resources is a company with a sky-high (some might claim "pie in the sky") goal: to find and mine asteroids for useful minerals and other compounds. It is also a company that uses Linux and lots of free software. So two of the engineers from Planetary Resources, Ray Ramadorai and Marc Allen, gave a presentation at LinuxCon North America to describe how and why the company uses FOSS—along with a bit about what it is trying to do overall.

Ramadorai, who is a Principal Avionics Engineer with the company, began the talk by relating that he joined the company by noticing that it was forming in Bellvue, Washington in 2012 and phoning the CEO. He started the talk with a question: what does asteroid mining have to do with Linux? It turns out, he said, that as they looked at the requirements for the spacecraft and compared it with those of a data center, there was a "lot of overlap" between the two. A spacecraft is a distributed system that requires high availability. In addition, power efficiency is important as the spacecraft are powered by solar panels. Using free software was an opportunity to use what's already been done in those areas, he said.

[Ray Ramadorai]

By way of some context, Ramadorai explained "why asteroids" and "why now". Technology has reached a point where it is viable to build small spacecraft capable of prospecting and eventually mining near-Earth objects (NEOs). Part of the reasoning is idealistic as many of the employees are "space fans", but there is also a significant opportunity for financial returns, he said.

There is more of an awareness about asteroids recently. The Russian meteor earlier this year is one example, but NASA in the US is also talking about capturing a NEO and orbiting it around the moon. There are a lot more launch opportunities these days due to the private space companies (e.g. SpaceX, Orbital Sciences). That means companies can get things to orbit for less than hundreds of millions of dollars. There has been a steady growth of the small satellite industry because of that. It's not so much that the price is coming down, but that there is much more capacity available for launches, he said.

Hardware has also gotten cheaper and more powerful. The MIPS per unit watt have been increasing, at least in standard (not space-rated) parts. There has been a lot of resistance within the aerospace industry to using off-the-shelf parts, but the cost and performance difference is huge. What's needed is a system that can handle some failures caused by space radiation.

It has gotten to the point where a small company can actually build and launch a spacecraft. FOSS has played a large role in that. The Planetary Resources software team is small, and Ramadorai estimates his team will only write 5-10% of the code that runs on the craft—the rest will come from existing free software.

He emphasized that this was a long-term endeavor for the company. Actually mining asteroids is a long way off. The first steps are to validate the technology by starting to prospect and visit NEOs. There are some 1.5 million asteroids larger than 1km in the solar system, with nearly 1000 of those being near Earth. If you look at smaller asteroids, those 100m or less, there are around 20,000 of them near Earth. Seventeen percent of those NEOs are "energetically closer" (i.e. require less energy to reach) than the Moon.

He showed some images of various NEOs that had been visited by probes, then showed one of the smallest on that slide (Itokawa) to scale with the Space Needle—it is wider than that building is tall (184m). The idea is that these are immense objects. They also can contain a great deal of interesting material. A 75m C-type asteroid has enough H2 and O2 to have launched all 135 Space Shuttle missions, while a 500m LL-Chondrite asteroid can contain more platinum than has been mined in human history.

Unfortunately, the US International Traffic in Arms Regulations (ITAR) restrict the kind of information Planetary Resources can share. Spacecraft are classified as munitions, which means that the company can't work with free software communities the way it would prefer to. The company strives to contribute as it can, while working within ITAR. It is "annoying" and in Ramadorai's opinion, "spacecraft should not be classified as munitions". He suggested that those interested "write Congress" about the problem.

The first step is the Arkyd 100 spacecraft that will be tested in low-Earth orbit. After that is the Arkyd 200 that will travel to Earth-crossing asteroids, and the Arkyd 300 that will actually land on asteroids. These are small craft; the Arkyd 100 can be relatively easily held by a human (say three shoe boxes in size).

Part of how they can be that small is by dual-purposing everything that can be. For example, the telescope that is used for prospecting and imaging asteroids is also used for laser communications with Earth. When a spacecraft is out 1-2 astronomical units (AUs), sending directional communication is a must for a low-power device. But at 2 AUs, the round-trip time is 32 minutes, so autonomy in the craft is essential.

The "state of the art" space-rated processor is the Rad750, a 32-bit PowerPC running at 133MHz. It uses 10 watts of power and costs $200,000. He compared that with an Intel Atom processor running at 1.6GHz, consuming 2 watts, and available for less than $1000. That is why the team is planning to use off-the-shelf parts and to deal with faults that will happen because the processor is not space rated.

Linux is important because they can run the same operating system on the craft, in the ground station systems, on their desktops, and in the cloud. The cloud is useful for doing simulations of the system code while injecting faults. It is common to spin up 10,000 instances in the cloud to do Monte Carlo simulations while injecting faults for testing purposes, he said.

[Marc Allen]

Ramadorai then turned the floor over to Allen, who he described as one of the Jet Propulsion Laboratories (JPL) refugees at Planetary Resources. While at JPL, he worked on the backup landing software for the Curiosity Mars rover. He was, Ramadorai said, one of the few software people that was quite happy that his code never actually ran. Allen noted that he worked on flight software at JPL for five years, which gave him a different perspective than some others at the company; there is a "mix of both worlds" on the team.

Traditional deep space missions are expensive and take a long time to design and launch. There is a tendency to pick some technology (like the Rad750 processor) and stick with it. There are at most 2-3 vehicles built per project, but Planetary Resources has a different philosophy and set of motivations. It needs to look at "lots of asteroids" to find ones of interest.

That means using cheap, commodity hardware which can be upgraded as needed throughout the life of the project. Because the company is a low-cost spacecraft developer, it wants to use Linux and FOSS everywhere it can. Traditionally, each separate component was its own silo, so the software for flight, ground station, and operations were completely separate.

There is so much free software available that it is easy to find to reuse and repurpose for their needs, Allen said. The challenge is how to stitch all of the disparate pieces together into a high-availability system. But a proprietary system would have far fewer contributors and wouldn't get new features as quickly, he said.

For example, inter-process communication (IPC) has traditionally been created from scratch for each project, with custom messaging formats, state machines, and serialization mechanisms—all written from scratch. Instead of doing that, the Planetary Resources team specified a state machine model and message model in XML and fed it to some Python code that auto-generated the state machine and IPC code. It uses protobuf and Nanopb for serialization and ZeroMQ for message passing (among other FOSS components). "Why reinvent the wheel?", he asked.

That could not have been done in the past, because processors like the Rad750 would not support it. By using commodity hardware and handling faults when they occur, it opens up more possibilities. For example, a hypervisor is used to simulate redundant hardware in order to support triple modular redundancy. Three separate versions of the flight software can be run in virtual machines, voting on the outcome to eliminate a problem caused by space radiation in one of the programs. It isn't a perfect solution, but "we're not NASA, we don't have to have 100% reliability".

The team is considering putting a SQL database in the spacecraft itself. The communication availability, bandwidth, and reliability is such that there needs to be an easy and compact way for operations to request data sets to be retrieved. "Fundamentally, we a data company", Allen said, but much of the data will never reach Earth.

There are a number of challenges with maintaining database integrity in space. But there is only a need to make it "reliable enough" to be useful, corruption will happen, and the software must be designed with faults in mind. Using features like ECC memory or RAID for storage are two techniques that can be explored because of the flexibility that commodity hardware and FOSS provide.

Perhaps surprisingly, the cloud has been instrumental in developing the software. Simulating faults, stressing the software, and so on have all used the cloud extensively, he said. ITAR plays a role there too, as they must use Amazon's GovCloud, which is ITAR compliant.

Ramadorai wrapped things up with a quick pitch for anyone looking for employment. The company needs both hardware and software engineers; judging from the discussions going on after the talk, there were some interested folks in the audience.

The plan is to launch a prototype next year sometime, Ramadorai said. It is difficult to be more specific because launch dates (and carriers) frequently change. Beyond that, it would depend on how the early tests go. Mining asteroids is, alas, still quite a ways off it seems.

[ I would like to thank LWN subscribers for travel assistance to New Orleans for LinuxCon North America. ]


(Log in to post comments)

Asteroid "mining" with Linux and FOSS

Posted Sep 18, 2013 22:56 UTC (Wed) by smoogen (subscriber, #97) [Link]

I wish them luck. Having been on the periphery of similar projects back in the 1990's that were trying off the shelf parts they will need to make sure that they have multiple systems and that I am guessing their first test box is going to run in the Van Allen belts to give a good test on "high" radiation environments.

Asteroid "mining" with Linux and FOSS

Posted Sep 18, 2013 23:19 UTC (Wed) by dchichkov (subscriber, #90878) [Link]

Interesting approach. Virtual machines instead of hardened hardware. I would guess it is going to be soft real time are a result. With jitter on the level of hundreds of microseconds... Scary.

Not sure what could be an alternative. Perhaps a C compiler that generates radiation hardened executable code?

Asteroid "mining" with Linux and FOSS

Posted Sep 19, 2013 1:30 UTC (Thu) by MrMorden (subscriber, #62781) [Link]

Physical machines could still handle the RT while coming in under the cost and power budget of a rad-hardened processor. So that just leaves the mass budget.

Besides which, there's no way you could get Kerbal to run on a Rad750. (Sorry; had to get a KSP reference in this thread somehow.)

Asteroid "mining" with Linux and FOSS

Posted Sep 19, 2013 1:50 UTC (Thu) by mathstuf (subscriber, #69389) [Link]

But you could get a Kerbal to run a Rad750 in the test environment :D . Poor Jebediah :( .

NSEU on the hypervisor?

Posted Sep 19, 2013 1:35 UTC (Thu) by pflugstad (subscriber, #224) [Link]

So what happens when the hypervisor itself gets hit by a NSEU, either in it's instruction stream, or some data structure it maintains (page tables, vm function pointer table, etc). I don't see how triplicate VMs addresses that. I guess you could just reboot the system, but then what controls the craft while that happens. Are they planning on sending dual/triplicate hardware as well to address that problem?

NSEU on the hypervisor?

Posted Sep 19, 2013 1:46 UTC (Thu) by Cyberax (✭ supporter ✭, #52523) [Link]

Spacecraft do not really need to have millisecond-precise control all the time. Indeed, quick reaction is important only during the powered flight stages.

I'd personally just install a very simple additional microcontroller to issue timed commands during these stages.

NSEU on the hypervisor?

Posted Sep 19, 2013 2:06 UTC (Thu) by pflugstad (subscriber, #224) [Link]

I thought of that as well, but then what if the microcontroller has a NSEU... its RAM is not protected either.

NSEU on the hypervisor?

Posted Sep 19, 2013 2:07 UTC (Thu) by dlang (✭ supporter ✭, #313) [Link]

They are planning to send redundant hardware, that's how they will be able to survive and recover from radiation induced errors.

But, 3 $1K processors are still cheaper than 1 $200K processor, not to mention faster.

Now, they may discover that NASA is right and you really need a space rated processor to keep going

Or they may find that a little bit of redundancy and shielding lets them get by for a fraction of what a NASA mission would need, and have a lot more performance available as well.

NSEU on the hypervisor?

Posted Sep 19, 2013 5:08 UTC (Thu) by Cyberax (✭ supporter ✭, #52523) [Link]

Unfortunately, shielding doesn't really help against cosmic rays. They are just too high-energetic. It helps against fairly slow-energy particles trapped in radiation belts, though.

Radiation hardening

Posted Sep 19, 2013 8:51 UTC (Thu) by kleptog (subscriber, #1183) [Link]

I actually found the wikipedia article on this quite informative:

http://en.wikipedia.org/wiki/Radiation_hardening

Essentially you have two kinds of damage. Actual physical irreversible damage which degrades your chips over a long period. And transient spikes which can cause problems. The latter can sometimes be fixed by turning it off and on, but that seems a little difficult if you're doing it on one chip with VMs.

Still, given the price difference you can play the odds. If for the same price you can send up 100 of these things and a dozen fail you're still way ahead (except, perhaps, for the space junk).

Radiation hardening

Posted Sep 27, 2013 15:39 UTC (Fri) by Wol (guest, #4433) [Link]

The other to bear in mind is the steady shrinkage in the size of the die. What you really want is a last-generation processor made using this-generation fabrication.

Let's assume a particle strike causes a 5nm "area of destruction". Do you want that hitting a track that's 10nm wide, or 30nm wide?

In other words, based purely on physical component size, older hardware will be more robust.

Cheers,
Wol

NSEU on the hypervisor?

Posted Sep 19, 2013 14:21 UTC (Thu) by cate (subscriber, #1359) [Link]

BTW it is the same method (triple hardware instead of radiation-resistant-hardware) used successfully by SpaceX Dragons (see e.g. https://en.wikipedia.org/wiki/Dragon_(spacecraft)#Radiation_tolerance)

NSEU on the hypervisor?

Posted Sep 19, 2013 13:39 UTC (Thu) by kh (subscriber, #19413) [Link]

I read it as right now they are simulating redundant hardware, but will launch with redundant hardware.

ARKYD on Kickstarter

Posted Sep 19, 2013 19:14 UTC (Thu) by jnareb (subscriber, #46500) [Link]

Is it the same ARKYD as ARKYD: A Space Telescope for Everyone funded project (at $1.5M/$1.0M goal) on Kickstarter?

ARKYD on Kickstarter

Posted Sep 21, 2013 4:42 UTC (Sat) by mtaht (✭ supporter ✭, #11087) [Link]

Yes.

While I am glad the kickstarter campaign was such a a success, it never seemed clear to me that was being expressed and sold there was only the first fragments of the bold and magnificent dream. While getting a small spacecraft up and running in LEO IS a giant challenge in itself - getting to the next step - OUT of orbit, finally, into the nearer parts of the solar system - is the next big goal after that.

"Once in orbit, you're halfway to anywhere." - R.A.H.

I'd like to see Linux in use throughout the asteroids in the fairly near term.

Asteroid "mining" with Linux and FOSS

Posted Sep 23, 2013 12:29 UTC (Mon) by jospoortvliet (subscriber, #33164) [Link]

I find the idea of having a large number of half-independent, half-intelligent tiny robots exploring and mining our solar awesome and exciting. Soon they'll be self-duplicating, after all, that saves a lot of money over sending new ones up! Humanity will have millions of little workers out there, sending down resources, progressively further developed. At some point, much of our goods could be coming from space!

Meanwhile, a few stray ones with broken communication modules will get lost but continue to duplicate. The radiation will modify the code they are running, and where it doesn't impair duplication, become part of the new versions. Sometimes, a modification will give one a head start over others, allowing it to take the resources it needs for duplication - perhaps with force. Soon after, the strong will start to prey on the weak and 100 years from now, earth will be invaded by a swath of intelligent little killer bots.

...

Interesting, I haven't seen a SF book exploit this possibility - yet it now seems the most likely way of a human-AI war to go ;-)

Asteroid "mining" with Linux and FOSS

Posted Sep 23, 2013 16:58 UTC (Mon) by Cyberax (✭ supporter ✭, #52523) [Link]

> Interesting, I haven't seen a SF book exploit this possibility - yet it now seems the most likely way of a human-AI war to go ;-)

That's exactly the plot of "Code of the Lifemaker" by James Hogan. An alien mining spacecraft is damaged by a supernova explosion and crash-lands on Titan. It starts building mining robots, but some of their programs are damaged by the supernova explosion. So they start to evolve (eventually evolving full sentience).

Asteroid "mining" with Linux and FOSS

Posted Sep 26, 2013 18:11 UTC (Thu) by nix (subscriber, #2304) [Link]

Of course, it couldn't actually work that way. The reason natural selection has room to work is that mutations in nature are both relatively frequent compared to bit-flips in hardware, much wider in scope (there is no bit-flip analogue of translocation!) and very often do not break or even greatly affect the organisms they happen in, because the code is redundant, many genes are duplicated, and most gene products are parts of networks with multiple redundant paths to the same end. *None* of this is true of human-written software, ergo there is no evidence that it can evolve (and plenty of evidence that it cannot).

As for Hogan, the depth of his understanding of evolution's "modern synthesis" is suggested by the fact that he become a creationist. (The depth of his understanding of other things is suggested by the fact that he became a Velikovskian (!) and multiple conspiracy theorist. Truly a man with a mind so open that his whole brain fell out.)

Asteroid "mining" with Linux and FOSS

Posted Sep 26, 2013 19:30 UTC (Thu) by Cyberax (✭ supporter ✭, #52523) [Link]

Well, we can imagine that robots use a pre-weighted neural networks or something like it. In this case small changes can, indeed, accumulate.

And yes, the later Hogan's books grow increasingly more and more bizarre. I stopped reading after the fourth one. But the first book in the series is quite brilliant.

Asteroid "mining" with Linux and FOSS

Posted Oct 2, 2013 14:29 UTC (Wed) by oever (subscriber, #987) [Link]

there is no bit-flip analogue of translocation!
Yes there is: instead of moving the code, you flip bits in the address referring to the code.
*None* of this is true of human-written software, ergo there is no evidence that it can evolve
unless it was written to mutate and evolve.

Asteroid "mining" with Linux and FOSS

Posted Oct 4, 2013 11:48 UTC (Fri) by nix (subscriber, #2304) [Link]

Yes, obviously one can design software specifically to evolve under directed mutation, and this has been done and is useful -- but unless we design *everything* that way (including OSes), we'll never get a machine analogue of evolution, and even software designed to evolve under directed mutation isn't going to evolve in the same way under random bit-flips, because the virtual machine that executes that is a) almost certainly much larger than what it's executing and b) not implemented in this fashion, so the first bitflip there and you're probably toast.

Copyright © 2013, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds