Linux and the future of drones

At the 2015 Embedded Linux Conference Europe in Dublin, Lucas De Marchi presented an update on recent developments in the field of Linux-based drones, including a look at where things are headed in the near future.

De Marchi works on the ArduPilot flight-control package, which is one of several open-source drone projects under the umbrella of the Dronecode organization. He identified himself as a relative newcomer to drones, having only started work on ArduPilot in late 2014, but said he has several years of experience working on realtime embedded Linux.

Drones are, first and foremost, hardware projects, regardless of whether they are fixed-wing airplane designs or are quadcopters (or any other rotor-wing design). The same basic software architecture applies in either case, though. The flight-control package (such as ArduPilot) is tasked with keeping the drone aloft and in its intended position. That requires monitoring input from an array of sensors (GPS, altimeters, accelerometers, gyroscopes, magnetometers, and so on). The flight controller's "output" (so to speak) is commands that manipulate the drone's engines: pulse-width modulation (PWM) to control motors, commands sent to a UART or CAN Bus controller, and so forth. A tertiary input is the radio link that most drones have to a ground-control station or RC controller, but that is about the full extent of a flight controller's worries.

Recent developments

The year-to-year changes that can be observed in drones can happen in hardware or in software. In hardware, De Marchi noted that the first ArduPilot code ran on Arduino-compatible microcontrollers. The boards supported have grown in complexity in the years since, adding more processing power, more and better (usually faster) sensors, and additional memory. A beefier board can process more sensor samples, but it also allows the flight controller to use more sophisticated algorithms to track and plot its position. ArduPilot is no exception; as it has implemented more complex positioning algorithms, it reached the limits of the early Arduino boards, and recently had to drop support for the earliest hardware revisions.

But microcontroller-based flight controller boards are no longer the only option. ArduPilot now runs on several small-form-factor Linux boards with the addition of a daughterboard for sensor connections. The BeagleBone Black and Raspberry Pi have been supported for some time now; during the talk, De Marchi switched over to a browser window and filed a pull request to formally add support for a third board, the MinnowBoard Max.

The move from microcontrollers to Linux system-on-chip (SoC) boards is a significant one, he said, because it signifies that the drone industry has moved beyond "dumb" drones of years past. Many people have seen video of drones in controlled environments doing complicated tasks, he said, but the unspoken secret is that most of the higher-level computation in those demonstrations is being done by a separate, ground-based computer that sends commands to the drone over its radio link. This is what is changing; with Linux running on the drone itself, the drone can run high-level programs to perform computer vision (CV), do photography, take survey readings, and much more. Eventually, increases in processing power will make Linux-based drones capable of fully autonomous operation, which is what most of the drone-related predictions about the future tend to focus on.

Putting Linux into drones has had another side effect that few foresaw: drone makers are starting to take advantage of standard Linux features. For instance, radio-control links are no longer the only ground-to-drone communication method; more and more drones are being operated over SSH running over WiFi. Since no additional development is required to implement that sort of standard Linux feature, it is an appealing addition to most drone projects.

On the other hand, he said, the more complexity that gets put into the software running on the Linux side of things, the harder it becomes to manage all of the processes. It is no good running a CV program if it comes at the cost of keeping the drone aloft. This is why many Linux-powered drones choose to offload sensor-reading and radio-link management to microcontrollers or to coprocessors running a realtime OS (RTOS). It is a serious misconception that Linux cannot keep up with the realtime requirements of a drone flight controller; many drones use the PREEMPT_RT patch set, and De Marchi has no problem running ArduPilot on a standard kernel.

The misconception stems from fear that missing a sensor-reading timer will crash the drone, he said, but this is a misunderstanding. All fight-control software has to be robust against missed or bad sensor readings: readings are buffered and multiple samples are averaged. Even RTOS flight controllers encounter bad readings and bugs of other sorts; drones cope with the adversity.

Nevertheless, as Linux drones start running more complicated high-level software, it often makes sense to offload the repetitive tasks (like the tight sensor-reading loop) to a cheap microcontroller, and let the CPU spend its valuable time on more CPU-intensive things.

The future

Making more and more processor cycles available to high-level applications is one of the concerns that De Marchi predicted would be important to Linux drone projects in the near future. Those that do not offload sensor input to another board will likely rewrite their sensor support, moving it from user space into the kernel. User-space sensor reading requires sampling the sensors over the SPI or I2C buses directly (which requires context switching) and it requires ArduPilot (or other flight-control projects) to write sensor drivers and to maintain a bus-abstraction layer.

The kernel-based solution will make use of the Industrial I/O (IIO) subsystem, which is already used by Android and other mobile Linux platforms to handle sensors. The code sharing will reduce work for drone projects, although it does have the unfortunate side effect of making the Linux and microcontroller versions of ArduPilot diverge. De Marchi did express his hope that, given the Linux Foundation's announcement of the formal Real-Time Linux Project, the PREEMPT_RT patch set will soon make its way into the mainline kernel.

The other changes coming in the future include support for a range of new sensor types, such as LIDAR, optical-flow sensors, and various chemical sensors. Most of these additions are being driven by high-level applications, which is a sign of the increasing maturity of the Linux drone industry. Using IIO for sensors will also enable future drones to support multiple simultaneous sensors (such as redundant altimeters or gyroscopes). That will enable more autonomy, and will allow for even more sophisticated flight-management algorithms to be implemented. De Marchi also noted that, when his official MinnowBoard Max pull request is merged (as it almost certainly will be), ArduPilot will have added its first x86 support. The addition of new and larger platforms will probably also be more common in the future, as onboard processing power will continue to rise.

The session ended with questions from the audience. Most were interested in getting started with Linux drones. The cost of a "starter" drone is several hundred dollars, it seems, which is not insignificant, but as interest in the field grows, one should expect to see more and more options reach the market, perhaps bringing costs down.

[The author would like the thank the Linux Foundation for travel assistance to attend Embedded Linux Conference Europe.]