User: Password:
|
|
Subscribe / Log in / New account

LWN.net Weekly Edition for June 14, 2012

LinuxCon Japan: OpenRelief launches

By Jake Edge
June 13, 2012

Shane Coughlan got hands-on experience in the kinds of problems faced by disaster aid teams while helping with relief efforts after Japan's earthquake, tsunami, and nuclear accident in 2011. That experience was part of a discussion on open source technical measures to assist such efforts at LinuxCon Japan 2011, but it also led to a new project, OpenRelief, that was announced at LinuxCon Japan 2012. The project is aimed at developing inexpensive—"disposable"—drone aircraft to assist relief teams in seeing over the horizon to detect people, changes in terrain, smoke, radiation, and other conditions in places that may be difficult or dangerous for on-the-ground exploration.

[OpenRelief and airplane]

OpenRelief co-founders Coughlan, who is a consultant based in western Japan, and Karl Lattimer, who is UK based, came to Yokohama to announce the project and to show off the prototype drone aircraft that Lattimer built. The plane itself was an eye-catching prop for the talk, but some of the most interesting parts of OpenRelief are on the inside: open source software for route following, image acquisition and processing, and so on. Much of that code comes from existing projects, with OpenRelief integrating it into the airframe to create a mobile reconnaissance platform.

Problems and solutions

Gathering information on the state of various locales was one of the biggest problems that Coughlan saw when bringing aid from western Japan to areas affected by the earthquake and tsunami. There were locations that had supplies and doctors, but didn't know where to take them. In addition, sometimes aid arrived at locations that were already fully stocked and had no more storage, so the aid had to return back to where it came from.

The situation was "like a big fog" covering the disaster zone, he said. OpenRelief is trying to help penetrate that fog using technical measures. It would act as a supplement to existing response mechanisms. The goal is "for the responders on the ground to do their job more effectively", Coughlan said, so they don't go where they aren't needed and do go where they are. That was "quite a challenge" last year in responding to the earthquake.

"So, obviously the solution is a robot airplane", he said with a chuckle. More seriously, a robot airplane can help answer some of the questions that are hard to get answers to like "can we get there?" or "do we need to go there?". There were situations where a car couldn't get through to a particular nearby location to assess the situation, but "an airplane could have gone there to see".

Robot airplanes (or drones) have gotten a bad reputation in places like Afghanistan, Coughlan said, but they can be "immensely useful". Unlike those in various war zones, these airplanes are "full of love and peace". They are intended to provide a low-cost solution for mapping and investigating disasters.

The plane will be able to take off and land from a foot path, fly a pre-programmed route, and gather various kinds of information while traveling. Using on-board processing, it will be able to recognize roads, people, and smoke. There are also a variety of sensors that can be deployed to collect weather data, radiation levels, or other kinds of environmental conditions which can relay data via the plane.

The plane and its capabilities are "not really news", Coughlan said, as the technology has been available for some time. OpenRelief has just tied together multiple off-the-shelf and open source pieces for its use case. The technology is "phenomenal and astonishingly cheap". With that, he turned things over to "someone who can build stuff", Lattimer that is, to describe more of the technical details of the plane.

The guts of the plane

It took about a week to assemble one of the drones, Lattimer said, and few more days to finish it. The airframe has a simple design that comes mostly already constructed. It is made out of fiberglass and covered in plastic vinyl. The first that he built was "a challenging project", but the second was much easier.

The plane has an autopilot system, the Arduino-based ArduPilot, which uses a combination of GPS, airflow monitors, and air pressure sensors to fly the plane on pre-programmed flight plans. The flight plan can contain up to 600 3D waypoints that can be reprogrammed in flight from the Raspberry Pi main controller. It takes off using a standard radio controller, then the autopilot is turned on and the plane follows the flight plan.

The Raspberry Pi is "ideal" for an initial test computer, Lattimer said, because of its low cost, but other, faster main CPUs are possible down the road. The Raspberry Pi Another board used for testing (Samsung Orion Exynos) has a Mali graphics chip, which was reverse engineered by his employer, Codethink, and can be used to do a variety of image processing tasks. The main board runs Debian 6.0 ("Squeeze").

For imaging, the plane uses a CCD with a 170° fisheye lens that provides five separate images. Those feed into the Open Source Computer Vision (OpenCV) package on the Raspberry Pi for doing visual recognition of smoke, people, and roads. There are also plans to do "structure from motion" (SfM) processing to detect landscape changes (e.g. flooding, height changes) but that is fairly processor intensive and will likely require processing after the plane returns from its mission.

Lattimer also described a low-cost, ground-based radiation sensor that can be dropped or placed in areas of interest to relay its readings to the plane. He built the sensor inside of a treacle tin (for those lacking the UK context, it is a can that held a syrup not entirely unlike molasses). The sensor employs an ionization chamber that measures relative ionizing radiation, rather than absolute radiation levels like a Geiger counter would do. It uses a Nanode controller, which is an open hardware device with a long-range, low-power radio to communicate with the plane. Other types of sensors (for chemicals or weather conditions) could also be built using the Nanode.

Mission control

There is a need to tie the robot and sensors together, Coughlan said, and that is where "mission control" comes into play. Most of the technology to do that already exists, so OpenRelief has just integrated those pieces onto a laptop that runs Debian, Ubuntu, or Windows. For example, the autopilot has a sophisticated mission planning application that is written for .NET, but can be run with Mono on Linux.

The output from the mission control system is formatted to be compatible with existing disaster relief packages (e.g. Sahana Eden). The information gathered can be processed into geographic information system (GIS) formats that can be directly integrated into the maps used by those applications. Rather than trying to reinvent the disaster relief application wheel, OpenRelief is adding new capabilities to the existing systems, Coughlan said.

That is one of the keys to the OpenRelief plan: integrating mostly available open hardware and software into the existing systems so that the drones can be put to work in the near future. The system uses OpenStreetMap data for its maps and can even contribute updates to that map repository for things that have changed. "Working alongside existing processes is critical", Coughlan said. During last year's discussions there was talk of redoing much of the disaster relief infrastructure, but that is not the route that the project took; "we just want to fit in with what's there".

The project started, at least conceptually, at the disaster relief panel in 2011. After thinking about airframes, autopilots, people recognition, and the like for a bit, development started in January. The team will be testing and refining the prototype with the hope of being production-ready in December.

The equipment is relatively inexpensive, with a retail bill of materials (BoM) for the prototype at around $750. Getting it into any kind of manufacturing process will make it "ridiculously cheap", he said. The target for the final BoM is $1000 which may include a more powerful main CPU (perhaps Tegra-based) and additional capabilities.

Help wanted

The team is around 25 people currently, consisting of a variety of specialties including engineers, makers, political scientists, mathematicians, and more. The team started out with "crazy ideas", but "those ideas turn out to not be crazy at all", Coughlan said. There is still lots of work to be done, but "it is doable" and the project is looking for help to get it done. The project is hoping to find people to donate time to develop, test, and improve the hardware, but it is looking beyond that as well.

OpenRelief is "kicking off with an ecosystem", he said. Coughlan's company Opendawn along with Codethink and Nanode have all made donations of hardware and time. But there are also a lot of individuals involved. "We want you to join our ecosystem". The project is looking for "your brains, not necessarily your money" (though he admitted it wouldn't turn down monetary contributions).

The ecosystem needs technologists as well as professional and volunteer relief workers to help refine and test the platform, and to help recognize the problems that need to be solved. In addition, there is a need for commercial enterprises to "make buckets of money" by building and selling the drones. The project's focus is to help save lives, but the platform could easily be repurposed for other uses, including for farmers or local governments. While it won't be useful "for anything naughty", Coughlan said, because it is a very visible and slow plane that won't be stealthy, there is a need for this kind of technology for various uses all over the world.

He invited everyone to check out the web site (which has been translated from English into several Asian languages), mailing lists, and various social media (Facebook, Twitter, and Pinterest) for more information. The slides [PDF] from the talk also give more technical details for those interested. Much of the code (with more coming) is on Gitorius and schematics are available at solderpad.com.

So far, the plane has only been flown in radio-controlled mode, at least partly because of regulations in Japan. Lattimer hopes to test with the autopilot sometime in the next month in a free-fly zone in the UK. Regulations on autonomous aircraft vary, but will be a challenge for some time, Coughlan said. He is hopeful that the disaster relief use case, as well as the very limited threat posed by a 3kg aircraft, will help change those regulations, though it will take time.

OpenRelief is an interesting project that combines a certain "geek appeal" with a useful function for reconnaissance in a disaster area. One can certainly imagine low-cost drone aircraft being employed in a variety of situations, but one that can potentially save lives by getting aid where it is needed most is more interesting still. By supplementing existing disaster relief systems—rather than trying to supplant them—OpenRelief's drones could be a nice addition to the disaster relief toolkit. One gets the sense that the drone is just the start, however, so it will be interesting to see what else the project comes up with down the road.

[ The author would like to thank the Linux Foundation for assistance with his travel to Yokohama. ]

Comments (6 posted)

The accounting quest: Ledger

By Jonathan Corbet
June 13, 2012
Last month, your editor described LWN's need for a non-proprietary accounting system and the difficulty of finding such a thing in the free software community. While there is no shortage of systems that can perform basic double-ledger accounting, it is rather harder to find a system that has everything needed to actually run a business. Since then, a number of people have mentioned a system simply called Ledger. After some experimentation, it is now time to pass on some impressions of how this system works.

Ledger takes an interesting approach to the problem in a couple of ways. One is that it is a purely command-line program; there is no graphical interface of any type. The other is that Ledger does not actually maintain any sort of database of accounting data; instead, it reads that data from a plain-text file that the user manages separately. The result is a system that is quite powerful in some ways while falling short in others.

The file format is relatively simple; entries span multiple lines, with all but the first being indented. An example shipped with the program looks like this:

    2004/05/27 Book Store
      Expenses:Books                 $20.00
      Liabilities:MasterCard

This transaction took place in a book store, where $20 was moved from the Liabilities:MasterCard account to Expenses:Books. The final line lacks a dollar value, so Ledger fills it in to make the transaction balance; more complex split transactions would require explicit values for each account involved. The transaction above has not been reconciled; otherwise there would be an asterisk between the date and the description fields.

Other types of entries are possible. There is a mechanism for adding "period entries," describing transactions that happen at regular intervals. These entries are not really a form of recurring transaction—ledger seems to lack support for those. Instead, period entries are used to describe budgets that can be compared against actual income and expenses. "Automated transactions" are also supported, but those aren't recurring transactions either; they are a mechanism to add splits automatically to other transactions. The syntax for both types is terse and arguably obscure (period entries start with a "~", for example); it can take a little while to understand everything that can appear in the ledger file.

What has been described thus far doesn't involve the Ledger program at all; this file must be created and maintained via some other mechanism. In many cases, that mechanism will be a text editor; inevitably, there is an Emacs mode to make things easier. For those who want a more graphical experience, Ledger can read (uncompressed) GnuCash files, but there are a couple of caveats. One is that the Ledger developer (John Wiegley seems to do almost all the work on Ledger) is somewhat unenthusiastic about this approach; the manual asks: "why would anyone use a Gnome-centric, multi-megabyte behemoth to edit their data, and only a one megabyte binary to query it?" The other is that this support appears to be lacking from the in-development 3.0 release. So, while using GnuCash works for now, it's not clear that it will continue to work in the future.

One could also imagine various automated mechanisms for generating the file from other data sources. Ledger itself never modifies the file; it is simply a consumer of the data contained therein.

As consumers go, it's a powerful one. There is a fairly long list of reports that can be generated. As one might expect, Ledger supports a complex command-line interface that can be used to filter transactions and control the format of the resulting report. The syntax is (once again) terse, but, given time, a suitably motivated user can create almost any sort of text-oriented report one might like. Pie chart aficionados will be disappointed at the outset; there is no graphical output from Ledger. With a bit of work, though, one can get Ledger output into a tool like Gnuplot and produce all kinds of pretty charts.

In summary, Ledger isn't really an accounting system; it's a report-generation utility designed to work with a specific text file format. There are both advantages and disadvantages to its approach. On the up side, Ledger could be an ideal solution for relatively small operations where the people involved are happy to use command-line utilities and do some scripting to make the pieces of a full business hold together. The plain-text ledger file is ideal for anybody wanting to use a tool like Git to track changes over time (or to merge changes done by more than one user). The tool is fast, lightweight, and will never imprison one's accounting data in a black-box binary data file.

On the other hand, it is hard to imagine scaling Ledger even to a business the size of LWN, much less something larger. The tool has clearly been written with speed in mind; it can process a file with many thousands of transactions quickly. But the file itself can only become unwieldy, especially if there are many sources of data feeding into it. Plain-text files lose some of their appeal when they get to be tens or hundreds of thousands of lines in length.

Ledger also lacks features that a typical small-business accounting program would be expected to support. These include:

  • Invoicing. One can certainly enter information about an invoice as an accounts receivable entry, but there is nothing tying it to the actual invoice sent to a customer. Anybody wanting standard invoice formats, summaries of outstanding invoices, or per-customer reports will have to implement them separately.

  • Check printing. Yes, it is still necessary to write checks at times.

  • Integration with bank accounts. That said, LWN has to do its own processing of files from banks before feeding the data into QuickBooks now, so things would not necessarily change a lot in a move to Ledger.

  • Recurring transactions. Again, one can certainly script such things, but it must be done separately.

  • Tracking of contractors and creation of tax data (and, in the US, 1099 forms).

  • Easy execution of routine tasks like account reconciliation. Ledger can certainly provide a register view showing unreconciled items, but the editing of those items must be done elsewhere (in that big text file).

The other problem with a tool like Ledger is that it can complicate dealings with one's accountant, assuming said accountant is not comfortable with command-line tools. Generating the basic reports that an accountant would want to see will not be that hard. But a good accountant will want to dig through the entries to be sure that everything makes sense and, perhaps, add a journal entry or two to straighten them out. Ledger makes that level of interaction impossible for most accountants, and that will complicate life for many businesses trying to use it.

The bottom line, so to speak, is that Ledger is probably not the solution to LWN's accounting problems, though it might work quite well for simpler businesses with fewer customers, transactions, and accounts. For us, there are too many pieces that would have to be added, some of which would be less than straightforward. So the quest continues; stay tuned.

Comments (8 posted)

On mocking

By Jonathan Corbet
June 13, 2012
LWN's coverage from LinuxCon Japan included an article on a talk by Greg Kroah-Hartman on the kernel development process and, in particular, how to avoid making kernel subsystem maintainers grumpy. The article got a number of comments, almost all of which were inspired by its last sentence, which read: "Sometimes public mocking is part of the process and can actually help instill that pride more widely." Quite a few LWN readers clearly see the mocking of contributors as a problem; some accused LWN of making the problem worse. Perhaps it is a time for a look at the role of mocking in free software development discussions.

There can be no doubt that the environment in our mailing lists, IRC channels, bug trackers, and other electronic communication channels can be intimidating and off-putting. Some developers handle such environments without trouble; others go away unhappy, and, perhaps, never return. Awareness of this problem has grown over the last decade or two, and, as a general rule, things have gotten better. Even so, one need not look too hard to find examples of harsh messages from high-profile developers. One might well wonder why such behavior persists.

One possibility is that the meritocratic nature of our community makes us willing to tolerate any sort of behavior in the name of technical excellence. We need top-level developers and will put up with unpleasantness if they can produce the code; as Rusty Russell once put it: "If you didn’t run code written by assholes, your machine wouldn’t boot." We want our machines to boot, so we have to accept the developers who can make that happen. There is almost certainly some truth to this claim; it is the same calculation that leads us to accept unpleasant politicians, doctors, managers, and plumbers.

That said, there is evidence that, in our community, we are becoming less accepting of such behavior than we once were. The recent changes in the GNU C library project could be seen as an example.

In the conversation following the LinuxCon article, it was asserted that the kernel development community continues to grow. That was put forward as evidence that things can't be all that terribly bad—the community is doing enough things right that people still want to be a part of it. Once again, there must be some truth to that claim. But it is also hard to imagine that the community is so rich in developers that it need not worry about those who do not handle the environment well. In truth, the community does worry about those developers. Improving the environment in the mailing lists and beyond is a perennial kernel summit topic, and has been the focus of a lot of private communications as well. For all its fearsome reputation, certain counter-examples notwithstanding, linux-kernel is a far more pleasant place than it used to be.

That said, one can still run into trouble on a list like linux-kernel, or in many other places. The previous LWN conversation hinted at a plausible reason for much of the remaining hostility: the lack of reviewer bandwidth. Any project that reaches a certain size tends to discover that it has far more people posting code and asking questions than people who review that code and answer the questions. Review bandwidth tends to be the limiting resource that slows the development process as a whole. Projects can try to deal with this problem in a number of ways; they can, for example, force developers to perform reviews to get their own code merged, or they can simply slow their process to the rate their reviewers can handle. A third alternative—skipping the code review step—has also been tried, but it tends not to produce pleasing results.

In this situation, anything that wastes reviewer time and slows the process further is going to be unwelcome. Additionally, code review can be a time-consuming, tedious, and thankless task; code reviewers are, as a result, often grumpy. Mercurial maintainer Matt Mackall recently described it this way:

Being the project leader puts you in a role of being the defacto bad guy. Someone has to make decisions and some of those decisions are going to be "no". And many of those "no" decisions are ones that each wave of newcomers will question. So I spend lots of my time saying "no, compatiblity", "no, known bad idea", "no, design choice", "no, performance" at newcomers, and I _really don't enjoy it_. And because no one else enjoys it either, I end up doing the bulk of it. Burnout++, every day

Code reviewers often see the same kinds of problems over and over again, either because developers don't read the documentation, don't pay attention to previous discussions, or do not listen to the advice they have been given. When this happens, it is only human nature to strike out with strong words in the hope of making the irritation go away so that some real work can get done. This kind of response can be seen on the lists for a wide variety of projects.

Of course, one could just as easily argue that it's basic human nature to club one's neighbor with a rock to get at that nice stash of saber-toothed tiger bones in his cave. Anybody who has raised children understands how long it takes to learn to function properly in human society. So the fact that it's natural to strike out against mailing-list irritations does not mean that doing so is correct. Much of the time, it's the sort of impulse that we all (hopefully) learn to resist as we grow up.

As the free software development community has grown up, it has, indeed, learned to resist many of those impulses. We have become more adult, and more professional. Most communities tolerate far less unpleasant behavior than they once did; the kernel community can certainly be included in that group. For all the talk of public mocking, it does not actually happen all that often.

In the end, though, we live in a world that is not perfect. There will be people who come into our communication channels and call down impoliteness upon themselves; our world still contains trolls, people who feel entitled to free services from others, fanboys and fanatics who do not understand what "no" means, and those who simply refuse to listen. Some communities will respond to such people by trying to flame them off the list. Others might just quietly ban them from the list—an effective solution, but not necessarily a more civilized one. Not all who are flamed or mocked deserve it, but attempts to eliminate such behavior altogether risk being a cure that is worse than the disease.

In the end, there is value in having a sense of humor about things. It is not at all hard to find examples of developers being called morons or idiots on linux-kernel, but, much of the time, those developers are directing those words toward themselves. We are all morons sometimes; occasionally, we will get caught at it in public. Our community is usually quite accepting of those who understand when they've had one of their moments; it's those who refuse to listen who get the worst of it. Yes, we can do a lot better at dealing with each other respectfully, but, in the process, we should not close our community to real discussion, expect perfection, or lose track of how well we're doing now.

Comments (47 posted)

Page editor: Jonathan Corbet

Inside this week's LWN.net Weekly Edition

  • Security: MySQL flaw leaves some systems wide open; New vulnerabilities in FlightGear, java, kernel, php, ...
  • Kernel: The word-at-a-time interface; One zImage to rule them all; A big.LITTLE scheduler update.
  • Distributions: Guacamayo media player distribution; CentOS, Debian, ...
  • Development: printerd; Emacs, MPlayer, X on Wayland, ...
  • Announcements: LF Linux Scholarship program, IFLOSSLR, Torvalds interview, Apple v Google, ...
Next page: Security>>

Copyright © 2012, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds