Leading items

GUADEC: open source and open "stuff"

Developer conferences like GUADEC tend to be dominated by technical content, so talks with a different tenor stand out. Alex "Skud" Bayley's July 28 keynote "What's Next? From Open Source to Open Everything" was one such talk. Bayley has spent time in both the open source and open data movements, and offered a number of insights between those and other online, grassroots-community movements — including what open source can teach newer communities from experience, and what it can learn.

Bayley's talk was based loosely on a blog post she wrote in January 2011 while working at the Freebase "open data" project. After roughly a decade of working full-time on open source software, she decided it was no longer the "fringe" and cutting-edge movement it was in the early years, and consequently stopped being an interesting challenge. That assessment is not a criticism, however; as Bayley put it, the fact was that open source software had "won." "No one seriously says 'lets use ColdFusion running on Oracle' for their Web site anymore". She subsequently encountered the open data movement and found that it "freaked out enough people" that she was certain she was onto something interesting again.

But the real insight was her discovery that the nascent open data movement was grappling with the same set of challenges that open source software had tackled roughly a decade earlier. For example, it was grappling with licensing (as open source had), and was still in the process of distilling out its core principles and how best to enshrine them in appropriate licenses. Similarly, she said, in the early days open source struggled to build its software support tools (such as build systems and version control), find working business models, and discover how to interact with governments and other entities that found the movement odd or suspicious.

Open data was repeating the same process, ten years later. Bayley relayed an anecdote about a New Zealand government project that attempted an open data release as a Zip archive. Nat Torkington reacted with a number of questions illustrating how a Zip release failed to make the grade: what if there is a bug, or an update, or a patch? Open source and open data are not the only movements, however: Creative Commons and Wikipedia have dealt with similar issues, as have open education, open healthcare, open government, open access (e.g., to academic research), and open hardware — and Bayley found the parallels interesting. In short, when asked what her current interest is, she now replies "open ... stuff".

An even broader circle encompasses not only the open technology movements, but other recent grassroots and peer-to-peer online communities, including "scanlation" groups that crowdsource translations, music or video remixing communities, unconferences, and even fan-fiction communities. Some of these groups might not seem to have any connection to open source, Bayley admitted, but the parallels are there: they are self-organizing, decentralized, non-hierarchical, and are based around the tenet of making things free. That makes them kindred spirits that open source could assist based on its own experiences, and it makes them worth learning from.

What open source can teach

The first area in which open source can offer assistance to other open movements is licensing, Bayley said. Licensing is "really, really important", but most online communities don't think about it in their early days. Open source has generally settled on a small set of licenses that cover most participants' needs, and it has done so largely because it started from the FSF's four freedoms. Newer communities could benefit from open source's work articulating its core principles, writing definitions, and figuring out the boundaries that determine "who's in and who's out".

She cited several examples in the open data movement where lack of licensing standards confuses the issue. One was a genealogy site advertising its "open data" at OSCON 2010 — data that was licensed CC-Attribution-NonCommercial-NoDerivatives-ShareAlike, a choice that breaks three of the FSF's four freedoms. Another was a "community mapping" project run by Google, which used participatory and community language in its marketing, but in which all contributions became the sole property of Google.

The second area where open source can assist other movements is tools. Open source has a complete toolchain all the way down the stack, but many other communities do not. Many creative- or publishing-centric communities have no concept of version control, she said. But telling ebook authors to "just use GitHub" is not the answer; they would balk at the suggestion, and rightfully so. Rather, the open source community needs to ask "what would DocHub look like?" and help the community build the tools it requires.

Finally, open source can teach other communities about the value of working and communicating completely in the open: open mailing lists, open documentation, and "release early, release often" workflows. The benefits may seem obvious to open source developers, but it is a scary prospect to those not "soaking in it" already, like the open government movement. But transparency has benefits for all open source communities, she said. It allows outsiders to see what the community is like and how it operates, so that they can put themselves into the situation with fewer surprises. It also means more accountability, which is particularly important in movements like open government.

What open source can learn

Open source software's relative maturity puts it in a position to offer experience-based advice to other online communities, Bayley said, but that fact does not mean the other communities have nothing to teach of their own. After all, she said, no one recruits the thousands of teenagers who write and share their own Harry Potter fan-fiction — they build and organize their own communities online. There are several potential lessons from these other groups, which she described in random order.

The first is the value of hands-on events. Hackerspaces often hold short, practical events where "people can walk-in, learn something, and walk out." Open source rarely does this, expecting newcomers instead to sign up for online courses or figure out what to do on their own. But "no one says 'I spent all weekend reading documentation; it was awesome!'" Many minority or marginalized groups in particular require a slight push to get involved; a physical event after which they can walk away having learned something will provide this push in ways an online event cannot. It is easy — but fundamentally selfish — to tell others that they must learn it hard way because you did, she said.

Many other open communities also have much more age diversity than open source. Environmental groups and music communities tend to be all-age, she said, but open source is not. Developer conferences like GUADEC tend to be dominated by attendees in their 20s and 30s, while system administration conferences are much older. But as an example, she asked, why are there no children at GUADEC? Some might expect them to find the talk sessions dull, or to be disruptive, which is valid, but there could still be other program content designed to reach them. She told a story about a punk music venue in Berkeley California that held only all-age concerts, and how she witnessed adults helping kids enjoy the mosh pit by lifting them onto their shoulders. "If you can run a public mosh pit for kids, you can probably solve the problem for linux.conf.au".

Finally, many other open communities operate with a strong "nothing about us without us" ethic. The phrase comes from the disability rights community, and it means that the community tries not to embark on projects ostensibly for disabled people unless there are people with disabilities involved. Otherwise, the results can easily fail to meet the needs of the target community.

An example of failing to exercise this approach happened after the 2010 Haiti earthquake. After the quake, a number of open source developers volunteered to write software to support the relief effort, but did so without partnering with the relief workers on the ground. The developers felt good about themselves — at least at first — but were ultimately disappointed because their efforts were not of much practical help. In addition to producing better outcomes, she said, the "nothing about us without us" approach has the added benefit of empowering people to build things for themselves, rather than building things for them.

Bayley's talk encompassed such a wide view of online and "open something" communities that at first it was hard to see much that connected them. But in the end, she is right: even if the reason that the other community congregates has nothing to do with the motives that drive open source software, these days we have a lot in common with anyone who uses the Internet to collaborate and to build. In her first few years of open source involvement, Bayley said, she frequently told people to switch over to Linux and open source software in tactless ways that had little impact. She hopes that she is more tactful today than she was at 18, she said, because open source has lessons to teach about freedom and community. Those lessons are valuable even for communities that have no interest in technology.

[The author would like to thank the GNOME Foundation for travel assistance to A Coruña for GUADEC.]

Comments (1 posted)

GUADEC: Motion tracking with Skeltrack

At the beginning of his GUADEC 2012 talk, developer Joaquim Rocha showed an image from Steven Spielberg's oft-cited 2002 film Minority Report. When the movie came out, it attracted considerable attention for its gesture-driven computing. But, Rocha said, we have already surpassed the film's technology, because the special gloves it depicted are no longer needed. Rocha's Skeltrack library can leverage Microsoft Kinect or similar depth-mapping hardware to find users and recognize their positions and movements. Skeltrack is not an all-in-one hands-free user interface, but it solves a critical problem in such an application stack.

Rocha's presentation fell on Sunday, July 29, the last "talk" day of the week-long event. Although GUADEC is a GNOME project event, the Skeltrack library's primary dependency is GLib, so it should be useful on non-GNOME platforms as well. Rocha launched Skeltrack in March, and has released a few updates since. The current version is 0.1.4 from June, and is available on GitHub. For those who don't follow Microsoft hardware, the Kinect uses an infrared illuminator to project a dot pattern onto the scene in front of it, and an infrared sensor reads the distortion in the pattern to map out a "depth buffer" of the objects or people in the field of view.

How it works

Like the name suggests, Skeltrack is a library for "skeleton tracking." It is built to take data from a depth buffer like the one provided by the Kinect device, locate the image of a (single) human being in the buffer, and identify the "joints." Currently Skeltrack picks out seven: one head, two shoulders, two elbows, and two hands. Those joints can then be used by the application, letting the user manipulate objects, or for further processing (such as gesture recognition). The Kinect is the primary hardware device used with Skeltrack, Rocha said (because of its low price point and simple, hackable USB interface), but the library is hardware independent. Skeltrack builds on the existing libfreenect library for device control, and includes GFreenect, a GObject wrapper library around libfreenect (because, as Rocha quipped "we really like our APIs in GNOME").

One might be tempted to think that acquiring the 3D depth information is the tricky part of the process, and that picking a human being out of the image is not that complicated. But such is not the case. Libfreenect, Rocha said, cannot tell you whether the depth information depicts a human being, or a cow, or a monkey, much less identify joints and poses. There are three proprietary ways to get skeleton information out of libfreenect depth buffers: the commercial OpenNI framework, Microsoft's Kinect SDK, and Microsoft's Kinect For Windows. Despite its name, OpenNI includes many non-free components, the skeleton-tracking module included. The Kinect SDK is licensed for non-commercial use only, while Kinect for Windows is a commercial offering, and only works with the desktop version of the Kinect.

Moreover, the proprietary solutions generally rely on a database of "poses" against which the depth buffer is compared, in an attempt to match the image against known patterns. That approach is slow and has difficulty picking out people of different body shapes, so Rocha looked for another approach. He found Andreas Baak's paper A Data-Driven Approach for Real-Time Full Body Pose Reconstruction from a Depth Camera [PDF]. Baak's algorithm uses pattern matching, too, but it provided a valuable starting point: locating the mathematical extrema in the body shape detected, then proceeding to deduce the skeleton.

Heuristics are used to determine which three extrema are most likely to be the head and shoulders (with the head being in the middle), and which are hands. Subsequently, a graph is built connecting the points found, and analyzed to determine which shoulder each hand belongs to (based on proximity). Elbows are inferred as being roughly halfway along the path connecting each hand to its shoulder. The result is a skeleton detected without any "computer vision" techniques, and without any prior calibration steps. The down side of this approach is that for the moment it only works for upper-body recognition, although Rocha said full-body detection is yet to come.

How to use it

Skeltrack's SkeltrackSkeleton object has tweakable parameters for expected shoulder and hand distances, plus other measurements to modify the algorithm. One of the more important parameters is smoothing, which helps cope with the jitter often found in skeleton detection. For starters, Kinect depth data can be quite noisy, and on top of that, the heuristics used to find joints in the library result in rapid, tiny changes. Rocha showed a live demo of Skeltrack on stage, and with the smoothing function deactivated, the result is entertaining to watch, but would not be pleasant to use when interacting with one's computer. The down side is that running the smoothing formula costs CPU cycles; one can maximize smoothing, but the result is higher latency, which might hamper interactive applications.

Rocha also demonstrated a few poses that can confuse Skeltrack's algorithm. For example, when standing hands-on-hips, there are no "hand" extrema to be found, leading the algorithm to conclude that the elbows are hands. With one hand raised head-height and the corresponding elbow held at shoulder height (as one might do while waving), the algorithm cannot find the shoulder, and thus cannot figure out which of the extrema is the head and which is the hand. Nevertheless, Skeltrack is quite good at interpreting common motions. Rocha demonstrated it with a sample program that simply drew the skeleton on screen, and also with a GNOME 3 desktop control application. The desktop application is hardcoded to a handful (pun semi-intended) of actions, rather than a general gesture input framework. There was also a demo set up at the Igalia (Rocha's employer) expo booth.

Skeltrack provides both an asynchronous and a synchronous API, and it reports the locations of joints in both "real world" and screen coordinates — measured in millimeters in the original scene and pixels in the webcam image. Currently the code is limited to identifying one person in the buffer, but there are evidently ways to work around the limitation. Rocha said that a company in Greece was using OpenCV to recognize multiple people in the depth buffer, then running Skeltrack separately on each part of the frame that contained a person. However, the project in question was not doing the skeleton recognition in real-time.

Libfreenect (and thus Skeltrack) is not tied into the XInput input system, nor is Skeltrack itself bound to a multi-touch application framework. That is one possible direction for the code to head in the future; hooking Skeltrack into the same touch event and gesture recognition libraries as multi-touch pads and touch-screens would make Kinect-style hardware more accessible to application developers. But that cannot be the endpoint — depth buffers offer richer information than 2D touch devices; developers can and will find more (and more unusual) things to do with this new interface method. Skeltrack is ahead of the competition (libfreenect lacks skeleton tracking, but its developers recognize the need for it), and that is a win not just for GNOME, but for open source software in general.

[The author would like to thank the GNOME Foundation for travel assistance to A Coruña for GUADEC.]

Comments (none posted)

The Nexus 7: Google ships a tablet

When life presents challenges, one can always try to cope by buying a new toy. In this case, said new toy is the Nexus 7 tablet, the first "pure Android" tablet offered directly by Google; it is meant to showcase what Android can be on this type of device. The initial indications are that it is selling well, suggesting that the frantic effort to prepare Android for tablets are finally beginning to bear some fruit. What follows are your editor's impressions of this device and the associated "Jelly Bean" Android release.

The Nexus 7 (N7) is an intermediate-size tablet — larger than even the biggest phones, but smaller than, say, a Xoom or iPad device. It features a 7" 1280x800 display and weighs in at 340 grams. There's 1GB of RAM, and up to 16GB of storage; the CPU is a quad-core Tegra3 processor. The notion of a quad-core system that fits easily into a back pocket is amusing to us old-timers, but that's the age we live in now. The N7 features WiFi connectivity and Bluetooth, but there is no cellular connectivity; it has 802.11n support, but cannot access the 5GHz band where 802.11n networks often live. The only camera is a front-facing 1.2 megapixel device; the N7 does not even have the camera application installed by default.

The N7 runs Android 4.1.1, the "Jelly Bean" release. 4.1.1 offers a lot of enhancements over 4.0, but is, for the most part, similar in appearance and functionality. The first impression, once the setup formalities are done, can be a little disconcerting: the home screen is dominated by a large ad for Google's "Play Magazines" service. It makes one think that "pure Android" devices might be going the crapware route, but the ad widget is easily disposed of and never appears again.

As of this writing, there is no CyanogenMod build available for the N7. That is unsurprising, given the newness of the hardware and the fact that CyanogenMod has not yet moved to the Jelly Bean release. But the N7 is an unlocked (or, at least, easily unlockable) device, so one can expect that alternative distributions will become available for it in due time.

Using the N7

Android on tablets has matured considerably since the initial "Honeycomb" release featured on the Xoom. For the most part, things work nicely, at least as far as the standard Google applications are concerned. The ability of third-party applications to work well on larger screens is still highly variable. One bit of remaining confusion is the "menu" button, which appears in different places in different applications, or is absent altogether. Playing the "find the menu" game is a common part of learning any new application. One gets the sense that the Android developers would like to do away with menus altogether, but there are many practical difficulties in doing so.

Perhaps the most jarring change is the switch to Chrome as the built-in web browser. The standard Android browser wasn't perfect, but it had accumulated some nice features over the years. Chrome is capable and fully-featured, and it arguably makes sense for Google to focus on supporting a single browser. But your editor misses the "auto-fit pages" option and the "quick controls" provided by the Android browser. Getting around with Chrome just seems to be a slower process requiring more taps and gestures. Undoubtedly there is a way to get the Android browser onto the N7, but, so far, time has been short and a quick search came up empty.

The N7's front-facing camera is clearly not meant for any sort of photographic use, unless one is especially interested in self portraits. It is useful for the "face unlock" feature, naturally. It is also clearly meant for use with applications like Skype; the N7 should make a very nice video network phone. Unfortunately, video calls in Skype fail to work on your editor's device. Some searching indicates that it works for some people and fails for others; sometimes installing the camera application helps, but not in this case. At this time, the N7 does not appear to be ready for this kind of use.

One need not have an especially conspiracy-theoretical mindset to surmise that Skype's owner (a small company called "Microsoft") might just have an incentive to ensure that Skype works better on its own operating system than on Android. But the truth of the matter is probably more prosaic: by all accounts, the Skype application is just not an example of stellar software engineering. Unfortunately, it is an example of proprietary software, so there is no way for anybody but Skype to fix it. There should really be a place for a free-software video calling application that (1) actually works, and (2) can be verified to lack backdoors for government agencies and anybody else interested in listening in on conversations. But that application does not seem to exist at this time, alas.

Electronic books

Another obvious use case for a 7" tablet is as an electronic book reader. The N7 has some obvious disadvantages relative to the current crop of electronic-ink readers, though: it weighs about twice as much, has a fraction of the battery life, and has a backlit screen that is harder to stare at for hours. Still, it is worth considering for this role; its presence in the travel bag is more easily justified if it can displace another device.

The N7 hardware, in the end, puts in a credible, though not stellar, performance as a book reader. The extra weight is noticeable, but the tablet still weighs less than most books. The rated battery life for reading is about nine hours, possibly extendable by turning off the wireless interface. Nine hours will get one through an international travel experience of moderate length, but one misses the battery life of a proper reader device that can go for weeks at a time without a recharge. The lack of dedicated buttons for page-turning and the like (which are commonly present on dedicated readers) is not a huge problem. The backlit display can actually be advantageous in situations where turning on the lights is frowned upon — when the spouse is sleeping, or on some airplanes, for example.

On the software side, there are a number of reading applications available, ranging from the ultra-proprietary Google Books and Kindle applications to the (nice) GPL-licensed FBReader program. Experience shows that the rendering of text does not always work as well in applications like FBReader or Aldiko, though; white space used to separate sections within chapters can disappear, for example, and block quotes can be smashed into the surrounding paragraphs. Readers like Kindle do better in this regard. Another annoyance is that the tablet uses the MTP protocol over the USB connection, meaning that it does not work easily with Calibre. One can, of course, move book files manually or use Calibre's built-in web server to get books onto the device, but it would be a lot nicer if Calibre could just manage the on-device library directly.

In summary, while the experience for users of walled-garden book services is probably pretty good, it remains a bit rough for those wanting to take charge of the books that they so foolishly think they, by virtue of having paid for them, actually own. Beyond that, for content that goes beyond pure text — anything with pictures, for example — a tablet can provide a nicer experience. And, of course, the tablet offers the full Internet and all the other Android applications; whether that is considered to be an advantage in a book reader is almost certainly in the eye of the user.

In the long term, it seems clear that general-purpose tablets will displace dedicated reader devices, but the N7, arguably, is not quite there yet.

In general, though, the N7 works nicely as a media consumption device. It plays videos nicely and is a pleasant device for wandering around on the web. For people who are fully hooked into the Google machine it naturally provides a nicely integrated interface into all of the related services. For the rest of us the experience is a bit more uneven; your editor still yearns for a better email client, for example. But, even with its limitations, the N7 fills in nicely where one does not want to deal with a laptop, but where a phone screen is simply too limiting. This new tablet from Google is a nice device overall; it is likely to remain in active use for some time.

Comments (155 posted)