LWN.net Weekly Edition for August 13, 2015

GNOME and the future of hardware enablement

At GUADEC 2015 in Gothenburg, Sweden, Bastien Nocera related his recent personal experience getting Linux running on a cheap Windows tablet. Along the way, he challenged the GNOME project with a vision for how working with new hardware devices could go, a few GNOME releases down the line. The goal he described is one in which developing software for an arbitrary new hardware device is as simple with a GNOME desktop as it is for Android developers already.

The first half of Nocera's talk was presented as a case study of how he got Linux and GNOME to run on a generic hardware device. At the end, however, he backed up and retold the sequence of events as they had "really happened" (which was somewhat less successful). But the more optimistic version of events, he argued, was entirely within reach—if a few changes can be made to GNOME.

The hardware tale started when Nocera bought an inexpensive "iPad knock-off" tablet built by a Chinese manufacturer and decided to install GNOME in place of Windows 8, which came pre-installed. The biggest hurdle was getting the WiFi driver to work. The tablet used a Realtek chip and he was able to get a "code drop" driver from the company—but that driver was in remarkably poor shape.

It probably worked on Android, he said, but it did not with a mainline kernel. Digging into the source, he found that it was filled with useless code: tests, code targeting numerous pre-production hardware revisions, and so on. Since the WiFi module was connected to the motherboard over USB internally, the driver included all sorts of generic USB code that was unneeded for the tablet. Eventually, he and a few other hackers cut out all of the excess code—reducing the size of the driver by a factor of 20.

They then proceeded to figure out how to patch it to work on a current, standard kernel. That effort took time, but it worked in the end and he posted the driver on GitHub and began looking at what else the tablet required. He next got USB On The Go (USB OTG) running on the device's USB controller, and connected the tablet to his computer with a USB cable. That allowed him to use the Android Debug Bridge (adb) tool to catch system hangs early on in the boot process. He patched the adb daemon (adbd) to run on glibc and systemd (rather than Android's C library and init system) and posted that work on GitHub as well.

From there—he said—he set out to get EFI mixed mode working on the tablet's motherboard, so that he could boot a 64-bit kernel. After that, he replaced the tablet's bootloader with GRUB, set up Secure Boot shim, and ran the Fedora installer to install a complete, 64-bit Linux operating system. The adb tool allows developers to sideload packages onto the tablet, so, with a GNOME-supporting adbd on the device, developers could create xdg-app application bundles using GNOME Builder on their machine and upload them over the USB cable to the tablet and test them immediately. There would be no need to run emulators, simulate touchscreen and gesture events, or employ other such workarounds that are common today.

As Nocera quickly revealed, however, the last half of that story has not actually happened—at least, not yet. He has successfully gotten the WiFi driver running—with valuable help from Larry Finger. And the work on adbd and EFI mixed mode is real—although not all of the patches are ready for upstream.

The vision his story describes is what he hopes to see GNOME capable of in the near-term future, though, and that is what he most wanted to discuss. The pieces are falling into place: xdg-app is capable of producing "install anywhere" packages that, like Android .apk files, a developer could install and test with ease. Builder could make generating xdg-app packages part of its default feature set, and it could be made to integrate with adb.

There are other challenges as well, he said, some of which are not entirely in GNOME's hands. Windows 8 does not support USB OTG, for example, so even if the system-on-chip used in the tablet you buy supports USB OTG, the manufacturer may not have wired it up on the motherboard. One alternative would be USB host-to-host cables; they are cheap, but they will require some work to get running reliably on Linux desktops. Inside the connectors, most of them use cheap USB network adapters, and place a miniature Ethernet hub somewhere in between, so supporting them requires writing the appropriate device drivers.

A potentially bigger challenge is that EFI mixed mode support is not ready yet. In practice, at least, one can usually boot a 64-bit operating system on 32-bit EFI, but frequently things start to break when 64-bit drivers try to access EFI shortly after boot. Nocera also acknowledged that "cheap devices" change rather frequently, so keeping up with whatever is currently manufactured in China can be difficult. But in general, he said, if it can run a slightly older version of Windows, it can run a modern Linux distribution, too.

Nocera ended his session by declaring: "No refunds. I'm very sorry that I lied to you in the first part of the talk. We still need to do a lot of work." But, he added, developing support for the kind of plug-and-play development that he described is important for GNOME's outreach. Mobile devices are here to stay, and Android has a compelling "developer story" wherein anyone can hook up a USB cable and start writing software that they can test and run on a tablet or phone. GNOME could offer the same experience, if it sets out to do so.

[The author would like to thank the GNOME Foundation for travel assistance to attend GUADEC 2015.]

Comments (none posted)

Lessons from the GNOME-Groupon trademark battle

At GUADEC 2015 in Gothenburg, Sweden, Pamela Chestek delivered a keynote presentation that provided insight into one of the biggest stories of the past year for the GNOME project: the trademark battle between the GNOME Foundation and the web service Groupon. While the fight was concluded months ago—in GNOME's favor—she pointed out several lessons learned along the way that she thinks can be instructive for other free-software projects.

Chestek began by introducing herself and briefly describing her history with open-source software. She is a US-based attorney who worked for several years in various other corporate environments before entering the technology field. After some time working at proprietary software vendors she landed at Red Hat, at which point, she said, it took about a year to get truly familiar with the distinct differences inherent in open source.

The Groupon case was a major news event in the free-software arena, but its full history may not be familiar to all. Chestek's involvement began on May 22, 2014, when GNOME Foundation Executive Director Karen Sandler sent her an email noting that Groupon had filed for a trademark on "GNOME" for a recently announced point-of-sale tablet product—and asking if she "had any interest in helping the GNOME Foundation out." Looking into it, Chestek found that Groupon had actually filed 10 trademark applications for variations on the "GNOME" name. The product in question was really an iPad in a custom frame, designed to be a competitor to Square's point-of-sale consoles.

Since "GNOME" is a registered trademark in the technology business, Groupon's applications were clearly infringing, Chestek explained. On May 23, Chestek sent a politely worded email to Groupon's attorneys that said, in part, that she was "willing to assume that somehow your company was unaware of the GNOME Foundation's trademark" ... despite the years of continuous usage, millions of installs, multiple active social-media accounts, Wikipedia and press coverage, and so forth. That got a laugh from the audience but, she added, she has no doubt that Groupon was well aware of the GNOME Foundation's trademark when it filed its own—and probably saw that the Foundation was a nonprofit or that GNOME was a "niche" project and assumed it could get away with ignoring the matter.

Groupon's attorneys replied that they felt there was no chance that the two marks would be confusing to consumers, since its product was intended to be pronounced "Gee-nome." That claim meant nothing, Chestek said; the courts would clearly assume the dictionary pronunciation was the default for both marks—and, besides, many people do pronounce GNOME with an emphasis on the "G" due to its association with GNU. In the meantime, Groupon kept filing additional trademark applications. Chestek exchanged multiple letters and phone calls with Groupon's attorneys but, she pointed out, at no point did Groupon offer to drop its applications or change the name of its product.

By late July, the original ten applications had moved into the US Patent and Trademark Office (USPTO)'s "opposition" phase, which meant that they could be formally challenged in a legal proceeding. Sandler raised the possibility of going public with the conflict, which would let Groupon see that—despite what it might have assumed—GNOME was a large and popular product. Also, that would let the Foundation begin to raise money to mount the trademark oppositions: roughly $80,000 would be needed to challenge the original ten applications (although Groupon had continued to file for even more).

Matters came to a head that November. At 8:00p.m. on October 31, Chestek got a call from Groupon's lawyer (an outside attorney, as always, she added; no one inside the company ever responded), who told her that Groupon had filed for several more GNOME-related trademarks "without notifying us," bringing the total to 28. She got copies of the filings a few days later, and on November 9, told the GNOME board that she "was sorry to say that I think it's time to raise money" for a legal fight. She had previously warned Groupon's lawyer that "we haven't gone public yet, and you're not going to like what happens when we do"—which elicited a scoff. What followed, she said, was a testimony to the power of community, and the GNOME and free-software community deserves the credit.

Board member Sriram Ramkrishnan replied to Chestek's email about fundraising with a memorable "Well then: here we go..." and wrote a blog post [Wayback Machine link] explaining the situation. It went live on November 10, accompanied by a tweet; the board prepared to do follow-up work to try and raise awareness and gather donations. Instead, "people just went crazy"—recirculating the story on Twitter, Google+, news outlets, and discussion forums. They called Groupon to complain, many people blogged, and the Freenode IRC network even broadcast a system-wide message alerting users of the service and pointing them to the fundraiser on the GNOME site. Needless to say, none of the public was taking Groupon's side.

By sheer coincidence, Groupon had an investors' meeting on November 11 that highlighted its "GNOME" product, which is amusing in retrospect. But one of the most negative parts of the story also took place on November 11, when an engineer from the Groupon technical blog posted a story presenting Groupon's side, which claimed that Groupon had been "communicating with the GNOME Foundation for months" trying to find an amicable resolution, and that would be happy to find an alternative name. None of those statements were true at all, Chestek said, and worse yet, the claims damaged GNOME by making it look like the aggressor. The Groupon post got picked up by a number of general news outlets.

But less than 24 hours later, two things had changed. The GNOME Foundation fundraiser had already met its $80,000 goal, and the overwhelming public pressure forced Groupon to back down. The company's blog was updated to reflect this, announcing that it was dropping all 28 trademark applications. In terms of success, Chestek said, the fastest relief that they could have hoped for by challenging the marks in the USPTO opposition process would have been 21 months—and that would only have challenged the registrations; it would not have prevented Groupon from using the marks.

Lessons

Ultimately, the free-software community raised over $102,000 for the effort with donations from more than 5,600 people. "I don't know of any other similar effort orchestrated like that," she said. Who else, she asked, would respond in such a way? People only do that for things they are genuinely passionate about. They feel an attachment to a brand, but no one would come to Apple's defense in the way that they came to GNOME's defense. That was the first lesson from the experience.

Although the trademarks were dropped (the Groupon tablet eventually hit the market without a name at all, she noted—"although you always worry, so I've kept an eye on it"), the tale was not yet finished. Chestek continued to seek action from Groupon to rectify the damage it had caused to GNOME's reputation with the blatantly false blog post. That took a lot of iterations, she said, but the company eventually agreed to put out an update to the offending blog post acknowledging that it had, in fact, been notified about the trademark issue by GNOME and apologizing for the misleading statements.

Getting that apology published was indicative of the second major lesson, she said: commercial companies simply don't know what to do with free-software principles. Groupon had wanted to conclude the deal with a sealed agreement that no one would be allowed to discuss in public, since that is the normal process in big business. Chestek had to explain to them that putting a gag order on thousands of individuals worldwide simply wasn't possible—and that it went contrary to GNOME's principles of transparency.

The final lesson was that GNOME is a powerful brand, even within the free-software movement, she said. "I'm not sure how many other projects could engender that kind of passionate response," she said, but it is few, and the project deserves credit for having built something so many people were motivated to come to the defense of.

Chestek took questions from the audience at the end. One audience member asked if any legal action should be brought about the misleading Groupon blog post. She said no, noting that it was in all likelihood someone inside the company who had imperfect, filtered information about the case, and who felt called to come to the company's defense.

Someone else asked why the GNOME Foundation board remained silent about the details of the case for so long. Chestek said that the board agreed to "not go out on the speaking circuit" about the issue because it had no desire to cause long-term business harm to Groupon, who had eventually settled the issue. The board agreed to post an update on the GNOME web site and point people to it, but to otherwise "try to let the issue fizzle out" on its own. It is a delicate balance to strike, she said.

Most of the questions, though, were about what other software projects can do to protect themselves from similar trademark infringement. By far the most important thing, Chestek said, is to register trademarks used by your project. "Part of my mission is to encourage projects to register as many trademarks as they can in as many jurisdictions as they can," she said. Too many developers fail to do so, thinking they will take care of it later, or underestimating the time that it will require. For that reason, she also encouraged developers to set up a formal relationship with a larger project (like GNOME), which can help with legal issues, or to consider joining an umbrella organization like the Software Freedom Conservancy.

But trademark registration can be complicated and, in some cases, expensive. The US is a particularly good jurisdiction for trademarks, she said. The EU is also vital for most free-software projects, but beyond that it can be hard to say how many jurisdictions one should register in. Furthermore, the sad reality is that trademark disagreements, like most areas of the law, often come down to who has the time and money to put up a fight—if indeed it comes to a fight. Unfortunately, not even the GNOME Foundation's successful challenge to Groupon can really do much (such as providing any legal precedents) to prevent similar attacks in the future. But it does illustrate that, despite what anyone might say, a trademark in a free-software project is just as strong as a trademark in a commercial company.

At the end of the session, longtime GNOME veteran Federico Mena-Quintero presented Chestek with a gift on behalf of the project: a handmade wooden plaque inlaid with the GNOME logo. That presentation was, of course, accompanied by a long round of applause from the attendees also expressing their individual gratitude to Chestek.

[The author would like to thank the GNOME Foundation for travel assistance to attend GUADEC 2015.]

Comments (1 posted)

Git + FUSE + Python = GitFS

Typically, filesystems are developed in C, often as part of a kernel like Linux. But the folks at Presslabs took an unconventional approach by developing a filesystem based on Git as a user-space filesystem written in Python. Vlad Temian and Manu Danci, who both work for the company, came to EuroPython 2015 in Bilbao, Spain to talk about that filesystem, which is called GitFS.

Presslabs is a Romanian WordPress hosting site with as many as 2.2 billion monthly page views. One of the problems its customers have encountered, though, is conflicts between site publishers (i.e. site owners) and their developers. Publishers will try to change the code that developers have written, but many don't have the knowledge required, so the site becomes broken. Nobody knows who did what to the code base, which results in a "big pile of chaos", Danci said.

So Presslabs came up with the idea of a self-versioning filesystem that uses Git behind the curtains. But in order to be user-friendly, GitFS turns the Git version tree into something more accessible using the timestamp of each commit. The mount point has two directories: current, which contains the latest snapshot, and history, which contains directories of other snapshots based on the date and time of the commit:

    mnt/history/
	2015-08-09/
	    14-32-12-eb18db93bc9/
	    14-37-39-e256395058f/
	    16-03-23-25b39c058fe/
	2015-08-10/
	    11-17-48-ef1fdf9fbff/
	...

Users can read and write to files in the current directory, but the history directories are all read-only. The idea is to "humanize" the contents of the .git directory.

Mounting the filesystems is done with the gitfs command, as Temian and Danci demonstrated live. The command requires the remote repository and a mount point; there are also options that can be specified for timeouts and other parameters. When someone makes a change to the repository (e.g. a developer commits changes), GitFS updates the current directory and adds a new entry into history in the mounted filesystem. Similarly, when someone makes a change in current, a commit is generated after a specified timeout occurs (to try to package up multiple changes into a single commit).

The filesystem is written entirely in Python and has been released under the Apache 2.0 license. There is more that could be done with GitFS, Danci said. He suggested that anyone interested contribute to the project so that "maybe we can grow it further".

As part of the research that eventually led to GitFS, the team came up with some requirements. There were two main hurdles to overcome. The first was how to handle Git objects efficiently, both in terms of speed and memory usage. The other was to be able to implement the filesystem operations in an efficient manner.

Python-based solutions were found for parts of the problem. GitFS uses pygit2 for its Git bindings. Pygit2 is written in C, so it uses Git's C API and doesn't waste time using the shell to access Git commands. GitFS also uses fusepy for access to the Filesystem in Userspace (FUSE) API.

Temian then took over to describe some of the architecture for GitFS. The main concept in the filesystem is that of a "view". There are several types of view, depending on what kind of object is being accessed. For example, an open() call goes to a router that chooses between the view types: CurrentView, HistoryView, CommitView, or IndexView. There are also some higher-level view classes, including PassthroughView (used for current) and ReadOnlyView (which is the parent of the history views).

Eventually there will be conflicts that need to be resolved between different versions of the files. GitFS has a adopted a simple strategy called "always accept mine" that is safest for Presslabs's customers. It effectively takes the local changes and applies them on top of any changes to the remote repository. That strategy is not implemented in pygit2, so it was added as a plugin. Others can write their own strategy to be used with the filesystem.

One of the first problems encountered with GitFS was its performance on real-world repositories. As a test, the WordPress repository with some 17,000 commits was used, but it took 34 minutes to do a directory listing on history. Some profiling led to the addition of three levels of caching. After that, the directory listing completed in around three seconds.

If you unzip 1000 files, you don't want to do 1000 commits for them, Temian said, so a smarter mechanism for upstream synchronization is needed. That led to some additional components in GitFS: FUSE threads, a commit queue, sync worker, and fetch worker. The FUSE threads are not directly under the control of GitFS — FUSE determines how many there are, for example. Those threads add commit jobs to a commit queue when they have completed writing a file.

That commit queue is consumed by the "sync worker" process, which batches up the changes based on the sync timeout that has been specified for the filesystem. When it is making a commit, it locks out any other writes to the filesystem by notifying the FUSE threads, waits for completion of the merge and push, then unlocks the filesystem. There is also the "fetch worker" that periodically (also dependent on a timeout) checks for changes on the remote repository. It is also told to pause during sync operations.

Danci then took back the microphone for some concluding remarks. First off, there is an Ubuntu personal package archive for GitFS that is maintained by Presslabs, as well as community-maintained versions for Fedora and Arch Linux. There is also a version available for Mac OS X.

There are a number of things that attendees should take away from the talk, Danci said. You can build a production filesystem in Python. Presslabs did it and has been using it in production for almost a year now. People suggested doing it in C, but the team preferred Python and it is working well.

Building a FUSE-based filesystem is straightforward, he said, but getting it right can be challenging. The current model was not the first the team came up with. It took a number of refinements to get to where it is today. His final point was that by combining appropriate technology, you can "put powerful tools in the hands of non-technical users". The main purpose of GitFS is to make people's lives easier by putting Git in the hands of regular users.

Comments (3 posted)

No LWN Weekly Edition next week

The week of August 17 is just a little bit busy. The schedule includes LinuxCon, the Linux Plumbers Conference, DebConf, the Linux Security Summit, and more. For added fun, two LWN staff members are (lightly) involved in the planning for Plumbers, and we will have staff out of the office for non-conference reasons. All of that will make it rather more difficult than usual to get a weekly edition out the door.

So, we're not going to do it. There will be a steady stream of content over the week, and an extra-thick edition the week following, but the August 20 edition will not be published. We apologize for the slowdown, but we're looking forward to meeting many of you at conferences worldwide next week. Thanks to all of you, as always, for supporting LWN.

Comments (3 posted)