Leading items

GNOME Maps and the tile problem

The GNOME project's Maps application provides access to an array of mapping features (trip routing, address lookup, zoomable maps, etc.) from the desktop. Implementing that feature set requires hooking into a number of online services, but none of them is as prominent as the map tiles—the background images on top of which everything else is added in overlays. Recently, the tile provider that had served GNOME Maps well for several years ended its free service, suddenly cutting off all of GNOME Maps's users and forcing developers to consider new approaches for the future.

Since its initial public release, GNOME Maps had used the open tile API offered by commercial map vendor MapQuest to fetch its map tiles. Like a lot of online mapping providers, MapQuest incorporates OpenStreetMap (OSM) data into its own product offerings, a usage fully allowed under OSM's licensing (and surely a significant cost-saver for the company). In order to give back, MapQuest announced its OSM-based service in 2010.

GNOME Maps, in turn, made use of the MapQuest open tile data because it is end-user-ready—unlike the raw data provided by OSM itself. Although OSM does have a publicly accessible tile server, in practice that server acts as more of a technology demonstration or development aid than it does a full-blown public web service, and access to the tile server is limited. OSM's primary focus remains on creating and editing the map data itself.

To put it another way, ultimately OSM produces a data set; it does not run a public service. And the limitation is a technical one, too, not mere policy. Any end-user service built on top of the OSM database must, at the very least, perform its own filtering and other transformations (selecting which road and path types are of interest, dropping authorship and update information, perhaps selectively choosing land-usage type, points-of-interest and other metadata, and so on). On top of that, its must render the map data into images and apply appropriate labels and markers, plus undertake all of the tasks of running a large-scale public web service.

For years, MapQuest's free OSM-based service did just such processing, supplying ready-to-use PNG image tiles. But, in June, MapQuest abruptly changed its usage policy, making the existing free service unusable and steering existing users toward a new, paid service. The new MapQuest service did advertise a free tier, but with an API cap far too low to work for GNOME Maps.

The official shutdown occurred on July 11, causing GNOME Maps (and many other services and applications) to display a repeating display of MapQuest's shutdown announcement in place of map tiles. On July 13, Maps co-maintainer Jonas Danielsson wrote to the development list, calling attention to the service shutdown and warning that the project has "no clear alternative" for a map-tile source.

He noted that there were patches available that could switch Maps over to OSM as a direct tile source, but added that "we would probably violate the terms of service" unless GNOME could convince OSM to grant it an exception. He asked for help contacting OSM, but concluded: "I think we are going to need our own tiles.gnome.org for a map application/platform to be feasible."

GNOME could perhaps run a server that fetched and cached OSM tiles for Maps users without raising the OSM project's ire, although it is not clear whether or not such a caching server would run afoul of OSM's rate-limiting rules and require asking for special permission. And, certainly, a caching server could act as a temporary solution, something that could be swapped out in the future for a GNOME server that generated its own image tiles directly. But, as Emmanuele Bassi noted in a Reddit discussion thread, the computing power for the server may be doable, but the bandwidth could rapidly become a problem.

PNG image tiles are typically 256 pixels square but, for any given area on the map, they must be generated at multiple resolutions—OSM supports 20 zoom levels, and users like to zoom in and out at will, expecting the image tiles to be served up nearly instantaneously. Multiplied by all of the GNOME Maps users, that adds up to a lot of network traffic.

One hypothetical alternative would be to render the tiles locally. The map server could send vector data for the region of interest and have the user's CPU or GPU render the data directly to screen. Danielsson noted that such client-side rendering is an item on the Maps roadmap, but it is several steps from being practical. OSM's vector-tile API is still in the experimental stage (as there are additional challenges to solve, such as clipping paths to bounding boxes), for one thing.

But users may not like vector-tile rendering either. Local rendering would cost battery power, and it could introduce lag. The Android app OsmAnd already uses client-side rendering, and OsmAnd users will no doubt be familiar with the delay experienced after panning or scrolling to a new area on the map, waiting for the roads and buildings to render on screen.

Another approach suggested on the list was having the application fetch map tiles in advance and store them locally. GNOME Maps can already run in local-tile mode, if one downloads tiles for the area of interest ahead of time. But this was also regarded as suboptimal; it requires the user to dedicate considerable disk space to store map tiles even in the best of circumstances, and the space required to store the entire OSM global map is hundreds of gigabytes.

In the long run, GNOME may have to run its own tile server. There would certainly be advantages to having control over that bit of critical project infrastructure. In the interim, however, the project seems to have come across a viable substitute. Przemo Firszt suggested reaching out to any of the several OSM-based service providers geared toward developers. One of those service providers, Mapbox, responded a few days later with an offer to give GNOME Maps free access to its tile server for a year.

Danielsson and Mattias Bengtsson then discussed the offer with Mapbox and agreed to the switch, with Bengtsson adding: "Personally I'm interested in exploring how we could continue with Mapbox in even in the future. It might mean crowdfunding in some way."

The MapQuest bug is listed as a blocker for the release of GNOME 3.22, so implementing the fix via Mapbox is a high priority. But it could still be a while before the patched application reaches end users; Ubuntu was already forced to pull Maps from its installation images for the most recent stable update to Ubuntu 16.04 (although the package could easily be restored in the next update). The Maps team has tentatively planned to discuss a long-term solution a few weeks from now at GUADEC.

The series of events is frustrating for Maps users, of course, but it is worth remembering that the root of the trouble is MapQuest's sudden reversal of its original pledge to support the open-source community. Reliance on the external service led to the disruption, which would be the case regardless of whether that service was free or proprietary.

But, at present, reliance on third-party resources for map applications seems to be unavoidable due to the size and complexity of global mapping. GNOME Maps already relies on quite a few other services to provide functionality on top of the tile layer, including Nominatim, which converts addresses to map coordinates, GraphHopper, which calculates routes, and so on. That makes it susceptible to interference on a number of fronts—even if outages like the current one remain a rare occurrence.

Comments (32 posted)

Apache OpenOffice and CVE-2016-1513

The Apache OpenOffice (AOO) project has suffered from a lack of developers for some time now; releases are infrequent and development of new features is relatively slow. But a recent security advisory for CVE-2016-1513 is rather eye-opening in that it further shows that the project is in rough shape. Announcing a potential code execution vulnerability without quickly providing a new release of AOO may be putting users of the tool at more risk than they realize.

The bug is a memory corruption problem that can occur when opening crafted .odp or .otp presentation files in the Impress presentation editor. The advisory says that "the conditions under which arbitrary code can be executed are complex and difficult to achieve in an undetected manner"; it has been given a "Medium" severity level by the project. The patch shows that a condition enforced in debug mode needs to also be enforced at runtime. The fix is simple and straightforward, which should seemingly make it relatively easy to get an updated release into the hands of users.

There is an anti-virus signature available to detect these kinds of crafted files and there are no known exploits in the wild—at least there were no exploits known when the advisory was published on July 21. Since that time, it is not hard to imagine that some folks with a malicious bent might well have worked something up. And, while creating a reliable exploit may take some work, it certainly isn't all that hard to get users to open slide decks from email or elsewhere—a spear phishing attack where the targets are known may even make that easier.

The advisory notes that keeping anti-virus software up to date is one defense, as is running AOO without administrative privileges. Another workaround for documents of unknown provenance includes a recommendation to use competing office suites:

Just after the advisory was published, Dennis E. Hamilton posted a note to the AOO development mailing list announcing the vulnerability and an effort to test some "hot fix" binaries that would fix the problem. Hamilton is the chair of the AOO project management committee (PMC) and he was looking for more users to help test the fix:

A few days later, PMC member Andrea Pescetti posted a follow-up with an outline of the steps to take to address the vulnerability. "I assume we will want to keep the effort minimal", he said. Notably absent from his outline is making a full binary release of AOO with the updated code in the near term, or really any provisions for regular users, though he did expect to get to that eventually:

For now, he suggested simply applying the patch to the 4.1 branch and making a 4.1.2.1 source release, as well as providing "repaired libraries for power users". The new version of the libraries would still be marked as version 4.1.2. That would not give users or administrators a way to tell whether an AOO installation was vulnerable, however, which concerned Don Lewis: "Not being able to tell at a glance whether a copy of AOO has been fixed or not is bad."

Lewis suggested making the hash of the library available, so that the different versions could be distinguished. That would not work for anyone who built the library from source, though. He also suggested adding a static string (e.g. "CVE-2016-1513 Fixed") to the library so that tools like strings could be used to make the distinction between the versions.

Unfortunately, changing the library would invalidate some the testing that had already been done with the binary hot fix, as Hamilton noted. He suggested that a combination of file size and creation date (coupled with the hash of the new library) would be enough to indicate updated versions. Lewis agreed with that plan.

So far, though, the new binaries have not been released, so users are left either building their own or using other tools to avoid the flaw. One of the more surprising things, perhaps, is that there is no established procedure to quickly produce an updated version in the presence of a security vulnerability. This not the first time that the project has struggled to get out a release for a security vulnerability, so it would seem that being a bit better prepared would be in order.

That is especially true when considering that a recent report shows 600,000-850,000 AOO downloads per week. That's an awful lot of potentially vulnerable software being installed weekly, which makes prompt security updates all that much more important.

Even if this particular vulnerability is hard or impossible to exploit, that may well not be true for the (inevitable) next vulnerability. If a project cannot keep up with its vulnerabilities—especially for a program that often deals with untrusted, complex input like document files—it is likely doing its users an extreme disservice by trying to keep up appearances. It seems clear that AOO needs more developers, builders, testers, and so on, so that it can get out security updates (at least) in a timely fashion. In the meantime, though, users should probably strongly consider moving on from the project.

Comments (7 posted)

Downsizing at Cyanogen Inc.

Steve "Cyanogen" Kondik's CyanogenMod has been one of the leading (if not the leading) Android derivatives for the past several years. Started in 2009, the effort was initially a volunteer open-source project that only produced a free, aftermarket version of Google's Android releases. In 2013, the project's leaders branched out and formed Cyanogen Inc., to develop a commercial offering as well. Recently, however, reports have circulated that the company is in financial trouble and may have laid off its operating system (OS) team to refocus its efforts on writing Android apps. Kondik, however, disputes those reports.

The original report of layoffs at Cyanogen Inc. came from the Android Police blog, which on July 22 cited "multiple sources" that the company was letting go "a significant portion of its workforce around the world". The Android Police story specifically claimed that the company was shutting down the team that worked on the open-source components of Cyanogen OS (the company's commercial OS offering) and that, moving forward, Cyanogen Inc. would shift its focus to app development.

Recode reported much the same thing, adding that the "pivot" to app development was being spearheaded by the company's new Chief Operating Officer Lior Tal. On July 25, however, Cyanogen Inc.'s CEO Kirt McMaster explicitly denied the app-pivoting rumor on Twitter, saying "we are an OS company and our mission of creating an OPEN ANDROID stands."

Finally, Kondik himself posted an update at the CyanogenMod blog on July 25. He summarized the situation as: "CyanogenMod isn’t going anywhere, nor has Cyanogen Inc. discontinued it's efforts towards the goal of bringing it to a larger audience." He went on, however, to say:

And, later:

Such wording does not refute the reports that the layoff affects primarily the OS developers, of course. All told, those layoffs appeared to account for 30 people out of a staff of 136, or around 20% of the employees.

But, even if most of those laid-off employees came from the OS division, it is difficult to imagine how Cyanogen Inc. could compete as an app vendor in the already crowded Android-app field. Volunteers in the CyanogenMod project have contributed to app development, largely to provide substitutes for the default apps that Google supplies in its own Android releases—apps that are increasingly proprietary and tied into Google's remote services.

But the distinction between CyanogenMod and Cyanogen OS has not been primarily based on app selection. The version numbers are kept in sync, reflecting the fact that the base systems are fundamentally identical. The apps that Cyanogen OS did provide on top of CyanogenMod were even released as a free download for CyanogenMod devices.

In fact, recently the main differences between the two have hinged on partnership arrangements made with other companies. For example, Cyanogen OS replaced the phone-dialer app with an alternative developed by Truecaller. The company also partnered with Microsoft to replace the Google apps found in stock Android. The replacements, of course, were proprietary apps tied to Microsoft services.

In February, Cyanogen Inc. launched an initiative—rather confusingly named "Cyanogen MOD"—that provided an alternative app store focused on system components like on-screen keyboards and dialers. It has yet to make a significant impact, and it remains possible that the layoffs and the "what doesn't work" alluded to by Kondik relate more to the MOD initiative than to CyanogenMod's open-source underpinnings.

Most directly, the layoffs likely reflect the company's trouble selling Cyanogen OS, which is still in search of a major distribution deal. That said, several smaller OEMs have shipped Cyanogen OS on devices. The OnePlus One (released in 2014) was perhaps the most high-profile device to include Cyanogen OS, but OnePlus and Cyanogen Inc. ended their partnership in mid-2015.

And, ultimately, pre-installation deals are what will make or break Cyanogen OS. Unlike CyanogenMod, the Cyanogen OS releases are not made available for download to the general public. Phones with Cyanogen OS pre-installed are also shipped without root access, although gaining root on such devices is little different from rooting other smartphones. In the end, then, a Cyanogen OS phone can always be upgraded to run CyanogenMod instead. Given that fact, convincing the user that the pre-installed version remains the better option could be a challenge.

Business Insider reports that Cyanogen Inc. has roughly three years' worth of funding available, which can be quite a long time in "software years." Until the company provides a clearer explanation of its plans, however, it is anybody's guess in what direction it can be expected to move next. At the very least, whatever the company does, CyanogenMod as an open-source project appears to be safer than Cyanogen OS, since the source code remains available and there are still community developers interested in keeping the project moving.

Comments (4 posted)