User: Password:
Subscribe / Log in / New account Weekly Edition for July 24, 2014

Handling ARM architecture changes

By Jonathan Corbet
July 23, 2014
Current Linux systems running on x86 processors will run binary programs created in the 1990s. A strong focus on not breaking the user-space ABI deserves much of the credit for this state of affairs, but there is another factor at play: the x86 hardware architecture has also been managed with an eye toward keeping old executables working. The ARM architecture is different, though; it evolves in ways that will prevent some older applications from running properly (or at all). That presents a challenge for kernel developers, who do not want to break applications but who also want to keep the kernel maintainable in the long term.

Recently, Colin Cross (of Google) raised an issue for consideration. The ARM architecture is occasionally put through a major revision, the latest of which is ARMv8. This version adds 64-bit support, among other things; it also provides compatibility support for applications built for older versions of the architecture. But that compatibility only goes so far; it lacks, in particular, support for the SWP instruction (which performed an atomic read-modify-write operation), the SETEND instruction (used for changing the endianness of data accesses), and certain types of barriers. This lack of support is not surprising; these features were all deprecated in v6 or v7 of the ARM architecture. ARM is following a well-established schedule in its removal of these instructions.

The problem, of course, is that there are applications out there that are still using those instructions. Any application that is built today should be free of deprecated instructions, but there are plenty of applications sitting in the dustier corners of the Google Play Store that have not been built recently, and which, perhaps, will never be rebuilt again. As things stand, if a user loads such an application onto an ARMv8 device, it will fail to run. That runs counter to the plan that ARMv8-based Android devices are expected to be compatible with existing applications.

The solution to the problem seems straightforward enough: the kernel can trap attempts to use the deprecated instructions and emulate them. Applications using those instructions will then continue to run, though they will run more slowly than they otherwise would. Colin asked if the kernel community would be willing to accept a set of patches implementing this emulation. The alternative, he said, was for Google to maintain the patches in a separate tree that all Android vendors would use; in this case, the mainline kernel would fail the Android compatibility test suite.

Given the importance of Android in the ARM world, one would think that there would be little opposition to keeping the kernel compatible in this way. But ARM developer Will Deacon expressed his opposition to the idea, complaining that, once emulation for instructions like SWP goes in, it will have to be maintained for a long time. He also argued that use of SWP, in particular, almost certainly indicates a bug in the application. It would be better, he said, to keep this code out of the kernel and to simply fix the affected applications.

Catalin Marinas added that anybody moving to an ARMv8 device is going to have to download all their applications from the store anyway. In his mind, Google should push to get older applications rebuilt so that, when users download them on their shiny new ARMv8 devices, they will simply work. But Grant Likely objected to this reasoning, saying that forcing developers to rebuild applications makes the platform as a whole more hostile. Also, he said, many of those older applications may no longer have a functioning company or developer behind them; getting them rebuilt may simply not be possible. But they work well enough now, and should not be broken.

After some back-and-forth, the conversation seemed to reach a point where most developers agreed that breaking existing applications is not a good thing. How that will translate into patches going into the mainline kernel is not yet entirely clear, though. The developers also reiterated the usual rule about backward compatibility: it is, in fact, perfectly fine to remove support for deprecated features as long as nobody is actually using that support. Given that ARM is likely to continue deprecating hardware features in the future, kernel developers are going to have to find a way to minimize support for deprecated features going forward. Nobody wants to turn the kernel into an emulator for all older versions of the ARM architecture.

To that end, Catalin put forward a proposed timeline for how to deal with deprecated features in general. The first step happens as soon as a feature has been deprecated in the hardware, but while current hardware still supports it. That is when the kernel community needs to find ways of calling attention to the deprecated feature and encouraging developers to move away from it. In particular, the right way to replace the deprecated feature should be worked out and publicized.

Step two comes about in the next hardware version, where, by ARM's normal practice, the deprecated feature is still available but can be explicitly disabled. At this point, the kernel will disable such features and emulate them in software; warnings will be emitted when the deprecated features are used by applications. Everything will continue to work (though maybe more slowly). There is one challenge here, in that it can be hard to get anybody to see kernel warnings on a mobile device; users tend not to read the log files often, if, indeed, they have access to the logs at all. For this type of device, warnings may have to be implemented at other levels; in the Play Store, for example.

In the third phase, another hardware revision has removed the feature entirely. At this point, Catalin says, the kernel should stop emulating the instruction and, instead, deliver an illegal-instruction signal (SIGILL) to the offending process. The emulation code might still be in the kernel, he said, but it should not be enabled by default. Finally, in the fourth stage, the emulation support would be removed altogether.

There appears to be widespread agreement in the ARM kernel community around this set of rules for dealing with hardware deprecations. There may, however, still be some dissent over the issues at hand today. It can be argued that the SWP instruction, deprecated in ARMv6, should not yet be at stage three because the effort to move users away from it was not made at an appropriate level. So, while some developers would still like to see applications rewarded with SIGILL for using SWP on ARMv8, it may be necessary to carry emulation for that instruction for a while yet.

For ARM hardware features that are deprecated in the future, though, the above timeline seems likely to be followed. As long as the word gets out to developers in time, there should, with luck, not be users of the deprecated features remaining by the time the support is removed entirely. That should help to keep the kernel maintainable as the hardware architecture evolves. But it will be disappointing to those who wish they could carry forward applications from their old G1 phone and have them run properly on future devices.

Comments (34 posted)

Numbering the next major PHP release

By Nathan Willis
July 23, 2014

The PHP community faces a difficult decision: what to call the upcoming major release of the language. Such questions are rarely the source of controversy, but the new version will be PHP's first major update in quite a few years, and the project finds itself in the awkward position of having to work around potential confusion between the most obvious option—PHP 6—and an abandoned project from several years earlier.

PHP 5 was released in 2004. Like many languages, it took some time to supplant existing PHP 4 installations in the field, but it has remained the focus of continued development. Shortly after its release, however, the project began looking at what problems could be solved by a successor.

In 2005, Andrei Zmievski started an effort to create a PHP 6. There were several high-profile items on the to-do list, including 64-bit integers and basic static typing. But Zmievski's largest undertaking was a rewrite to fix one specific shortcoming: the lack of native Unicode support. Zmievski's fork incorporated the ICU library, but it also involved changing the internal representation of text, from PHP's original "series of bytes" format to proper UTF-16. The reason is understandable enough; in PHP 5, a string is a series of arbitrary bytes. The application (e.g., a code editor or PHP interpreter) has to interpret those bytes as belonging to some particular character encodings, out of a wide variety of possibilities. Guessing wrong produces mojibake, or gibberish. Settling on one canonical character encoding like UTF-16 should remove all ambiguity.

But this was a change that touched nearly all corners of PHP: string variables and functions, HTTP handling, data structures, and even identifiers like variable, function, and package names. As befitted such a deep reworking of the system's internals, Zmievski referred to it during the development process as PHP 6. Unfortunately, though, it never saw the light of day.

In a 2011 talk examining what went wrong, Zmievski noted that UTF-16 itself was a poor choice, since UTF-8 was backward-compatible with ASCII, more widely-used than UTF-16 elsewhere, and avoided the pitfalls of worrying about endianness that plagued UTF-16. Perhaps because of the complexity of working with UTF-16, coupled with a paucity of code contributors, development of crucial parts of PHP 6 (such as the database extensions) began to lag, he said. Ultimately, the project abandoned the rewrite in March of 2010, and the PHP 5.4 branch became the trunk.

Fast forward to 2014, though, and work on a separate PHP core rewrite called PHPNG has progressed to the point where it is nearly ready for release. Naturally, this raises the question of whether PHPNG should become PHP 6, or if it should be called something else—the most obvious option being PHP 7. The PHP community has discussed this topic on several occasions in the past, but as PHPNG gets close to releasable status, the subject obviously takes on additional urgency.

On July 5, Andrea Faulds posted an RFC to the PHP development list asking that a vote be taken "to settle the matter once and for all." Faulds proposal was initially a simple "yes or no" vote on the name PHP 6. Zeev Suraski replied that this proposal was too simple, and that the options should at least pit PHP 6 against PHP 7 and the option to defer the decision until later.

In the subsequent list debate, considerable attention was devoted to crafting the wording of the proposition being voted on—in particular, to making sure that both the PHP 6 and PHP 7 options were fairly represented by text that supports the position.

The main objection to the name PHP 6 is its association with the abandoned Unicode refactoring effort of 2005. Not only was that project referred to as PHP 6 at the time, but there is published material that discusses it by that name—including web articles, archived conference talks, and a multitude of printed books. It might be clear by examining the dates of those books that they do not discuss a PHP 6.0 released in 2014 or (more likely) 2015, but there is still potential for confusion, since the PHPNG codebase is an entirely separate affair. Books, unlike web pages, cannot be silently updated, and for years numerous copies purporting to instruct the reader on PHP 6 have been occupying the shelves in bookstores and going home with shoppers. In addition, the failure of the 2005 effort has resulted in a lot of material criticizing PHP 6, some of which could be misunderstood as criticism of the new release.

On the other hand, the case for calling the next major release PHP 6 focuses on continuity with the established version-numbering scheme. Skipping a number is not addressed by the project's official release process, and it would likely confuse users to jump from 5 to 7 . As for the existing references to the 2005 project as PHP 6, eventually, the argument goes, new articles and books will be written about the new release and will supplant the old material. In addition, there are already colloquial references to PHPNG as PHP 6 within the community, so there is potential user confusion to be found in either path. Furthermore, while the 2005 effort was referred to as PHP 6 in its development branch, that designation was never used as a Git tag, which would complicate its recycling as a practical matter.

There are certainly examples of software projects skipping version numbers before, although none seem to set a precedent that other projects universally accept. The proposition on the wiki noted (under the "in favor of 6" section) that most such number skips are accompanied by a change in the overall numbering scheme, as when Java jumped from version 1.4 to Java 5.

Suraski argued that calling the next release version 7 does not break the numbering scheme anyway, because "PHP 6 was a live project that was worked on by many people, and known as such by many many more; Even though it never reached GA – there was definitely software named PHP 6." But, Brian Moon ominously observed, "it occurred to me that naming things before they are released is how we ended up in this situation to begin with."

There was intermittent consideration of other naming options (Faulds mentioned PHP 6.1 as a possibility; Suraski noted that 14, in reference to the year, had been suggested), but the debate soon turned into a straight 6-versus-7 contest. Although several voices expressed dissatisfaction with how much the version-numbering debate sapped time away from actually working on PHP, eventually both the 6 and 7 camps settled on the wording of their respective arguments in the RFC, with Suraski contributing the case in favor of PHP 7 and Faulds the case for PHP 6.

After a brief false start, voting on the RFC started on July 23, and is scheduled to remain open until July 30. As it stands now, PHP 7 is in the lead by 35 votes to 15. A lot can still happen in the time remaining but, with any luck, the PHP project will soon have settled on its next major version number by democratic vote of the project's active developers—at which point all of the development community's attention can be re-focused on getting that next major version out the door.

Comments (18 posted)

Why the Vivaldi tablet never came to market

July 23, 2014

This article was contributed by Bruce Byfield

In January 2012, Aaron Seigo announced the Spark tablet. Planned as free hardware with KDE's Plasma Active as its desktop environment, the tablet was quickly renamed Vivaldi. Delay after delay followed, with few details, until July 1 of this year, when the project was officially canceled. Recently, Seigo gave his perspective about what happened, and the lessons to be learned from the experience.

Vivaldi and its related projects came directly from the first release of Plasma Active, the KDE desktop tailored to mobile devices, in September 2011. "Everyone agreed that if it wasn't on a tablet or other mobile device, what we were doing was just an academic exercise," Seigo said. "So we took Plasma Active around to various vendors, [and] we got very positive feedback, but nobody was really willing to be the first to release a device with Plasma Active."

In response, Seigo established a cooperative company, MakePlayLive, to produce and distribute the tablet and other free software and hardware products. As might be expected, many of those involved with MakePlayLive were also involved in KDE. In addition, MakePlayLive used to store its code, and KDE sites sometimes promoted MakePlayLive.

However, Seigo emphasized that "there was no official endorsement from KDE." In fact, one reason for establishing MakePlayLive was to create some distance between KDE and the effort to build a tablet. Consequently, the failure to produce Vivaldi has no effect on Plasma Active or any other part of KDE. "Having this hardware would have been a massive strategic and practical boost [to Plasma Active], but without it, Plasma Active is right where it always was," Seigo said.

MakePlayLive's efforts began with several months of researching and testing off-the-shelf devices from Asian companies. Many manufacturers were simply "out of our league," Seigo said, and they refused to take anything less than six-figure minimum orders. MakePlayLive had just settled on a possible device when its efforts were made obsolete with the manufacturer's release of a revised device. To make matters worse, the manufacturer could not say when the next revision would happen, or what its specs might be. Nor would the manufacturer agree to release any of the kernel code under a free license. "And this was one of the better companies we could find," Seigo commented.

Finding another manufacturer was difficult, especially because many of those that claimed Linux support proved, on investigation, to mean that they were planning to add it. However, by December 2013, MakePlayLive had settled on QiMOD, a UK-based company working on open hardware.

What happened then depends on who you listen to. According to QiMOD's public statements, MakePlayLive failed to communicate and kept changing the specifications. However, Seigo claimed, the demands for revised specifications came from QiMOD. Even worse, he said that the parts received were not functional without considerable modification.

QiMOD decided to redo the printed circuit board from scratch, but the delays meant Vivaldi would be released with year-old specifications, making it non-competitive. Moreover, after eighteen months, Seigo was "tapped out" financially. In desperation, MakePlayLive announced Improv, a commercial grade engineering board that would give MakePlayLive something to show for its efforts and help to finance Vivaldi while providing a technical base. The idea was that Improv would help open hardware developers to build prototypes quicker and bring products to market sooner. MakePlayLive got as far as taking pre-orders, but received too few to place an order with a manufacturer.

"The interest for Improv just didn't materialize," Seigo said. "Some people couldn't see past Improv as a 'Raspberry Pi++,' which it really wasn't, and some people thought it was entirely irrelevant. One KDE developer said that they just didn't feel any need to compete against Android like they did for MS Windows or MacOS."

The twin failures of Vivaldi and Improv left MakePlayLive in the position of having to offer refunds on pre-orders and paying for orders of components for which it no longer had a use. In all, the immediate debt "came to a little under $6K. A number of people who purchased an Improv actually told us to keep the money; some even sent more to help defray costs of winding down."

However, Seigo personally claimed to have invested $200,000 over the last two and a half years. This sum consisted of travel expenses, sample purchasing and test runs, support for free software projects, and "some salaries." He added that "my income during that period was pretty insignificant. I was taking a tiny wage at this point, just enough to pay some bills. Thankfully, my wife was working. Everything else went to other people working on [MakePlayLive] and material costs." This information may serve as a partial answer to complaints about transparency, emphasizing exactly how under-funded MakePlayLive actually was.

Lessons learned

Seigo takes a stoic perspective on the failure of MakePlayLive:

Certainly there was a certain amount of egg on my face. The failure was public. I got raked over the coals within KDE more than once for setting up a company that would dare to put money into these things. But that's not something I can't live with. I swung and missed. Next!

However, as might be predictable from such a frequent blogger, Seigo has derived several conclusions from events. To start with, Seigo noted that MakePlayLive was under at least two major handicaps. First, it was under-funded. "I think a more reasonable budget would have been closer to $500k. I could easily make it into a million dollar project," Seigo commented. Second, breaking into manufacturing is "very hard without a track record because you're building the relationship network from scratch. Saying 'no' to you is very easy: there are no existing sales agreements at risk."

Seigo also concluded that trying to deal with Asia while resident in Europe was a mistake:

Were I to start over, I'd forget Asia entirely for this kind of project and focus on Europe and the Americas for engineering and design. There are an increasing number of companies in North America and Europe that are able to do this work from printed circuit board design to manufacture and assembly [with] better quality engineering, fewer cultural gaps to jump through, and fewer time zones to wrangle. The cost will be higher, but the engineering results will be better and, amortized across the production run, the cost per device would not be significantly more. [Besides,] the Asian market for these kinds of products is entirely oriented towards retailers.'

Perhaps most important of all, Seigo now suspects that he was looking for support in the wrong places. "There isn't much business experience in the free software communities," he said, adding: "It doesn't help that most of the companies out there with a focus on free software do not care about mobile in the least. Those that do have their own semi-proprietary platforms already that they are protecting (for example, Android)."

In particular, most free software advocates fail to understand "that everything can go right with a new product and it can still fail. Success is never guaranteed -- not because anything is necessarily wrong with it, but because most new products fail -- which is why this hasn't destroyed my world or anything."

More specifically, Seigo said, "I don't think KDE is the right community with which to engage in these sorts of things. There is not enough understanding of appreciation for why such a thing is important, and certainly too few people willing and able to step up and make a difference-making contribution."

Certainly, an appeal to the KDE community seems to have done little to increase the pre-orders of Improv board — or even to have generated any discussion.

However, the lack of interest ran even deeper, according to Seigo. Despite the publicity given the Vivaldi tablet, "Few applications were made available with a touch interface. With a few exceptions, those that did make this leap were done by the small group of Plasma Active developers." By contrast, "those working on middleware and libraries tended not to see Plasma Active as a worthwhile support target, and some openly stated more than once that Plasma Active was happening at the expense of Plasma Desktop [the main KDE interface]" — an assertion that Seigo, who is a lead developer of both interfaces, characterizes as "both factually incorrect and absurd."

Nor did KDE e.V., the non-profit that oversees KDE affairs, show any "political will" to support Vivaldi or Improv, or even to explore their potential.

Examining how to strategically take full advantage of the opportunities presented by having a mobile stack, a digital content delivery system and store and what was the start of hardware/software cross-over projects would have made sense. Few were interested and opportunities were undoubtedly lost as a result.

Seigo conceded that, "I do think there is amazing potential energy in the various relevant communities. I am, however, at a loss as to how to tap into it." For whatever reason, Vivaldi and Improv simply failed to capture the imagination of many free-software advocates or KDE supporters. Perhaps the new ties between business and free software that MakePlayLive seemed to imply were simply too radical for those who could have helped it most.

Prospects for open hardware

All in all, Seigo found the effort to launch open hardware devices "amazingly stressful at times." Yet, asked if he would consider another effort, he replied, "Crazily enough, yes." He would currently be unable to finance it to the extent that he did MakePlayLive, but he would definitely like to try again. "Beyond all other things, I really hope that people see what we did as evidence that it is worth trying, rather than taking it as a lesson that it can't be done."

If nothing else, Seigo's analysis provides an informed perspective from which to view other free-software-based efforts, such as the Canonical and Jolla phones. Not only does he give insight into the delays and difficulties of manufacturing and marketing, but, if he is correct, regardless of technical quality, hardware with any degree of openness is likely to require not just one or two efforts, but many, before even one succeeds.

Comments (20 posted)

Page editor: Jonathan Corbet

Inside this week's Weekly Edition

  • Security: Browser tracking by "canvas fingerprinting"; New vulnerabilities in httpd, java, kernel, mozilla, ...
  • Kernel: A reworked BPF API; getrandom(); Control groups and accounting.
  • Distributions: Looking in on CentOS 7; Debian, EFF's open wireless router project, Oracle Linux, openSUSE, Slackware, ...
  • Development: A look at Xiki; Firefox 31; ownCloud 7; GStreamer 1.4.0; ...
  • Announcements: TDF congratulates UK, events, ...
Next page: Security>>

Copyright © 2014, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds