A look at Tizen 1.0 on the developer device

On the final day of the Tizen Developer Conference in San Francisco, Samsung distributed developer devices to registered attendees. Although they are not fully-functional consumer devices by any means, the devices did provide the first opportunity for most developers to see the new platform's mobile phone incarnation outside of an emulator.

Developer hardware

Physically, the device is shaped like a largish mobile phone, with a 4.5-inch diagonal (720 by 1280) touchscreen, three buttons, dual cameras, audio jack, and mini USB port. Inside, the platform is based on Samsung's Exynos 4210 system-on-chip, which is also found in Samsung's production phones. The processors are ARM: a dual-core Cortex A9 CPU, paired with a Mali 400 GPU. 1GB of RAM is available, plus 16GB of storage and a microSD card slot. Additional hardware includes WiFi, Bluetooth, GPS, and NFC, plus a SIM card slot.

As a development aid, the size of the device works to its advantage. Although it is large for a phone, the on-screen keyboard and other touch widgets are much easier to hit than those on a pocket-sized device. On the other hand, the microSD and SIM card slots can only be accessed by taking apart the outer shell, which means rapid-swapping of cards will be difficult. There may be developers who lament the lack of a hardware keyboard, too, although the device is intended to be accessed through the Tizen SDK over a USB connection.

Thus far it is unclear if the SIM slot is supported in software; the SIM cards I tried are not recognized even for non-modem operations such as contacts import, and no one appears to have gotten it up and running. Bluetooth is provided by a Broadcom 4330, which also supplies the WiFi and is FM radio-capable, but out of the box Bluetooth and FM are not enabled. Similarly, the NFC chip is a PN544 from NXP, which is the most widespread NFC chip supplier, but it is not yet used by the software stack. A standard complement of accelerometer, digital compass, and three-axis gyroscope sensor is also included.

Software

The installed software is the Tizen 1.0 "Larkspur" release, which officially debuted on April 30. The code represents a merger of Intel's MeeGo work with the Samsung Linux Platform (SLP), which was derived from LiMo. Although use of the Enlightenment Foundation Libraries (EFL) in the graphics stack is the most widely-publicized piece of SLP, there are others — including SLP's 3G telephony stack. At the conference, the Tizen project made it clear that oFono (an Intel project pre-dating MeeGo) was due to be ported over as Tizen's telephony stack; this may explain the SIM compatibility issue, as the device is alleged to be the same hardware used in Samsung's Android-powered Galaxy S2, and the modem driver may not have been ported to SLP.

Other parts of the Tizen 1.0 stack are interesting, too. The display server is X, although here again at the conference it was explicitly said that a port to Wayland is in the works. The effort consists of completing the port of EFL to Wayland, and of optimizing the speed of the platform's web runtime. Input is handled by the SLP Input Service Framework and by the XGesturesExtension originally authored by Canonical. The latter has since evolved into the uTouch framework, which makes the platform an unusual blend of technologies of different ages.

The underpinnings are certainly worth exploring, but most developers will probably be more interested in exploring the HTML5-based application API. The device includes a small set of reference applications — browser, phone, messages, clock, contacts, calendar, image gallery and audio player — plus a comprehensive system settings utility.

The demo applications bear many similarities to their MeeGo counterparts; perhaps the phone application more than others, and the media applications a bit less so, but all will feel familiar. Stylistically, most use flat colors, with sharp edges and corners, subtle highlighting via box outlines, and what desktop Linux vendors have come to call symbolic, monochrome icons. Most provide top tabs for navigating between the application's pages and place function buttons at the bottom of the screen. Where vertical scrolling is required, the scroll bar widget is a thin, translucent overlay that appears only while scrolling, and fades out. It does appear that the outermost edge of the screen is sensitive to scroll events, which can be tricky on mobile devices. On the other hand, none of the demo applications seems to show off multi-touch gesture recognition.

Several of the applications show off a nice assortment of data entry widgets. For example, the alarm clock setting screen features a drop-down selector that scrolls through the hour or minute options horizontally. But it sports a separate radio-button selector for choosing the notification type, which unrolls a bit like an unfolding map. On the whole, all of the transitions are smooth, and navigation between pages and elements is simple.

That said, a few of the demo applications use a completely different widget style. The calculator uses rounded buttons with a slight 3D embossed effect, gradients, and a faux-leather background. The stopwatch and timer screens in the clock emulate an LCD screen and a different faux-leather look, respectively, and feature their own UI widget styles. Were this a commercial product, the differences in look and feel would be cause for concern; in a demo suite, though, they simply show competing options for GUI styling.

Samsung's J.D. Choi showed off several advanced applications in his conference keynote, utilizing the platform's OpenGL support, but there is no official word when they — or a system update enabling missing pieces like Bluetooth, NFC, and SIM support — might land.

Development

Of course, demo applications cannot be the end of the story. Tizen seeded the devices to developers at the conference in order to jumpstart independent application development. But it will be quite some time before anyone should expect to see output. The Mer project has started to dig into the lower levels of the system, including the u-boot bootloader, and is working on booting an OS from the microSD card. Thomas Perl has succeeded in building and running Python and PyGame on the device, and more and more developers are popping in to ask practical questions on the mailing list and on IRC.

The preferred method for writing applications is the Tizen SDK, which for the moment is only available for Windows and 32-bit Ubuntu systems, but enterprising hackers have found ways around that limitation already. We will provide a more in-depth look at the SDK and assorted development tools in another story soon. In the meantime, it is good to finally see Tizen running on a physical hardware device. Yes, the experience is much the same as that provided by the SDK emulator and reviewed around the web, but a modern development device makes all the difference in the world. There may be no third-party applications to install just yet, but that was also the situation shortly after the release of the Nokia N950 last year, and today there are hundreds. Which just goes to show that if you want software, the simplest thing you can do is give the open source community tools and a platform on which to build.

[ The author would like to thank the Tizen project and the Linux Foundation for support to attend the conference. ]

Comments (12 posted)

The new "community" Mandriva

Mandriva SA, the company behind the Mandriva distribution, has announced it will return control of the distribution back "to the community". But exactly how that will play out in practice remains unclear, since the company was unable to convince the Mageia distribution to participate in the new effort — but, conversely, is cooperating with Mageia for future products.

Communities and entities

Mandriva made the community hand-over announcement on May 17, saying that it had decided to "transfer the responsibility of the Mandriva Linux distribution to an independent entity." The announcement outlines only a rough plan, with the formation of a governing body that will include representatives from Mandriva SA, but will not be under the company's direct control. The company will also continue to contribute its engineering resources to Mandriva development. The announcement then states that the details of the organization's governance model, processes, and other infrastructure will be fleshed out over the next few months, a process to be handled by a still-in-formation workgroup of community members.

There have been several forks of the Mandriva distribution, but by far the largest is Mageia. Yet the Mageia board announced on May 21 that it had decided not to join the new Mandriva workgroup — at least, not as a group. The announcement enumerates five reasons for the decision.

First, the Mageia board feels that the Mageia.org organization already meets the needs that Mandriva SA gave for forming a new entity, and felt that the company should have joined it instead. Second, Mageia has "invested a lot of time and energy" to define Mageia as it is. Third, there is a lack of information about the future direction of the proposed Mandrake community entity. Fourth, the Mageia project does not have enough resources to take on a new project in addition to its existing work. Finally, because Mageia is already free software, code sharing between the projects can already happen without establishing any formal arrangement.

Some of the listed reasons are perplexing. Clearly the first two indicate that the project thought Mandriva SA should simply adopt Mageia as its community distribution as-is. That would offer technical challenges, since the two projects have diverged in key areas since the original split (package managers, for example), as well as trademark issues. The last two are easily justifiable — patches do already flow back and forth between the distributions, and one rarely hears of a distribution project with too much time and developer resources on its hands.

But the middle reason is a puzzle. It sounds as if the Mageia board rejected an offer to take a seat (or seats) on the Mandriva community governance entity, when the stated offer was a place in the workgroup that will define the entity's standards and practices. If the Mageia board was concerned about the future direction of the new project or entity, surely participating in the workgroup would be the best way to influence that direction for the better. Comments on the Mandrake SA announcement indicate that at least one other Mandriva fork, ROSA, is joining the workgroup.

Products, products, products

The other wrinkle in the Mandriva makeover story is Mandriva SA's May 20 product announcement. In the post, the company outlines new product plans and how they relate to the evolving situation with the base distribution. The desktop product will be based on the new community-managed version of Mandrake, as will its OEM and education offerings. Pulse2, the company's corporate system deployment and management product, will continue to be developed and contributions will be made back to the Mandriva community. The company's server offering, however, will be based on Mageia.

Mageia confirmed the arrangement in the blog post linked to above, saying that it arose out of talks with Mandriva SA. It is not clear whether the decision involves any sort of development effort on Mageia's (or individual Mageia developers') part, but the project did take pains to emphasize that it had no bearing on the future development of Mageia itself. Several commenters on the Mandriva announcement asked why the company would choose to build its server product on a different code base than its desktop product, particularly in light of the divergence between Mandriva and Mageia and Mageia's 18-month life cycle (which is brief for a server distribution).

So far, there has not been an elaboration on the business server arrangement from either party. But that could simply be lack of time; Mageia released the final version of Mageia 2 on May 22. We previewed the release in April. Mandriva, meanwhile, has started to flesh out the beginnings of its community development plan on its wiki, and ROSA has rolled out its latest release. Hopefully, with the new release out the door at Mageia, and the Mandriva workgroup taking shape, we will soon hear more details.

Comments (5 posted)

A uTouch architecture introduction

uTouch capabilities

The new X.org multitouch features allow for multitouch support in applications. We now have a software stack, uTouch, built on top of this multitouch support that can provide for practically any gesture scenario imaginable.

A "gesture" is normally thought of as a two-dimensional movement made by the user on some sort of input device—a two-finger pinch, for example, or a three-finger downward drag. Teaching a computer to recognize these movements requires a lower-level description, though; in uTouch, this description consists of values like the number of touches, movement thresholds, and timeout values. An application may register a "gesture subscription" describing a specific gesture and be notified when that gesture is recognized by the uTouch subsystem. Those notifications take the form of a sequence of events describing the gesture motion over time.

Key to understanding how uTouch works is knowledge of all the typical gesture use cases. First, we have the concept of gesture primitives: drag, pinch (including both "pinch" and "spread"), rotate, and tap. These primitives make up the foundation of all intuitive gestures. They can be strung together as needed for more complex gestures, such as a double tap. Stroke gestures, such as drawing an ‘M’ to open the mail client, may be recognized as a specific long gesture sequence, or as a sequence of drag gestures. Note, however, that uTouch does not have stroke gesture detection facilities built-in.

Second, there are two fundamental object interaction types: single motion, single interpretation gestures and direct object manipulation. The former involves gestures like a two-touch swipe to go backward and forward through browser history, while the latter involves gestures like a three touch drag to move an application window around the desktop.

The single motion, single interpretation gestures require thresholds and/or timeouts. For example, the colloquially implied difference between a swipe and a drag is that a swipe must be a quick motion in a given direction, whereas a drag may be any motion that manifests in a displacement in space. To put it in uTouch gesture subscription terms, a swipe is a drag primitive gesture with a displacement threshold that must be crossed within a specific amount of time. For example, when implementing browser history gestures a two-touch swipe may be implemented with a threshold of 100 pixels over half of a second. In contrast, direct object manipulation usually implies a zero threshold. For example, as soon as three touches begin on a window, the window should be movable.

Most simple gesture interactions may be handled through gesture subscriptions consisting of the required gesture primitives and the object interaction types. However, there are times when an application needs to have further control over gesture recognition. For example, a bezel drag gesture occurs when the user begins a drag from the bezel of the screen and moves inward. This gesture must be distinguished from the user initiating a touch at the edge of the screen. The problem lies in the fact that both the bezel drag and the direct touch near the edge of the screen look indistinguishable at the beginning of the gesture. The distinguishing aspect is that the bezel drag is perpendicular to the bezel and has a non-zero initial velocity as seen by the touchscreen, whereas the direct touch near the edge of the screen will likely not have an initial velocity and/or may not be moving perpendicular to the bezel. To cater for a client that cares about one of these gestures but not the other, uTouch requires the client to accept or reject every gesture. When a gesture is rejected, the touches may be replayed to the X server, which allows for the mixing of gestures and raw multitouch in the same application.

Another facet of uTouch, as hinted above, is that, by default, it operates through "touch grab" semantics. When used on top of X.org, uTouch gestures are recognized from touches received through touch grabs. One benefit of this approach is the ability to mix gestures and raw multitouch in the same application. However, it also allows for priority handling of gestures. For example, system gestures may be handled by a client listening to touches through a grab on the root window. When gestures are not recognized or are rejected by the uTouch client, the touches are replayed to the next touch grab or selecting client. Thus, global gestures, application gestures, and raw multitouch events are all possible when using uTouch.

The last major feature of uTouch is the ability to recognize multiple simultaneous gestures in the same area. For example, imagine a game where the user pinches bugs on the screen to squash them. The screen is one large gesture input area, but the user may use both hands to pinch bugs. In order to facilitate this interaction mode, whenever new touches begin within the gesture area they are combinatorially matched with other touches that begin within a "glue" time period. In our game example there is a two-touch pinch gesture subscription. If four touches begin in the game area within the glue time period, six combinations of potential gestures will be matched. As touch events are delivered, the state of each matched gesture will be updated and then checked against the threshold and timeout for the gesture subscription. If a gesture meets the threshold and timeout criteria, it will be delivered to the client. The client can then attempt to match up the touches of the gesture against its context to determine whether to accept or reject each gesture. In the example below, there will be four pinch gestures sent to the client:

(Bug icons licensed under LGPL)

There will be potential pinch gestures for: AB, CD, AD, and BC (AC and BD, by virtue of moving in the same direction, are not considered to be potential pinches). The application must determine which gestures make sense. One method would be to hit test the initial centroid of each gesture against the bugs on the screen. All gestures that hit a bug are accepted. Note that uTouch automatically rejects overlapping gestures, so as soon as AB and CD are accepted, AD and BC will be implicitly rejected.

There is a twist to this complex logic, however. Gesture events are received serially. The client may need to know if more gestures are possible for a set of touches. For example, if both one-touch and two-touch drag gestures are subscribed, a two touch drag will cause two one-touch drag gestures and a two-touch drag gesture. If the uTouch client receives a one-touch drag first, it may not realize that a two-touch drag is coming for the touch as well. To handle this issue, a gesture property is provided to denote the finish of gesture construction for all of its touches. When a gesture has finished construction, the client knows that it has received all possible gestures containing the same touches. Thus, in the one- and two-touch drag example the one touch gesture will not emit the gesture construction property until at least the two-touch gesture begin event has been sent to the client.

The uTouch stack was designed to be flexible and provide for all possible gesture use cases. However, it is recognized that not all clients will care about multiple simultaneous gestures. There are plans to create a gesture subscription option that precludes the ability to have multiple simultaneous gestures. This will effectively push some policy into the recognizer, such as a preference for gestures with more touches. This will be particularly useful when subscribing to gestures on an indirect device, like a touchpad, where multiple simultaneous gestures are likely not wanted.

Lastly, uTouch is a complete gesture stack that surpasses the functionality of all available consumer platforms. uTouch works well with both touchscreens and touchpads, and supports both gestures and raw touch events in the same window or region of an application. In contrast, Windows only supports touchscreens and either gestures or raw touch events, but not both, in a given window. OS X supports touchpads but not touchscreens. Mobile platforms are limited to touchscreen support and single-application gestures at a time due to their modal task design. In contrast to each of these platforms, uTouch has been designed from the ground up to support all device types and all known use cases, including multiple applications and windows at the same time.

The technical architecture of uTouch

uTouch consists primarily of three components: uTouch-Frame, uTouch-Grail, and uTouch-Geis. Each of these will be described briefly below.

uTouch-Frame groups touches into units that are easier for uTouch-Grail to operate on. Gestures are recognized per-device and per-window, so touches are grouped into units representing pairs of devices and windows. This is also where all backends for each window system are implemented. uTouch-Frame events are platform independent.

Some window systems, like X11, also have the concept of touch sequence acceptance and rejection. This functionality is provided through uTouch-Frame as well.

Touch sequence acceptance and rejection is a core aspect of the uTouch stack when used for system-level gestures. Imagine a finger painting application listening for raw touch events (not gestures) is open on a desktop environment where three-touch swipes are used to switch between applications. When the user performs such a swipe, uTouch accepts the touch sequences on behalf of the window manager and switches applications. This prevents the painting application from handling (or even seeing) the touches. In contrast, when the user performs a three-touch tap, uTouch rejects the touch sequences because they do not match a known gesture. The painting application then receives the rejected touch sequences.

uTouch-Grail is the gesture recognizer of the uTouch project. It takes the per-device, per-window touch frames from uTouch-Frame and analyzes them for potential gestures.

Grail events are generated by frame events. Rather than duplicate the uTouch-Frame data, grail events contain gesture data and a reference to the frame event that generated it. This allows for uTouch clients to see the full touch data comprising a gesture.

Grail gesture events are comprised of a set of touches, a uniform set of gesture properties, and a list of recognized gesture primitives. Again, the supported primitives are: drag, pinch, rotate, and tap. The gesture properties are:

1. Gesture ID
2. Gesture state (begin, update, end)
3. A list of touch IDs for the touches comprising the gesture
4. The uTouch-Frame event that generated the Grail event
5. The original and current centroid position of the touches
6. The original and current average radius, or distance from the centroid, of the touches
7. A best-fit 2D affine transformation of the touches from their original positions
8. A best-fit 2D affine transformation of the touches from their previous positions
9. A flag denoting whether the gesture construction has finished

Drag, pinch, and rotate properties are encapsulated by the affine transformations. For more detail on how to use 2D affine transformations, please see this excellent Wikipedia article on transformation matrices.

During operation, a pool of recently-begun touches is maintained. In the current implementation this pool includes any touches that have begun within the past 60 milliseconds of "glue" time. When a new touch begins, it is combined in all possible combinations with touches in this pool in order to create potential gestures matching any active subscriptions.

A new gesture instance is created for each combination of touches. Each instance has an event queue, and new instances have one begin event describing the original state of the touches. The events are queued until any gesture primitive is recognized. When frame events are processed, any changes to touches in a gesture instance generate a new grail event. The new touch state is analyzed, and subscription thresholds and timeouts are analyzed to determine if any of the subscription gesture primitives have been recognized. For example, the default rotate threshold is 1/50th of a revolution, and the default rotate timeout is one half second. If the threshold is met before the timeout expires, the rotate gesture primitive is recognized.

When a gesture primitive has been recognized, the grail event queue is flushed to the client. The client must process the gesture events and make a decision on whether to accept or reject each gesture.

uTouch-Geis is the C API layer for the uTouch implementation. uTouch originally began as a private X.org server extension. It has since been updated, bringing it out of the X.org server and into the client side of the X11 system. This required a complete rewrite of uTouch-Frame and uTouch-Grail. However, we have managed to maintain API and ABI compatibility through uTouch-Geis, albeit with a few behavioral changes. uTouch-Geis has two API versions, version 1, a simpler interface, and version 2, an advanced interface. Although both are currently supported, the first version is deprecated in favor of the more flexible second version.

uTouch-Geis also makes gesture event control simpler by wrapping much of the X.org interaction behind an event loop abstraction. The uTouch stack requires careful management of touch grabs and timers. Any client may use uTouch-Frame and uTouch-Grail directly, but uTouch-Geis vastly simplifies incorporating gestures into an application. See the uTouch-Geis API documentation for more information.

Toolkit and application development

We also have begun work on a gesture recognition system for the Chromium web browser. There are many potential gesture interactions that we hope to leverage in the browser. An initial implementation was proposed, but a rearchitecture of the gesture plumbing in Chromium required us to refactor it. We hope to merge an implementation into Chromium in the next few months.

Conclusion

Over the past two years the uTouch team has been working hard to bring multitouch gestures to the Linux desktop. We now have a complete stack that rivals, and in many ways surpasses, what is possible on other platforms. We look forward to further integration of uTouch gestures in desktop environments and applications, and we encourage everyone to take a look at what our stack has to offer.

Comments (12 posted)

Next week's edition will be one day late

Due to the US Memorial Day holiday on May 28, and a day off for the LWN crew, next week's edition will be published on Friday June 1. Whether you are celebrating on Monday or not, we hope you have a nice 28th of May, and we'll resume normal service on the 29th.

Comments (1 posted)

Exploring options for the openSUSE security policy

Partly in response to Linus Torvalds's (in)famous Google+ rant about desktop security—openSUSE in particular—Andreas Jaeger and others have started gathering use cases for the administration of Linux systems. The target is to try to find a balance between security and convenience for openSUSE users. Part of the difficulty is that Linux distributions are installed on a variety of systems with very different security needs, which makes it difficult to choose any particular default security scheme.

A single-user desktop or laptop has very different security needs from those of a shared desktop or server. Torvalds's complaint was specifically about the root password being needed to add a printer to his daughter's laptop, but he was also unhappy with needing privileges to change the time zone and wireless network. For a system where the only user is also the administrator—at least a semi-privileged administrator—it makes sense to allow those kinds of changes without the root password. But on other systems, like shared machines or servers, it almost certainly doesn't make sense for random users to have those powers.

That's where the balancing act comes in. If a distribution is meant to be installed for several different scenarios, there is no One True Way to set the privileges of users. Even for single-user systems there are differences. Torvalds undoubtedly administers his own laptop differently than he wants his daughter's handled. For the former, he may want to allow package installation using his own password, the root password, or possibly no password at all (though one would guess that's not likely). But, for his daughter's system he wants to hold the root password himself, while allowing a limited number of privileged operations to be done by her. While Torvalds is a high-profile user, his complaints are likely similar to those of others.

In order to help determine what the right security configuration is for openSUSE, Jaeger, Marcus Meissner, and Ludwig Nussel put together a list of use cases that describe tasks that users want to do along with a short security evaluation of each. Things like setting the system time, accessing the network, changing firewall settings, adding printers, package installation, and so on, are listed. Jaeger also posted a "call for action" to the opensuse-factory mailing list, asking for feedback and new use case suggestions.

Much of the resulting conversation centered around the "roles and profiles" that were also described on the web page. There is a tension between convenience for single-user machines and those with more complicated situations, thus higher security needs. But, even among those who would like to see less privilege required for some operations on single-user machines, there are differences of opinion on which operations—and what privilege to require. For example, Marguerite Su wants to be able to install software without the root password on a laptop, but others including Bryen M Yunashko are not so sure that's what they want.

There are also different classes of package installation to consider. Installing an update to a package from a "known good" repository is very different from installing a new package, downgrading a package to an earlier version, or adding a repository. The credentials required for each might be different depending on the scenario.

That part of the thread highlights part of the difficulty in finding the right default settings. The distribution will need to have some way to specify which of the profiles (e.g. single-user, administered single-user, multi-user, server, etc.) should govern, say at installation time, but will also need some way for the overall profile to change, while also allowing individual users to tweak the settings based on their needs. It is a more complex problem than it might seem at first.

Suggestions in the thread range from Carlos E. R.'s installation-time dialogs to determine the right profile to Su's idea of PolKit packages for different profiles and use cases to Hans Witvliet's more granular approach with multiple types of administrative roles that could be assigned to a user. Any or all of those could make up some part of a solution, but in a response to Witvliet, Jaeger focused on the question of defaults:

Finding workable defaults that will cover the majority of cases is clearly needed. Finding a set that will avoid rants like Torvalds's, while still giving a reasonable level of security to openSUSE users, is paramount. But there also needs to be a way for those with different needs to adjust the policies appropriately. Pulling all of that together in a way that is easy to understand, use, and tweak, will be an even harder problem. But it's a problem that needs solving and not just for openSUSE; there are opportunities for cross-distribution collaboration here as well.

Comments (19 posted)

Security quotes of the week

[...]
-       .UnicodeString          = L"LUFA Keyboard Demo"
+       .UnicodeString          = L"Keyboard (%n%n%n%n)"

Comments (1 posted)

Security vulnerability in sudo's netmask function patched (The H)

Comments (none posted)

The problem with nerd politics (The Guardian)

Comments (19 posted)

A Tale of Two Pwnies (Part 1)

Comments (44 posted)

android-tools: udev rules set insecure permissions

Comments (none posted)

backuppc: cross-site scripting

Comments (none posted)

chromium: multiple vulnerabilities

Comments (none posted)

feedparser: denial of service

Comments (none posted)

ikiwiki: cross-site scripting

Comments (none posted)

libxml2: code execution

Comments (none posted)

openoffice.org: code execution

Comments (none posted)

perl-Config-IniFiles: insecure temporary files

Comments (none posted)

pidgin-otr: code execution

Comments (none posted)

rubygem-mail: arbitrary command execution

Comments (none posted)

sympa: authorization bypass

Comments (none posted)

sudo: privilege escalation

Comments (none posted)

update-manager: multiple vulnerabilities

Comments (none posted)

wireshark: denial of service

Comments (none posted)

Kernel release status

The 3.5 merge window is open as of this writing; see the separate article below for a summary of interesting changes merged so far.

Stable updates: 3.0.32 and 3.3.7 were released on May 21, 3.2.18 on May 21, and 2.6.34.12 on May 22.

Comments (none posted)

Quotes of the week

	else if(!strcmp(str, "pony")) {
		tsc_clocksource_reliable = 1;
		sched_clock_stable = 1;
		tsc_perfect_smp_synchronization = 1;
	}

	else if (!strcmp(str, "real")
	     panic("Can't handle real TSCs!\n");

Comments (1 posted)

The 3.5 merge window opens

• The TCP connection repair interface, useful for the implementation of checkpoint/restart functionality, has been merged.

• The networking stack has gained support for RFC 5827 early retransmit, a mechanism aimed at speeding recovery from packet loss.

• The CoDel queue management algorithm, which, hopefully, will be an important component in the solution to the bufferbloat problem, has been merged.

• The seccomp filters mechanism has been merged; it allows processes to reduce the set of available system calls through the use of a mechanism based on the Berkeley packet filter. See Documentation/prctl/seccomp_filter.txt for details.

• The Yama security module has two increasingly restrictive modes for controlling access to the PTRACE_ATTACH functionality.

• The logging reliability patch set has been merged.

• The NUMA scheduler has been rewritten with the result that it will make different, hopefully better scheduling decisions. Also, as has been threatened for some months, the power-aware scheduling code has been removed in the hope that somebody will replace it with something that actually works.

• A lot of code has been removed in this development cycle, including the ixp2000 Ethernet driver, support for the sun4c SPARC CPU, the ip_queue netfilter module (superseded by nfnetlink_queue), all support for token ring networking, drivers for all MCA-based network cards, support for the Econet protocol, support for ARMv3 processors, support for Intel IXP2xxx (XScale) processors, support for ST-Ericsson U5500 development boards, the Motorola 68360 serial port driver, and the workqueue tracer.

• New drivers include:
  • Processors and systems: Blackfin BF609-based boards, and Renesas Armadillo-800 EVA and KZM-A9-GT boards.

  • Miscellaneous: TI TPS65090 power regulators, TI Palmas series power management chips, RICOH RC5T583 power regulators, Freescale MXS, IMX6Q, IMX53 and IMX51 IOMUX controllers, ST Microelectronics SPEAr3xx pin controllers, Renesas Emma Mobile SoC GPIO controllers and integrated serial ports, Intersil ISL29028 concurrent light and proximity sensors, TAOS TSL/TMD2x71 and TSL/TMD2x72 light and proximity sensors, Analog Devices AD8366 variable gain amplifiers, and Atmel AT91 analog to digital converters.

  • Network: WIZnet W5100 and W5300 adapters, Marvell Avastar 88W8797 wireless chipsets, Emulex One Connect InfiniBand-over-Ethernet controllers, and GCT Semiconductor GDM72xx WiMAX controllers.

  • USB: Marvell PXA USB OTG controllers, Broadcom BCMA and SSB host controllers, NXP ISP1301 USB transceivers, and NXP LPC32XX USB peripheral controllers. Also added is a "configurable composite gadget" driver that allows user-space configuration of enabled functions.

  • Staging graduations: the industrial I/O (IIO) core has moved into drivers/iio; VME drivers are now in drivers/vme, and the Intel management engine interface (MEI) driver is now in drivers/misc.

Changes visible to kernel developers include:

• The many variants of the NLA_PUT() macro used with netlink have been removed. Code should use one of the nla_put() versions instead and make its error handling explicit.

• The mac80211 layer has gained support for MBSS mesh synchronization.

• There is new core support for the writing of near-field communication (NFC) drivers using the HCI specification; see Documentation/nfc/nfc-hci.txt for details.

• The "regmap" subsystem, which centralizes the handling of banks of device registers, now has support for registers in I/O memory.

• The pin control subsystem now has full device tree support.

• The Android "switch" class has been brought into the mainline and extended into a general "external connector" framework.

• The "ramoops" mechanism has been reworked to use the pstore interface.

If the usual schedule holds, this merge window can be expected to close around June 4. Watch this space next week for coverage of the next sets of patches to be pulled into the mainline for the 3.5 development cycle.

Comments (5 posted)

Preparing for nonvolatile RAM

Matthew Wilcox raised the issue, noting that nonvolatile memory (NVM) is coming, that it promises bandwidth and latency numbers similar to those offered by dynamic RAM, and that, being cheaper than DRAM, it is likely to be offered in larger sizes than DRAM is. He later disclaimed any resemblance between this description and any future products to be offered by his employer; it is, he says, simply where the industry is going. Given that, it would be a good idea for the kernel community to be ready for this technology when it arrives.

One part of being ready is figuring out how to deal with nonvolatile memory within the kernel. The suggested approach was to use a filesystem:

A filesystem approach would allow the association of names with regions of NVM space; an API was then proposed to allow the kernel to perform tasks like mapping regions of NVM into the kernel's address space.

One question that came up quickly was: won't the use of the filesystem model slow things down? There is a lot of overhead in the block layer, which was not designed to deal with "storage" that operates at full memory bandwidth. Matthew was never thinking of bringing in the full block layer, though; instead, he said: "I'm hoping that filesystem developers will indicate enthusiasm for moving to new APIs". Such enthusiasm was in short supply in this discussion; that is probably more indicative of a lack of thought about the problem than any sort of active opposition (which was also in short supply).

James Bottomley, though, questioned the filesystem idea, suggesting that NVM should be treated like memory. He said that the way to access NVM might be through the kernel's normal memory APIs, with nonvolatility just being another attribute of interest. One could imagine calling kmalloc() with a new GFP_NONVOLATILE flag, for example. The only problem with this approach is that it is not enough to request an arbitrary nonvolatile region; callers will usually want a specific NVM region that, presumably, contains data from a previous use. So the memory API would have to be extended with some sort of namespace giving reliable access to persistent data. To many, that namespace looks like a filesystem; James suggested using 32-bit keys like the SYSV shared memory mechanism does, but admirers of SYSV IPC tend to be scarce indeed on linux-kernel.

So, while there are a lot of details to be worked out, some sort of name-based kernel API seems certain to come about. Then there will be a mechanism, either through the memory-related or filesystem-related system calls, to make NVM available to user space. But that leads to another, perhaps harder question: what, then, do we do with all that fast, nonvolatile memory?

Some of it, certainly, could be used for caching; technologies like bcache could make good use of it. The page cache could go there; Matthew suggested that the inode cache might be another possibility. Both could speed booting considerably, though it would be necessary to somehow validate the cache contents against filesystems that could have changed while the system was down. Boaz Harrosh suggested that filesystems could store their journals in that space, speeding journal access and reducing journal I/O load on the underlying storage devices. He also mentioned checkpointing the system to NVM, allowing for quick recovery should the system go down unexpectedly. Vyacheslav Dubeyko had some wilder ideas about how NVM could eliminate system bootstrap entirely and make the concept of filesystems obsolete; instead, everything would just live in a persistent object environment.

Clearly, many of these ideas are going to take some time to come to fruition. Nonvolatile memory changes things in fundamental ways; Linux may have to scramble to keep up, but, then, that is a high-quality problem to have. It will be most interesting to watch how this plays out over the coming years.

Comments (43 posted)

Removing four bytes from the kernel ABI

Four bytes may not seem like a lot of space—typically it isn't—but when that space is wasted millions of times, it starts to add up. In addition, if the extra space has become part of the kernel ABI (intentionally or not), it will be difficult to remove it. That particular problem came up again in a recent linux-kernel discussion regarding the trace event header.

Just over a year ago, we looked at the unused lock_depth field in event headers. Frederic Weisbecker had added the field temporarily to assist in removal of the big kernel lock (BKL), and once the BKL was gone Steven Rostedt removed those, now useless, four bytes from the header. Unfortunately, in the interim, PowerTOP had started accessing events in the perf ring buffer, so removing lock_depth broke PowerTOP. That field wasn't actually used by PowerTOP, but the tool expected the header to have a particular size, which changed after Rostedt removed the wasted space.

That led to a reversion of the removal, which means that every event recorded by ftrace or perf has added overhead. The event format is fully self-describing, however, so there is no need for utilities like PowerTOP to grub around in the binary data making assumptions about what the format is. It was, however, easier to read the data directly rather than parse the format description, which is why PowerTOP did so. Rostedt has created a library to parse trace events using the format data that the kernel provides to avoid that situation in the future. That library was picked up by the recently released PowerTOP 2.0, so Rostedt posted an RFC asking when the lock_depth field—renamed to padding as part of the revert—could be removed.

Linus Torvalds was not particularly concerned about the wasted space, but did want to understand which distributions were picking up the new PowerTOP. It turns out that the version in Fedora 14 (which Torvalds said he still uses sometimes) is old enough that it doesn't use perf events at all, so it is unaffected. More recent Fedoras (16, 17) are using PowerTOP 1.98 which won't work with kernels built without the padding.

The assumption in the thread is that distributions will be picking up PowerTOP 2.0 for releases coming later in the year, but that still leaves users who build their own kernels on existing distributions in a bit of a bind if the padding is removed. Existing distributions also have various lifespans, and some will not be picking up the latest PowerTOP at all. Rostedt asked how long the kernel needed to support older distributions. PowerTOP, it seems, is in a different category from other applications because it is a developer-oriented tool. So Torvalds was willing to see the kernel change even if some distributions get left behind:

Arjan van de Ven concurred, pointing to 3.6 as a potential time frame to remove the padding, noting that those who haven't updated their distribution to get the newer PowerTOP are unlikely to be updating their kernel either. Rostedt said he will queue the patch up for 3.6 or 3.7.

While the four bytes seems unimportant to both Torvalds and Ingo Molnar, Rostedt pointed out that it is a frequent problem for tracing users. Beyond that, though, he disagrees with Molnar's contention that the wasted space is merely a "cosmetic detail":

For memory-constrained situations, for example on embedded devices or for users trying to squeeze every process they can onto their systems, reducing the overhead of events can make a difference. By capturing more events in the same amount of memory, there is a better chance of finding the problem that tracing was enabled for. When the issue came up a year ago, David Sharp of Google noted that the size of events was a big problem for the search giant. Others undoubtedly face similar challenges.

While the format of the perf ring buffer data may soon be a solved problem—though it's possible, if unlikely, that other tools are manually pulling data from the ring buffer—tracepoints as a whole are still an unresolved ABI issue. Right now, much of the work is in adding new tracepoints, but some day one or more of those may need to come out or be modified. If tools are dependent on specific tracepoints providing the exact same information in just the right place in the code, changing those will be a real problem. And it will be one that is difficult for a library to paper over.

Comments (14 posted)

Moving on

In the Debian camp, Steve McIntyre recently raised the alarm: there appears to be no way that the upcoming "wheezy" distribution will fit onto a single CD, at least if one wants a GNOME or KDE desktop environment. Debian, it seems, may have to consider alternatives to those CDs; these could include requiring multiple CDs (bringing back memories of the old floppy days), requiring the use of DVD or USB media, or only supporting a combined CD/network installation. All of these options would have the effect of making Debian less accessible to users with older hardware, a change that is not seen as being desirable in the Debian camp.

As it happens, Debian may have another option: use XZ compression in its binary packages. The additional space gained by this move is impressive—down to about 2/3 of the size achieved with gzip. The biggest downside would appear to be the need to rebuild much of the Debian repository, which is not a small task. But, for the wheezy release, XZ may help the project to avoid some more difficult choices.

But any such gains will clearly be temporary; the size of the software shipped by all distributions is clearly on the increase. This is not just a problem for Debian; Steve Langasek noted that Ubuntu 12.10 will be the first Ubuntu release that will not fit on a CD. In the Fedora camp too, "some folks" are pushing for the abandonment of the CD image for the Fedora 18 or 19 release. Some developers are arguing for a stronger push back against software bloat, but, in general, the consensus seems to be that putting a working distribution with a contemporary desktop environment onto a single CD is an increasingly impractical thing to try to do.

One might well argue that, in an age when bottle openers come with 16GB of storage, worrying about an old medium with capacity measured in megabytes makes little sense. Finding a system that only supports CD as a boot medium is, at this point, possibly even harder than finding one with a floppy drive. There seems to be little point in limiting the size of the installation image to 700MB when computers with such limitations have not been sold in some time. At least, no point unless one's wish is to constrain the overall size of the system in general.

The problem, of course, comes in two forms: old systems and limited network bandwidth. In the parts of the world where distribution hackers tend to be found, CD-only machines are hard to come by. In many other areas where people would like to use Linux, such machines are still in use. One can argue that contemporary Linux distributions may not run well on such limited hardware anyway, but it would still be better to avoid making life harder for the owners of those machines. Network bandwidth also matters: for many people, downloading a full DVD-size image can still be a long and painful exercise; an image limited to CD size has a higher chance of being fully downloaded before the link goes down or other users become too grumpy. Forcing a multi-gigabyte download on such users is not a good way to win friends.

The best answer in this case may be to provide a minimal bootstrap CD image that can then perform an installation from either the network or a USB drive. That allows the installation image to grow for the large numbers of users who can benefit from more software on the installation media while not excluding those who have more limitations. It seems like an approach that can provide the best solution for nearly everybody involved.

The Debian community has been pondering a related question: is the time coming for the distribution to move away from 32-bit builds for the x86 architecture? Ben Hutchings recently posted a proposal to that effect, noting that most new systems have 64-bit processors, and that the number of 64-bit ("amd64") installations has nearly equaled the number of 32-bit installations. Ben would like to see the 64-bit kernel installed by default in the wheezy release and, after perhaps two or three more release cycles, no more 32-bit x86 kernels built at all. These are Debian-length release cycles we are talking about, so the end of 32-bit kernels is far from imminent, but it's still near enough to get attention. (This proposal would not end the building of 32-bit user space, which can run just fine on a 64-bit kernel).

Certainly there are a lot of good reasons to go with a 64-bit kernel on contemporary hardware. Even if user space runs in 32-bit mode (or the upcoming x32 mode), an option that works nicely at this point, a 64-bit kernel will run more efficiently for a number of reasons. Given that there are fewer and fewer reasons to use a 32-bit kernel, perhaps Debian should stop putting resources into building such kernels.

Here, though, the situation is not quite so clear cut. As some developers pointed out, 32-bit-only x86 systems are still shipping. Ubuntu considered moving to a 64-bit kernel for the 12.04 release, but backed off after somebody noticed that 25% of the submissions to the Ubuntu Friendly hardware database came from 32-bit machines. 32-bit processors are still well established in the x86 world, so it seems unlikely that distributors will be able to stop supporting them anytime soon.

The fast-moving nature of technology ensures that distributors will continue to face these questions indefinitely. In general, Linux distributors have erred on the side of keeping support for old hardware; few distributors want to alienate potential users. But supporting older systems is not free; those systems can place constraints on what can be packaged, slow down performance on newer systems, take up developer time, and present testing challenges when nobody has the relevant hardware around anymore. So there comes a point where older systems have to be cut loose. Some newer-style distributions (CyanogenMod, say) are showing signs of having less patience with older hardware—arguably an unsurprising decision for a project supporting well over 50 separate targets. Expect more discussions like the above as distributors try to make the most of their limited development and build resources.

Comments (28 posted)

Distribution quotes of the week

Comments (8 posted)

Mageia 2 is out

Full Story (comments: none)

Fedora 17 release pushed back to May 29

Fedora Project Leader Robyn Bergeron announced that the release of Fedora 17 has been delayed by one week, to May 29. "GA [General Availability] for F17 is now scheduled for 2012-05-29. Adjustments to the schedule and wiki will be completed later today. We will be meeting again next Thursday, 2012-05-24, for another Go/No-Go meeting." The decision was reached in order to close four outstanding blockers. A second F17 release candidate (RC2) will be spun in the interim.

Comments (36 posted)

Updated Debian 6.0: 6.0.5 released

Full Story (comments: none)

Bits from the Release Team: Freeze approaching!

Full Story (comments: none)

Fedora Board, FESCo, and FAmSCo elections: Town hall meeting schedule

Full Story (comments: none)

Mandriva Linux to "return to the community"

Comments (5 posted)

Mageia comes full circle

Comments (none posted)

Distribution newsletters

• DistroWatch Weekly, Issue 457 (May 21)
• Maemo Weekly News (May 21)
• Ubuntu Weekly Newsletter, Issue 266 (May 20)

Comments (none posted)

The Russians Are Coming: A First Look At Rosa 2012 Marathon (O'Reilly)

Comments (1 posted)

Differing intentions towards Web Intents

Developers from Google's Chrome team have merged support for Web Intents into the WebKit browser engine. That has the potential to give a big boost to the proposed web application inter-process communication (IPC) framework, which up till now has been largely theoretical — partly because, without browser support, there was little incentive for web developers to implement it. But the chicken-and-egg problem is not the only unsolved issue for Web Intents; the technology remains in nascent, draft form, with many people outside of the Chrome camp expressing concern over its completeness and its scope. Furthermore, Mozilla, which had been collaborating on the project in 2011, now seems to be headed in its own direction with its competing IPC framework.

Web Intents takes its name and broad strokes from Android's Intents IPC mechanism. The project is currently curated at the World Wide Web Consortium (W3C), with public discussion taking place on the public-web-intents mailing list.

Google Chrome and Chromium gained built-in support for Web Intents with the Chrome 18 release on March 29, although it still had to be activated with a runtime flag. Chrome 19, which arrived on May 15, enables this support by default. As a blog post on the subject explains, however, for the moment the only way for web developers to deploy a Web-Intents-speaking application is to publish it through the Chrome Web Store. That is because there is not yet an agreed-upon method for a web application to "register" that it understands Web Intents. The Chrome team would like to add an <intent> tag to HTML, but in the meantime the only solution is for the application to make its registration known through the Chrome Web Store's app manifest file.

Web Intents in a nutshell

The core concept of Web Intents is to permit lightweight IPC messaging between web applications during a browsing session, in order to cut down on interruptions to the user. Many people seem to find the nomenclature confusing, but in the project's parlance, an "intent" is an action that a user wants to perform — in other words, a function. The most frequently used example is a photo gallery site opening up an image in a separate photo editing site when the user clicks "Edit." In this scenario, the gallery application would call the EDIT intent on the image data:

    var intent = new Intent(Intent.EDIT, ‘image/png’, getImageDataURI());
    window.navigator.startActivity(intent, loadEditedImage);

    <intent
    action=”http://webintents.org/edit”
    type=”image/*”
    />

It is the browser's responsibility to notice the <intent> declarations, keep track of them, and provide some sort of UI for the user to select which application gets called when the "Edit" button is clicked. On the click, the browser clones the image object in memory, loads the editing web application of choice, and hands over the object for editing.

The specification lists seven default intents: discover, share, edit, view, pick, subscribe, and save. Each intent is intentionally broad; the action can have different meanings depending on the data type. For example, the share intent is discussed both for link-sharing (where it might replace the NASCAR-array of social media buttons) and for photo sharing (which in practice has fewer options than link sharing, plus a very different workflow). The Web Intents blog discusses many more, however, including calling a remote-printing service and calling a site-wide search using the browser's default search engine.

Open questions

On the discussion list and blog, the developers make it clear that despite the defaults intents list, they do not intend to strictly define a fixed set of acceptable intents. That position has led to quite a few questions about what exactly should be handled by an intent and what should not. One of the recurring questions is why an intent — which can pass any data structure from one browser frame to another — is better than other, existing methods for cross-site communication, such as postMessage, which do not require defining new HTML elements and DOM properties. Mozilla's Ben Adida wrote that combining postMessage and a small set of URI schemes (such as share://) would solve the same problems in a simpler fashion.

The generally cited answer is that Web Intents implement simple, generic operations between mutually-unknown web applications, while postMessage is used in full-blown APIs between sites that are designed beforehand to work together. But there is not a bright line distinction between the two use cases. In a lengthy thread from February, developer Greg Billock said "the problem web intents is trying to solve is more structured: 'how can web applications participate in fulfilling a single user-oriented task.'" But later in the same message, he says that Web Intents can be used to create generic message channels, and that "we're explicitly in favor of allowing generic connection establishment to exist alongside those more structured intents. "

John J Barton countered:

Barton also argued that the Web Intents specification is too vague to be implemented, and in particular leaves unaddressed the method by which a user would accumulate a set of options to handle any specific intent. In the image-editing example, for instance, when the user clicks "Edit" there must already be one or more third-party image editing web applications registered to handle the EDIT intent, or else browser cannot complete the command. "So you are counting on users to go around building up a list of useful services before they need them?"

Billock replied that there were three possibilities. First, that search engines will index sites that offer intents and build up registries of compatible applications to suggest. Second, that web application stores like the Chrome Web Store will highlight the intents of their listed applications. Or third, that the intent-"consuming" sites (such as the image gallery site in our example) will simply include links to compliant services that they recommend.

Mozilla's Web Activities

Last year, there were reports that Web Intents and Mozilla's similar Web Activities project would join forces. But that relationship was never formalized — in fact it may not ever have progressed beyond conversations between individual developers — and as of today, Mozilla seems to have dropped out of the Web Intents discussion. There is not a public position statement from Mozilla, but work on Web Activities is again progressing independently, spearheaded by the Mozilla WebAPI team that also tackles installable web apps and Boot-to-Gecko (B2G).

Mounir Lamouri posted a draft Web Activities specification in March, and updated it on May 23. The Web Activities proposal is narrower in scope than Web Intents. It discusses just three actions (share, edit, and pick), but it, too, asserts that any non-empty string should be considered a valid action. Nevertheless, in his proposal, Lamouri argues that Web Intents goes too far by trying to cover two-way, ongoing messaging between applications (citing the example Web Intents use case for remote-controlled video playback). That degree of interoperability should be handled by the site-specific APIs, he said, not be managed by the general-purpose Activities/Intents mechanism.

In Lamouri's proposal, an application declares its ability to handle a Web Activity with an activities {} property in its manifest file (rather than with a new HTML tag), a choice that the other Mozilla developers seem to agree on. The current Web Activities code — which is experimental and pre-dates the draft proposal — uses postMessage but it, too, is in an early stage of development. Lamouri said in an email that Mozilla's plan is to present Web Activities to the W3C Web Applications Working Group when it is finalized.

Ultimately the bigger story on Web Activities is not the specifics of the proposal, but the fact that Mozilla does not seem interested in collaborating with Google on Web Intents. It is noteworthy that Web Activities now falls under the purview of the WebAPI team, and the discussion around it revolves around B2G. B2G clearly needs an IPC mechanism compatible with its HTML5 application framework — and regardless of how trivial the example gallery-to-image-editor use cases seems on a desktop browser, it is also required by B2G's built-in camera and image editor.

Strangely silent on the Web Intents proposal is Apple, whose own Safari browser is also based on WebKit. Perhaps the company is just sitting out until the unanswered questions about the scope and implementation details get solid answers, but perhaps it sees Web Intents as a mechanism that interests Google only for use in its Apple-competing products like ChromeOS. Whatever the companies' underlying motivations are, users still have yet to put any of the competing options to the test on real world applications. With Web Intents in Chrome 19 and WebKit, at least they may soon get the opportunity.

Comments (4 posted)

Quotes of the week

Comments (none posted)

libgit2 v0.17.0

Full Story (comments: none)

LLVM 3.1 released

Full Story (comments: none)

Nmap 6 released

Full Story (comments: 1)

ownCloud 4 released

Comments (15 posted)

Perl 5.16.0 released

Full Story (comments: 9)

Announcing printerd

Comments (98 posted)

Development newsletters from the last week

• Caml Weekly News (May 22)
• What's cooking in git.git (May 20)
• Perl Weekly (May 20)
• PostgreSQL Weekly News (May 20)
• Ruby Weekly (May 17)
• Tahoe-LAFS Weekly News (May 19)

Comments (none posted)

A scientific basis for Open Source Software

Comments (49 posted)

Google wins patent case against Oracle

Comments (21 posted)

The Ada Initiative granted tax-exempt status in the U.S.

Comments (none posted)

The Make Play Live Partner Network (KDE.news)

Comments (none posted)

Simon Phipps is the new OSI President (The H)

Comments (none posted)

openSUSE Conference 2012 CfP

Comments (none posted)

Events: May 24, 2012 to July 23, 2012

If your event does not appear here, please tell us about it.