LWN.net Weekly Edition for February 19, 2009

How (not) to brick the Android Developer Phone

Your editor's adventure with the Android Developer Phone (ADP1) began just before the end of the year. This phone, remember, has the nearly unique selling point that it is lacking any sort of lockdown feature. It will happily run any software which is fed to it, from the kernel on up. It thus brings the promise of free software to a market which has traditionally gone out of its way to avoid enabling any sort of freedom. It's actually possible to control the software we run on our phones - but only if we buy the right phone.

The path to exercising this freedom is long and poorly documented, though. Eventually somebody will certainly pull together a definitive resource for developers wanting to hack on their Android phones; until then, one is left digging through a long series of web sites and forums (a few of which are listed below). This article will not be that resource, but, hopefully, it can help to point interested people in the right direction. Please note that this article assumes that you have an ADP1 phone; if you have a locked-down G1 you can still participate in all of the fun and games that follow, but you will need to root the phone first.

The first stop, unfortunately, is the decidedly non-free Android SDK. Actually, this package is only truly mandatory for those wanting to build Android applications of their own. But it contains a pre-built version of the Android Debug Bridge (adb) tool, which is essential for working with the phone over a USB connection. With adb, one can connect to a shell running on the phone, move files back and forth, forward network ports to and from the phone, and more. Yes, one can run a shell directly on the device using the terminal emulator application, but life is certainly much easier when one can use a real keyboard.

Note that it may be necessary to either (1) run adb as root, or (2) play with your udev setup to be able to access the phone via USB.

Putting new software onto the phone involves flashing its NVRAM. There are six partitions on the onboard flash:

    dev:    size   erasesize  name
    mtd0: 00040000 00020000 "misc"
    mtd1: 00500000 00020000 "recovery"
    mtd2: 00280000 00020000 "boot"
    mtd3: 04380000 00020000 "system"
    mtd4: 04380000 00020000 "cache"
    mtd5: 04ac0000 00020000 "userdata"

Details about these partitions can be found on this page. Most of them will be fairly obvious in purpose. The "recovery" partition holds a recovery image which can be used to un-brick the phone. In "boot" is the initial system image, while "system" is the root filesystem. Application settings and such go into "userdata". With any luck at all, it should be possible to put a new system onto the phone by flashing only the "boot" and "system" partitions, leaving settings and such in place.

First, though, comes that sweaty-palms moment when one realizes that one is about to overwrite the operating software on an expensive new toy. A fairly nice new toy that, mostly likely, has become an important working tool. The idea of turning this nice device into an expensive brick lacks appeal. It might be different if the second-generation Android devices were available; then, at least, one could rationalize an update disaster as a celestial sign that it's time to get a newer phone. In the absence of such an ulterior motive, your editor stepped back from the brink and pondered ways to recover from a failed update.

[PULL QUOTE: Everybody should be able to run vi on their phone (though emacs appears to be a bit too much to hope for). END QUOTE] One method your editor has seen recommended is to simply make copies of the various /dev/mtd/mtd? devices, then use adb to lift those copies off the phone. The system running on the phone has a rather minimal command set, so this copying must be done using cat and shell redirection operators. This experience gives a quick thrill, as if one were reliving the very earliest days of Unix before advanced commands like cp had been invented, but said thrill is quick indeed. Thereafter, one usually wants to go out and install busybox on the device. After making a few strategic symbolic links, one will have something that looks a lot more like an ordinary Linux shell environment. Everybody should be able to run vi on their phone (though emacs appears to be a bit too much to hope for).

Back to backups: an alternative is to use the nandroid script. With nandroid, a simple command will back up all of the useful partitions on the device in a way which lets them be quickly restored. Unfortunately, though, nandroid will not work with a stock phone. At a minimum, one must install busybox, then make links for commands like nc, tar, and md5sum. Alternatively, one can install the modified recovery image from the amazingly productive "JesusFreke," then back up the phone while it is in recovery mode. Either way, one will, once again, end up with a set of image files containing copies of the phone's flash partitions.

So what does one do with these image files? The key tool here is fastboot, a command-line tool which runs on a Linux-based host system. With fastboot, one can flash one or more partitions to a USB-connected phone, then reboot into the new code. First, though, one must know the secret handshake: power up the phone while holding down the camera button, connect the USB cable, then hit the "back" button until the display reads "fastboot." Needless to say, the manual that came with the ADP1 did not mention this little detail.

Of course, said "manual" is a single slip of paper showing how to insert the battery.

Once one is convinced of one's ability to recover from a disaster, it's time to try to put some new software onto the phone. If you have built the Android platform from source (a process which will be addressed in the next installment), the result will be new "boot" and "system" images which can be flashed to the phone using fastboot. System images built by others can also take that form, but the more common approach is to package the whole thing up into a Zip file. In such cases, the recipe is as follows:

• Using adb, put the update image onto the SD card on the phone (mounted under /sdcard) as update.zip.

• Reboot the phone into recovery mode (secret handshake: hold down the "home" key while booting).

• Press alt l, followed by alt s. Note that, unlike ordinary ADP1/G1 keyboard operation, you need to actually hold down the "alt" key while pressing the associated letter.

This sequence will cause the phone to rewrite its software with the image found in the update.zip file. Once the process is complete, hitting the "home" and "back" buttons together will reboot the phone into the new image.

So, what might one install via this method? The set of modified images provided by JesusFreke are a good place to start. The JFv1.31 image makes a lot of things work more nicely; it includes busybox with a set of useful links, a fancier recovery image with built-in backup capability, a version of su which asks the user for confirmation (and which, thus, should be harder to exploit from an evil application), and more. Also worth noting is that the JF images disable any over-the-air updates. Such updates should not be happening with an ADP1 phone in any case, but, when one has control over one's own phone, there is no reason to allow outside agencies to drop new software into it.

Rather more fun can be had by going to the JFv1.43 image. This version includes an update from Android, which is said to fix a number of small issues and improve battery life. It adds a voice calling capability which was notably lacking in the original Android distribution. Your editor's first attempt, "call home," was turned into "call mom" though; Google does not appear (yet) to have achieved a level of omniscience sufficient to know that those two have not been synonymous for some years now. Also added is a voice search mechanism. But that's not all: this update includes the multitouch functionality that Google left out; the "pinch" gesture now zooms web pages in and out. Other applications have not yet been enhanced to use multitouch, yet.

Arguably even nicer than multitouch at this point is the new "autorotate" setting in the browser. The ADP1 can report its orientation to an application, but almost no applications make use of that information. So, on a stock ADP1, using the browser in landscape mode requires opening the keyboard. With autorotate turned on, the browser senses when the phone has been turned and adjusts the display accordingly. It's one of those little features which should have been there from the outset.

And that, of course, gets to the heart of why an open phone is such a nice thing to have. We're no longer dependent on the manufacturer to get everything right, and we're no longer dependent on wireless carriers to hold off from crippling our devices. We bought the hardware, and we wrote the software. We are well within our rights to change how it all works - even if we want to do something crazy like install Debian over Android. It is unfortunate that, at this time, so few devices afford this kind of freedom; ADP1 and OpenMoko appear to be about the only options. With any luck at all, awareness of the value of this freedom will spread over time, and vendors will find that their customers will settle for nothing less.

Of course, real freedom doesn't stop at the ability to install software images created by others. The next installment in this series will start to delve into the process of generating a new system image from source. Among other things, your editor intends to take the "cupcake" development version - which includes, among other things, the much-requested on-screen keyboard feature - for a spin. Stay tuned.

Resources. Information about working with Android is spread around the net; here are a few useful places your editor has found:

• There is, of course, good information to be found at source.android.com.

• developer.android.com is the source for the software development kit and related information.

• The xda-developers site is a repository for vast amounts of useful - if noisy and slowly-served - information. In particular, the Dream Android development forums seem to be the primary gathering point for people hacking on this platform.

• Some information - notably new pre-built image announcements - can be found on andblogs.net.

Comments (22 posted)

FOSDEM09: RandR 1.3 and multimedia processing extensions for X

At FOSDEM 2009 (Free and Open Source Software Developers' European Meeting) in Brussels, your author attended a number of talks about the state of graphics in Linux. Two of them stood out: Matthias Hopf's talk about RandR 1.3 and Helge Bahmann's work on multimedia processing extensions for X.

RandR 1.3: panning, transformations and properties

Matthias Hopf of SUSE R&D gave an update about RandR 1.3 (Resize and Rotate Extension). So far, RandR 1.2 exposes an interface to dynamically set and query properties such as the displayed and known video modes, the framebuffer size, and attachment of a monitor. However, there are still some important features lacking. For example, querying the state of an output involves output probing, and there is no way for applications to distinguish between the internal panel and an external output, which could be interesting for presentation software. Panning is also lacking, just as displaying in a non-1:1 fashion. And last but not least: the framebuffer size of X is limited to its initial allocation.

RandR 1.3, which is to be released with X Server 1.6, should implement a number of these features. With the new version of the extension, it is finally possible to query the state without output probing. The function RRGetScreenResourcesCurrent is equivalent to RRGetScreenResources but does not use polling. However, you won't get notified of new monitors this way. The xrandr command to query the state of a VGA output would be:

    xrandr --output VGA --current

Other additions are multi-monitor panning and display transformations. When the mouse hits the screen borders, the viewport has to be changed. For a seamless movement without flickering, the graphics driver needs an update. The --panning option of the new xrandr command has three sets of four parameters, as in this example:

    xrandr --output VGA --panning 2000x1200+0+0/2000x1200+0+0/100/100/100/100

The first parameter set is the panning area, the second one is the tracking area, and the third one are the borders. For example, setting the right border to 100 means that panning begins if the user reaches a border of 100 pixels before the right end of the physical screen. The panning area is the area that might be visible on the screen, while the tracking area is the area in which the mouse pointer movements influence the pan. In most circumstances these two are identical.

There are still some conceptual problems to be solved, according to Hopf. He wonders what the combination of a dual-head configuration and panning could mean: should the whole space span when the user reaches the side of the virtual space, or should each physical space pan separately? Or should it be a combination of these two? Xrandr needs an update to accommodate to these possibilities. Another problem is that panning and display transformations don't fit together.

Display transformations in RandR 1.3 make it possible to transform the perspective of the CRTC content. This could be used for rotation, flipping, scaling and keystone correction. Under the hood, the code is using homogeneous coordinate transformations, implemented by a 3-component matrix-vector multiplication. The user has to specify this transformation matrix in the appropriate xrandr command, as in:

    xrandr --output VGA --transform 2,0,0,0,2,0,0,0,1

which scales the image down by a factor of 2. A more pragmatic use of this display transformation would be a keystone correction matrix, which transforms the distorted image of an incorrectly positioned projector to a perfect rectangle. It would seem that there is ample scope for the creation of more user-friendly interfaces to this functionality, though.

Distinguishing between different types of screens can be done in RandR 1.3 with standard properties, such as output and signal types. RandR will require graphics drivers to implement some mandatory properties to claim RandR 1.3 support. Hopf added that "unknown" is a valid value, so initial support is trivial. Two of these mandatory properties are SignalFormat and ConnectorType. The former describes the physical protocol format, such as VGA, TMDS, LVDS, Composite, Composite-PAL, Composite-NTSC, Composite-SECAM, SVideo, Component or DisplayPort. The graphics driver changes this property when the underlying hardware indicates a protocol change, and X clients can change this property to select a protocol. The ConnectorType property is immutable, and can have one of VGA, DVI, DVI-I, DVI-A, DVI-D, HDMI, Panel, TV, TV-Composite, TV-SVideo, TV-Component, TV-SCART, TV-C4 or DisplayPort as its value. A presentation application can use this property to detect unambiguously which is the laptop display and which is the projector display.

Other, non-mandatory properties are SignalProperties, ConnectorNumber, EDID (formerly EDID_DATA, the raw EDID data from the monitor), CompatibilityList and CloneList. Many of these properties haven't been implemented by any driver yet. A final problem that cannot be solved in RandR 1.3 is the framebuffer size limitation. The culprit is the current XAA implementation: XAA calls don't get the pitch as an argument and assume it stays the same for the whole life of the X Server.

Multimedia processing extensions for X

Helge Bahmann, a research assistant at the Technical University of Freiburg in Germany, talked about his experimental multimedia processing extensions for the X Window System. At this moment, multimedia applications either bypass X (e.g. by DRI), or they use X as a video playback service for computed images (e.g. by XVideo). The network transparency for which X is famous fails in both cases. If you want to display video remotely, you have to be able to transmit compressed media data, and you have to synchronize audio and video. For this purpose, Bahmann introduced three new experimental X extensions.

The TIME extension, part of Bahmann's master's thesis in 2002, introduces two new server-side objects: Clocks and Schedules. An X client can start, stop and query the X server's clock. The client also submits commands to the server with execution and expiration timestamps, and the scheduler executes these requests at the appropriate time. This mechanism allows the application to schedule drawing requests (using the RENDER extension). It's important to note the (non-)obligations of the client and server: the X server doesn't guarantee the timely execution of the commands, and the client thus cannot rely on a created state. At the other end, the client can "change its mind": it can retract scheduled commands and it can replace them with completely different commands. Retracting and replacing can fail if the server has already started the execution.

The other extension Bahmann introduced is the AUDIO extension: it implements a SampleBuffer object, which is server-side storage for audio samples, equivalent to pixmaps for images. A PCMContext object can serve as a clock: a client can bind an execution scheduler to it, which allows operations to be executed synchronized to audio playback or capture. This also allows a simple synchronization between audio and video. The AUDIO extension spawns a dedicated real-time thread, but the rest of the X server is completely unaware of the thread. Because of the audio thread, the X server has to be linked to a thread-safe libc.

The TIME and AUDIO extensions are the basic infrastructure for multimedia processing, but as you have to deal with huge amounts of data, this will not work well on low-bandwidth networks. That's why Bahmann introduced a third X extension: COMPRESS, which is actually a misnomer because it uncompresses data. The ImageSequenceDecompressor and AudioSequenceDecompressor can receive and buffer individual compressed JPEG frames, and convert them into an uncompressed representation which can be processed by the X server. The client must submit compressed frame data and hence has to understand the compression format.

Summarizing his talk, Bahmann stressed that his X extensions are conceptually relatively simple and reuse existing X functionality (such as communication, client/resource management and security) without duplicating existing X functionality. A drawback for the programmer is that these multimedia extensions are very low-level and hence not at all easy-to-use. The client application has a big responsibility: it has to understand, parse and partition the media data, to plan ahead, submit compressed data and schedule commands, and to handle synchronization. It also needs to back-track and reschedule commands, for example when the window size changes. Bahmann warns this can be very complex to implement.

Bahmann calls his extensions "experimental" because they work for him but probably require diving into the code if you want to use them. Audio, timing and synchronization basically work, and the protocol part of the compression is finished. However, the backend interface to plug in new decompressors is still in flux. But all in all, it works:

While implementing and thinking about these extensions, Bahmann encountered some deficiencies of the current design of the X server. There's the security problem of audio/image decompressors in a process running with root privileges. Knowing the bad security track record of media players such as VLC and MPlayer, one should be cautious with such complex code running with root privileges. Bahmann is currently auditing the decompressors, and that's the principal reason why there are so few codecs available. Secondly, the compute-intensive decompression operations of the COMPRESS extension may stall the X server. Bahmann suggests to give up the current single-threaded design of the X server, but that is an idea which has not been accepted by the X development community in the past. In the absence of multiple threads, decompression must be handled carefully so as to avoid interfering with other X operations.

Comments (24 posted)

Novell and Red Hat ask community help in patent case

In any legal case, the conventional wisdom is that you should say nothing outside of court. However, in the patent infringement case IP Innovation LLC et al vs. Red Hat Inc. et al, Red Hat and Novell have chosen to ask to the free and open source software (FOSS) community for help in finding prior art — that is, evidence that might disprove the validity of the patents involved. The decision, says Rob Tiller, vice president and assistant general counsel, IP, for Red Hat, is a mixture of practicality, experimentation, and diplomatic relations.

The case was filed in United States District Court for the Eastern District of Texas, Marshall Division, on October 9, 2007. IP Innovation is a subsidiary of Acacia Technologies, which has filed over 239 infringement cases since 2003 — almost three times as many as any other company. The infringement claim alleges that Red Hat's and Novell's GNU/Linux products infringe U.S. Patent Numbers 5,072,412, 5,533,183, and 5,394,521. All three patents involve the use of graphical multiple or virtual workspaces on the desktop, a feature common to most popular desktops, including GNOME, KDE, and Xfce. See this LWN article from 2007 for more information on the complaint.

For the past sixteen months, the case has ground onwards. According to Tiller, the case is currently in the stage known as claim construction, in which the court attempts to define the patents and the terms they use. Now, with deadlines for submitting prior art approaching in March, Red Hat has posted a request for aid from the community on Post-Issue Peer to Patent. Specifically, the request is for prior art from before 25 March, 1986, the year before the patents were filed.

This is not the first time such a request has been made. Last year, Barracuda Networks made a similar request to the community in its case against Trend Micro. Tiller, though, had apparently not heard of this earlier effort.

So far, the request has produced over 20 responses, as well as over 125 beneath the Slashdot link to it. If these responses prove valid, they could be used to discredit the patents on the grounds that they were either unoriginal or too obvious to have been granted, or that the claimed techniques had previously been invented elsewhere — two classic strategies in patent infringement cases.

As it happens, the request is a tactic agreed upon both Novell and Red Hat. Both companies are represented in the case by law firm Gibson, Dunn, and Crutcher, and are coordinating their defenses. "This whole thing is done with Novell's agreement," Tiller says, even though Red Hat was the company that actually file the request on Post-Issue. Jim Lundberg, VP Legal at Novell agrees. "Essentially, it is a joint effort by Novell and Red Hat. Both Red Hat and Novell are working closely together in defending against the patent litigation that was filed."

Neither explained why the request was posted to Post-Issue by Red Hat alone. However, given Novell's unpopularity with some of the community because of its agreements with Microsoft in the last few years, perhaps those involved calculated that a request from Red Hat alone would stand a better chance of receiving useful answers.

Certainly the idea seems to have carefully calculated from every other angle before being implemented. Acknowledging the unusual nature of the request, Tiller says:

In the end, Novell and Red Hat decided that the advantages outweighed the potential hazards. "The principle reason," Lundberg says, "is because of the background and the input that [the community] can provide — the wealth of knowledge that exists out there."

A secondary reason for going public, says Tiller, speaking as a representative of the Red Hat legal department, is that the request is consistent with open source methodology and corporate image. By making the request, he hopes:

Part of that decision, Tiller adds, is to keep the contributions available to the public. "We considered carefully whether we should drop the information in a locked box where no one else could see it" — a possibility that might have prevented IP Innovation from being pre-warned of the evidence that the defendants might use. However, in the end, Tiller says, "we decided that would be contrary to the spirit of free and open source collaboration, that it would be important for people to see what others had done, and what others' insights were".

Still another consideration is that sorting through the replies for prior art will require extra work in the few weeks left before final submissions. But, although replies are still arriving, the preliminary results seem promising. "We believe that they've been helpful," Lundberg says tentatively.

Similarly, Tiller says, "In fact, we think there are likely to be a good deal of antecedents that have a bearing on the claim in these patents, so we expect to see a variety of ideas." At this point, though, he says, "We don't want to inhibit the discussion. We're looking for as many ideas as we can get."

Having already taken such a large and nontraditional step, Tiller and Lundberg seem reluctant to say much more about the case. Although Tiller admits that the Red Hat legal department is keeping other lawyers with FOSS expertise informed, he says:

That said, the defendants in the case sound reasonably optimistic. "We feel very strongly about our ability to prevail in this case," Lundberg says. "With the assistance of the open source community, we think that potential is even stronger."

Echoing these sentiments, Tiller concludes: "We feel we have have good, strong defenses against the infringement claims already, but we really think that this is an opportunity for us to participate with the community, not only in opposing the particular entities that are attacking Linux here, but also in developing a body of experience that will guard both us and the community in the future."

Comments (4 posted)