LWN: Comments on "GUADEC: porting GNOME to Android"

GUADEC: porting GNOME to Android

nilesh — Sat, 25 Aug 2012 14:02:05 +0000

Isn't Ubuntu for Android taking care of a lot of the low-level stuff that could be useful here as well? Granted, that's work in progress and not yet available, but still there could some code-sharing possible here.

GUADEC: porting GNOME to Android

smcv — Sun, 19 Aug 2012 11:14:54 +0000

> At least autotools insists on running programs to do the feature
> detection which tends to be a nonstarter for cross-compiling use cases

It is possible for a configure.ac to require this, but that feature is relatively rarely-used (most Autoconf checks only require preprocessing, compiling or sometimes linking a program for the embedded architecture, not actually running it).

When a check can't be done without running a program for the embedded architecture (e.g. dbus checks for support for Linux/Solaris-style abstract Unix sockets), in a properly written configure.ac it's possible to tell configure what the result will be, which stops it from actually trying the check. For instance, when building for Android you know that the kernel does support abstract Unix sockets, and there's a configure option to say so.

GUADEC: porting GNOME to Android

alankila — Sat, 18 Aug 2012 21:16:40 +0000

Oh, some DSP probably is. The sort of thing I have in mind is not. I am not aware of any mediaplayer-based sound streams that would not touch the CPU, although as far as I know phone conversations occur without the main CPU being able to interface with either of those streams.

In any case, Android vendors can provide vendor-specific implementations of AudioEffects, and I suppose it might be possible to hardware-accelerate more of the AudioFlinger API. There's a lot of maybes to this, because last time I looked there was but one digital stream that everything got mixed to, and whatever "DSP" was involved was concerned entirely with the mechanics of generating analog audio from that single PCM data.

So there's multiple kinds of things called "DSP" here. The fact that custom DSP plugins are not possible is obvious given that they must be loaded to a root-owned process implemented in C which must not crash and must be as realtime as possible. So random code, or offloadng processing by some IPC approach is probably out.

GUADEC: porting GNOME to Android

Cyberax — Sat, 18 Aug 2012 18:05:04 +0000

> For security reasons, random DSP plugins can't be done on Android either,
Nope. DSP is often done in _hardware_ on Android (for instance, on Galaxy S phones) and sometimes sound stream doesn't even touch the CPU. So that's the reason for non-dynamic AF plugins.

GUADEC: porting GNOME to Android

alankila — Sat, 18 Aug 2012 17:56:31 +0000

1. At least autotools insists on running programs to do the feature detection which tends to be a nonstarter for cross-compiling use cases. In any case, Android's notion of the world is monolithic so there's but one configuration, one compiler, all libraries provided by the project itself if you need them, etc. So there's no features to detect and/or configure. Makefiles, though support if statements, so they are not entirely static in nature.

2. Lower latency, power usage, and freedom of sample rate are not useless features, and favor PA -- at least in some use cases. But until it has a comparable DSP story, it is inferior in other practical ways. I'm just bringing this out every time PA gets mentioned against AF so that people would keep in mind that PA is not a superior replacement in every way currently. That article you linked also mentions the OpenSL effects because I pointed them out last time too.

3. Okay. I suppose it ought to work for the basic use cases. I guess I've been really unlucky. No points for planning on distribute a modified android, though. That's precisely the sort of fragmentation Android doesn't need... ;-(

4. Does this imply that Wayland doing android-like UI is better at the job than SurfaceFlinger? Or is this just a general comparison against X? Of particular note is the fact that android 4 apparently managed to add hardware accelerated UI composition where SF apparently composes the application's UI by overlapping various buffers. It sounds fairly optimal already...

GUADEC: porting GNOME to Android

alankila — Sat, 18 Aug 2012 17:35:51 +0000

Last time I looked, the plugins couldn't be loaded dynamically or configured remotely. Maybe this has changed, but it's far cry from the productized experience that AF offers with OpenSL.

For security reasons, random DSP plugins can't be done on Android either, but the system must at boot-up time declare static list of files to be used for such purposes. But all ROM modders can change this list, so there's that. Afterwards, applications that wish to have their audio stream managed publish their session id to the wider system to work with, and DSP-related applications hear these broadcasts and can use RPC shims to connect to the main mediaserver process to add and remove these effects on a per-session basis.

It is obvious that it takes quite some effort to get from "can be done in principle" to something that can be distributed to end users. My belief is that PA's DSP story is still in the "can be done in principle" stage...

GUADEC: porting GNOME to Android

swetland — Sat, 18 Aug 2012 00:04:50 +0000

Is there any interest in just getting GTK+ (or X if you want to go lower level) working on top of SurfaceFlinger surfaces and arrange for the appropriate libraries to be built in an NDK-compatible way so one could port existing apps such that they'd run on any Android system from Gingerbread forward (which is quite a lot of devices in the field)?

I may be misunderstanding the goals, but it would seem like there are simpler paths to getting existing Linux apps running on Android than replacing big chunks of platform internals...

GUADEC: porting GNOME to Android

daniels — Fri, 17 Aug 2012 22:56:24 +0000

Nate, not Jake. Sorry!

GUADEC: porting GNOME to Android

daniels — Fri, 17 Aug 2012 22:24:16 +0000

You seem pretty vehemently opposed to all of this, so I don't think there's much/any point trying to convince you, but for the record ...

1) Static Makefiles don't support feature detection and configuration in the same way autotools do. Obviously they're not perfect, but they're a damn sight better than just writing plain static Makefiles. Converting the build system of every single project we've been working with is obviously a non-starter for any viable project.

2) This is completely possible with PulseAudio, and many vendors who've deployed it have written their own custom plugins to process streams, some extremely complex, and all without any need for custom knowledge on behalf of the applications. It also provides a number of very real benefits such as drastically reduced latency from source to hardware, and strong power savings. You can read more about this here: http://arunraghavan.net/2012/01/pulseaudio-vs-audioflinge...

3) StageFright has a number of issues, and is not particularly well-suited to more than just basic media playback (and even then ...). GStreamer, on the other hand, has been deployed in all major distributions for years, as well as part of uncountable devices (both consumer/gadget and enterprise), some of which have some hugely taxing requirements. I'm sorry to hear that it anecdotally doesn't work for you on OS X, but it certainly seems to work very well for tens of millions of people. In our Android port, it can indeed be used as part of the base system, because we're developing a branch of AOSP to use as a base for integration, rather than something you can just install from the Google Play store.

4) Wayland isn't, as you note, 1.0 yet (although it is 0.95, with a frozen core protocol), but it's something we're very interested in, as you can see from its commit stats. We think it has a lot to offer in terms of fluid and beautiful UIs which people particularly want in the mobile space, as well as media presentation. We're getting a lot of interest from both hardware vendors and their customers, so a port definitely seemed prudent.

These are all proven, solid technologies (with the exception of Wayland, which is obviously still maturing), which people have deployed and want to continue deploying on a wide range of hardware. Hardware vendors are now turning their attention to Android and only doing base enablement for the Android stack, and the number of hardware vendors who actually care about getting a proper Linux stack up on their hardware is small and diminishing every day. We don't want Android to become a completely separate silo to generic Linux, so we're trying both to offer people with Android-only hardware a much better stack, and also to use the interest from Android deployments to keep that stack more relevant so it can continue to grow and improve for everyone.

It would be nice if that weren't the reality, but what can you do?

Most of these technologies will never become mainstream Android, partially because of licensing concerns, but also partially because the Android development model is based around being almost entirely in-house within Google, rather than the traditional inclusive open source model.

Anyway, as you say, none of this will affect you in any way, so no need to be worried about it. :)

(Obviously I'm speaking for myself and not necessarily for Collabora here. One disclaimer from my brief presentation that Jake left out of the article was that I wasn't actually working on the Wayland port: it was all done by Pekka 'pq' Paalanen, but unfortunately he couldn't make it to GUADEC, so I spoke on his behalf.)

GUADEC: porting GNOME to Android

Cyberax — Fri, 17 Aug 2012 19:47:38 +0000

PulseAudio certainly supports system-wide and per-stream effects. It's just that nobody has written the required plugins.

GUADEC: porting GNOME to Android

alankila — Fri, 17 Aug 2012 18:37:00 +0000

Sounds like veritable nightmare combination of ideas to me. In literally every way, it is going to make Android worse. I can't even take this article entirely seriously, but let it be said that I couldn't personally be in greater disagreement about everything proposed here.

1. Autotools could be back. Android was fine without it. In general GNU makefiles are awesome and can do lots of things -- as a case in point, Android can build everything with just GNU makefiles. Can't other people just start writing those instead, and rather drop autotools entirely? If you have taken a look at Android.mk files, they tend to just list the module name, the source files, and some boilerplate that tends to be very common in all Android.mk files. Is this really too much to maintain?

2. PulseAudio. Despite the popular theory of it being superior, PA doesn't support OpenSL (audio effects on streams). Until it does, I am going to be bitterly opposed to any hint of it replacing AF. Android is one of the very few, if not the only, Linux-based systems that can do digital sound processing in user friendly way that anybody can use. The sound-producing application doesn't need to be aware of effects being applied, and nice GUIs to do it have be designed.

3. GStreamer. Android already has a media framework. I think gstreamer is responsible for videos playing without sound on OS X. I know this is an anecdotal punch under the belt, but IMHO, gstreamer has never worked properly for me. On Android, it can't be plausibly be distributed as common component to be used by multiple applications, anyway, so unless you like multi-megabyte application bundles, GStreamer remains a complete non-starter, regardless of its merits of faults.

4. Wayland. It's not vaporware, but also not ready for prime-time any time soon. The fact this plan requires complicated interaction with random vendor-provided GPU drivers that are only built for use with SurfaceFlinger makes me think that there's no hope with trying to make multiple graphics systems work simultaneously on one screen. What exactly is the story for needing Wayland at all? For instance, if GTK+ applications must run on Android, why can't GTK+ talk with SurfaceFlinger?

I know what I write here is going to come across as needlessly harsh, but the common theme I read here is "more fragmentation for android". It's not like Android wants or needs any of this. And if people seriously want to replace AF or SF with PulseAudio or Wayland, surely it would make most sense to talk with Google's Android team about it? The only consolation I have is that I can't see any of this work going anywhere, so it can be safely ignored.

GUADEC: porting GNOME to Android

Ford_Prefect — Fri, 17 Aug 2012 02:58:34 +0000

This is just how the ARM world works (at least for now). Nevertheless CyanogenMod _does_ run on a very large number of devices. Heck, hardware manufacturers have even become a lot friendlier, making it possible to unlock bootloaders and actually supplying hardware to devs to help the port.

I disagree with you about there not being a point in try to make a phone/tablet version -- we don't have to target every device, just as many as we can (and we can happily just aim for Google's reference devices to start with). More devices will end up being supported by the community if we provide a compelling reason to use our platform.

GUADEC: porting GNOME to Android

jmorris42 — Thu, 16 Aug 2012 19:01:49 +0000

The problem is there is no easy way to distribute new software to Android devices. Heck, just getting an updated Android is a crapshoot unless you have a very popular device. Ask the cyanogen mod guys.

With a PC we have a open and understood boot process and fairly standardized hardware. Removing the OEM preloaded OS and installing a free OS on a PC is therefore fairly easy. Each Android device has to be undertaken as a new port. Without even a plan to crack that that problem there isn't much point trying to make a tablet or phone version of any free OS.