LWN: Comments on "Bringing Android closer to the mainline"

memfd_create()

meuh — Wed, 01 Aug 2018 11:25:17 +0000

Yup -- that was the main limitation (not being able to get an fd to pass to another process for mapping) that ashmem was working around.

One could now use memfd_create(), which is available since kernel 3.17.

ashmem

niemshchanchani — Thu, 24 May 2012 08:20:50 +0000

i'm trying to share memory between android and chrooted ubuntu, I know I can mount bind /dev/ashmem to the ubuntu side , and i have tried out sockets between android and ubuntu and it works . Now the question is how do i pass the fd from android to ubuntu side?

will simply passing the the fd and then mmaping work? ( i think not, but haven't tried it )

or will i need to extend something in ashmem driver ( user / kernel side) to do that? If someone has already done / knows something abt this can you give me a starting point?

Wakelocks

jimmyflip — Mon, 19 Mar 2012 14:14:07 +0000

It's hard to imagine anyone doing even a small amount of Android kernel development wouldn't use 'fastboot boot' to test their kernel (assuming they have a capable bootloader). Any problems, just reboot.

Bringing Android closer to the mainline

zengtm — Tue, 24 Jan 2012 03:43:59 +0000

Just saw this email (1) today. It is a confirmed "kill": pmem is out. It is probably ION that may get mainlined (2)
(1)
http://lists.linuxfoundation.org/pipermail/ce-android-mai...

(2) http://lists.linuxfoundation.org/pipermail/ce-android-mai...

Bringing Android closer to the mainline

navaati — Fri, 06 Jan 2012 19:14:18 +0000

The Neo Freerunner does the same way

ashmem

kleptog — Sun, 01 Jan 2012 16:16:54 +0000

SysV shared memory already has this. Simply set the "delete" flag on the segment as soon as it's created and as soon and the last program detaches it it's automatically deleted.

Bringing Android closer to the mainline

mjg59 — Sat, 24 Dec 2011 21:44:38 +0000

Eh. I don't entirely buy that - most of Nokia's work (now obviously less relevant than it was) is pretty well aligned with upstream, and despite the change in course I think it's hard to argue that they failed to ship product.

The main reason (from my perspective) that Android is more attractive here is that it moves a lot of the responsibility away from app programmers - they can ignore a bunch of power saving and the OS looks after them. The upside of that is that it's much harder for a poorly written application to interfere with your battery life - the downside is that it makes it much easer to have a higher idle (but not asleep) power consumption. But making compromises that favour app developers results in a larger number of applications, and that makes it a more attractive platform for everyone.

Android may actually be one of the first good examples of Worse is Better in the Linux kernel sphere. The MIT people thought UNIX's signal handling code was awful. The Linux people (mostly) think wakelocks are awful. No matter how much we may dislike it on a technical front, I think the wakelock approach is sufficiently simple that it's going to outcompete other solutions.

Bringing Android closer to the mainline

augustz — Sat, 24 Dec 2011 21:22:55 +0000

I thought Arve Hjønnevåg tried to upstream wakelocks something like three years ago?

http://lists.linux-foundation.org/pipermail/linux-pm/2009...

If I remember it didn't go well (horribly).

Google seems to have managed the engineering on these interfaces such that they have shipped 200+ million units. They are not the only one trying to base a phone platform on linux, but do seem to have made the necessary trade-offs to actually ship product.

The time to ship pressure in the embedded space does seem to lend itself to more hacky results than mainline. I'm looking forward to seeing the convergence :) Also, as the pool of embedded folks in the mainline space grows, the feedback to new contributors from embedded may improve.

Good luck to everyone!

Wakelocks

ndye — Sat, 24 Dec 2011 15:42:13 +0000

. . . then pop up a widget saying "new music and viruses available - which should I play?" and wait . . .

First, thanks for the well-toned humor.

And in the midst of a non-childish conversation . . . made my day!

So long, Merry Christmas, and thanks for all the fish!

Wakelocks

Cyberax — Sat, 24 Dec 2011 12:30:56 +0000

You can use kexec, but I don't bother. On Galaxy S phones it's always possible to drop into "download mode" and just restore the old kernel image.

It's _always_ possible even if your firmware is totally corrupted and even battery controller is dead. In some cases it requires a magic jig, though (a 101kOhm resistor shorting two USB lines).

Bringing Android closer to the mainline

meyert — Sat, 24 Dec 2011 12:16:28 +0000

"Bringing the mainline closer to Android"

Wakelocks

mjg59 — Sat, 24 Dec 2011 11:50:20 +0000

If you hit problems you just drop back to recovery and flash the old kernel back. Alternatively, do your testing using fastboot to push the kernel and ramdisk over USB and don't touch your flash at all.

Wakelocks

dlang — Sat, 24 Dec 2011 06:20:57 +0000

the fact that I can't easily use the features that are in a kernel.org kernel with android userspace is one big barrier.

even if I am trying to compile my own android kernel, there doesn't seem to be any way of installing it on a device in a way that will let me fall back to the older kernel if I run into problems with the new one (the equivalent to the old lilo -R option)

the integration of things into the mainline will greatly help the first problem, but I still don't know a good solution for the second one.

Wakelocks

neilbrown — Fri, 23 Dec 2011 23:25:44 +0000

I don't quite follow - what exactly is the barrier that you see to experimentation?

Wakelocks

dlang — Fri, 23 Dec 2011 23:14:23 +0000

the problem is that userspace wakelocks don't solve the problem you are presenting either.

at least not unless every app can take them, but that's not what happens in android, only 'trusted' apps are allowed to take the wakelock.

in android they still use timeouts for most of userspace consuming events, and sidestep a huge amount of it by having the power controller have a wakelock for the entire time the screen backlight is on so that you don't have to scatter wakelocks through every app.

there are many of us who think that there are better ways of dealing with this userspace problem, but the problems of doing our own kernels makes it hard to experiment.

Wakelocks

neilbrown — Fri, 23 Dec 2011 22:10:24 +0000

The "token" is a "struct wakeup_source". You can embed these where ever you like. Alternatively you can use a string name and "wakeup_source_register" will create on for you.

I don't think the uevent code would be changed.

I think for the sd card case there would be two wakeup sources - one for the MMC interface and one for the block device. So on card-detect the MMC interface activates its wakeup_source. It then inspects the card, figures out that it is a storage device, calls mmc_blk_probe, then releases its wakeup_source. mmc_blk_probe would activate its wakeup_source, do whatever it does, ultimately call add_disk (which will make the uevent visible to userspace) and then release the mmc_blk wakeup_source. So any driver in a wakeup chain needs a wakeup_source and needs to activate/deactivate as appropriate.

I don't know the network code very well so I'm guessing more than usual here, but I think there is an 'skbuf' that travels from device to socket queue. I imagine a single system-wide wakeup_source and a flag in the skbuf to say if it is active (a wakeup_source has a counter and so can be shared). Device sets it when creating an skbuf if wake-up is enabled and socket code clears it when it is added to a queue (or when the skbuf is discarded).

A network device might want a separate wakeup source if there is a gap between the interrupt happening and the packet being read from the network device.

But of course the devil is in the detail, and adding anything to fast-paths in the network code is a not an option so we would need to be very careful.

One step at a time :-)

Wakelocks

mjg59 — Fri, 23 Dec 2011 21:24:27 +0000

Ok, so you take a wakeup source on card insertion, and then the uevent code has support for releasing the source when an event has been sent. You'd presumably need some sort of token to ensure that the event sent corresponds to the wakeup source. Userspace then attempts another suspend, and calls the prepare() method for the mounting agent. Either that or the confirm() method (or both) verifies that the event queue is empty (presumably by making sure that there's no active fds in a set it's poll()ing on, or something) and then the suspend is attempted.

If an event occurs between confirm() being run but before the suspend is requested, you'd miss that. So the suspend needs to then fail, but that's ok if you've got a token from before you called prepare() and that token's been invalidated by another wakeup event in the intervening time. So it does sound like this could be made race-free, but it does involve adding some extra complexity to the uevent code. That's not an obviously bad idea in any case - it'd be a useful thing to have for avoiding races in the existing suspend path.

Now, how about the network case? It ought to be possible in much the same way (you'd deactivate the source when the packet has been read), but there's a pretty huge distance between an irq from a network device (where you have to activate the source just in case the packet is magic, which you don't yet know) and userspace, and it could get lost at various points in between - so you need to map every path that it could take.

This is all obviously possible, and I think your approach could certainly be made to work. But making it work well is going to involve a lot of code. Not a criticism, just an observation!

This is one of the reasons why a scheduler-based approach is more attractive in many ways - you avoid the kernel timeouts problem without having to add a huge amount of kernel complexity. It obviously does result in a bunch of additional problems on the userspace side, many of which could probably be mitigated with timer slack - but I'm obviously speaking in a purely hypothetical sense, since I haven't found the time to actually do the work necessary to make that workable. You're actually doing the work, so you get to ignore me here :)

Wakelocks

neilbrown — Fri, 23 Dec 2011 21:08:02 +0000

This is how I would handle your SD card example.

The card-detect interrupt activates a wakeup_source which is then deactivated once the insertion is visible to user-space. I'm guessing that is just after the kobject_uevent() call which reports a 'change' or 'add'.

The user-space suspend daemon has a 'prepare' and a 'confirm' stage to requesting suspend. A wakeup_source activation after 'prepare' will cause 'confirm' to fail. Between these two stages it would do something like "udevadm settle". This would ensure that any kobject_uevent had actually been handled.

The udev handler for "sd card insertion" would fsck and mount, scan for viruses and new media files and sends a dbus signal to the UI. Then exit.

The UI would notice the signal, tell the suspend-daemon that it is busy, acknowlege the signal (thus allowing udev handler to exit), and then pop up a widget saying "new music and viruses available - which should I play?" and wait for the user to respond (or timeout).

So: no timeout needed in the kernel or at all until you get to user interaction. Also the user-space side seems entirely generic.

The kernel doesn't need to know when user-space consumes the event, only when the event is visible to user-space.

The suspend-daemon *can* know if user-space has consumed the event because it can ask every user-space subsystem "Have you consumed all your events", and the prepare/commit makes this race-free. It sounds expensive to ask everyone that question but I believe it can be done quiet efficiently. e.g. at start up the subsystem says "here is an fd - you just check it yourself, don't bother me". Then "ask everyone" becomes a single "select" system call.

If you have other examples that you think are more complicated I would love to hear them - I have little practical experience in this area and would love to benefit from the experience of others.

Thanks.

Bringing Android closer to the mainline

zengtm — Fri, 23 Dec 2011 20:52:27 +0000

I thought PMEM is to be phased out over time by Google, in favor of the new Android 4.0.x ION memory manager, as indicated by email from Google kernel engineer. If that is the case, why not include the ION driver as well in drivers/staging/android?

Wakelocks

mjg59 — Fri, 23 Dec 2011 19:57:18 +0000

That's not quite it. The timeouts aren't there because there may be more data coming - the timeouts are there because for most things more complicated than the input case, you don't know whether or not userspace has consumed the event. That means that on Android you have little things like 5 second timeouts in the MMC code purely to handle the case that there's no easy way to detect whether userspace is done dealing with the SD card that you just inserted or not, or (as in your case) how long it'll take some magic packet you received via wifi to bubble all the way up to userspace. It's difficult to see these kernel-level timeouts being acceptable.

You can defer the timeouts to userspace instead, but then your userspace has to be amazingly aware of how everything ties together - rather than having generic userspace, you now need userspace that knows every single wakeup event that the kernel may generate, and whether or not it needs to take a timeout on them. That's more complicated than the existing Android case.

I'm not arguing that the wakeup events code is bad or evil or does nothing to get us closer to a model where Android works, just that right now it only solves the simple cases and doesn't have a clear strategy for solving the difficult ones.

Wakelocks

neilbrown — Fri, 23 Dec 2011 19:42:55 +0000

I won't try to argue elegance - that is in the eye of the beholder. The point really is existence. This exists in mainline Linux today so the response to "we need wakelocks" is actually "we have them".

They haven't been tested heavily and there is no demonstration that they are sufficient for Androids needs so it my mind, that is the next step. I would like to see android running on a mainline kernel (which is nearly possible with what it in staging in -next) and then replace little bits so that it uses mainline-preferred functionality in place of Android-specific functionality and see how it works. Then we could move on from "you should do it like this" arguments - which never work - to "I have done it like this" arguments which tend to hold a lot more water.

Timeout-based wakeup sources are only needed when interacting with an external device (such as a human or a network switch) and in that case they are always needed. e.g. if you receive a wake-on-lan magic-packet you need to assume there will be real data soon but you cannot wait forever so you need a timeout. Similarly a key stroke is usually followed by another so stay awake for it. These sorts of things always need timeouts. You should need no more or fewer timeout-based wakeup sources than you would need equivalent timeouts with Android wakelocks.

Wakelocks

mjg59 — Fri, 23 Dec 2011 12:36:27 +0000

Right, but other than precise implementation details this is still equivalent to wakelocks (the main thing being that userspace has to repeatedly ask the kernel to go to sleep rather than it just happening automatically). It's no more elegant, and if you want it to carry on working in all the cases that wakelocks do then you still need to have timeout-based wakeup sources sprinkled throughout random bits of your code.

Wakelocks

brendan_wright — Fri, 23 Dec 2011 11:27:26 +0000

> this greatly undermines the "this is the only way to make things work" argument.

I don't think anyone here is saying they are the *only* way to "make things work". But they are a working implementation of a feature I think we need ASAP.

Wakelocks

brendan_wright — Fri, 23 Dec 2011 11:15:28 +0000

> Certainly the OS needs to know when it can suspend quickly and userspace apps need to provide that information.

> But the "kernel" doesn't need to be particularly involved in this.

I'm not fussed about how it works, just as long as we're not still arguing about either the need for them, or the implementation details, in two years time! :-/

Wakelocks

dlang — Fri, 23 Dec 2011 04:03:11 +0000

however, taking a wakelock is a privileged operation, so most programs are not allowed to do so.

this greatly undermines the "this is the only way to make things work" argument.

Wakelocks

neilbrown — Fri, 23 Dec 2011 01:52:49 +0000

You missed a part of the sequence.

When user-space hits a key the keyboard driver activates a wakeup-source. This has the effect of causing the suspend request to abort (yes, this can be done race-free). The CPU says awake, the keystroke is responded to, userspace notices that the user is doing something and arranges not to go to sleep just yet (user may still throw phone out the window, but it is probably because they hit the wrong key by mistake).

And as for "do the necessary work": Please see https://lwn.net/Articles/464144/. Just as soon as I can get my new phone suspending reliably I'll be testing this is a real-life situation.

Wakelocks

mjg59 — Fri, 23 Dec 2011 00:56:52 +0000

Userspace decides to suspend. It gets preempted by the user hitting a key, but the task that needs to read the input event is currently blocked on something else so userspace is unaware of this and continues the suspend. The user, who has just hit a key and sees their device turn off anyway, throws their phone out of a window and travels to Nepal to seek a technology-free life.

Wakelocks (or a functional equivalent) are entirely necessary for the Android power management model. Is the Android power management model the best one? I don't think so, but nobody (myself included) has successfully demonstrated otherwise so I think at this point people should either do the necessary work or stop complaining.

Wakelocks

neilbrown — Fri, 23 Dec 2011 00:08:05 +0000

Certainly the OS needs to know when it can suspend quickly and userspace apps need to provide that information.

But the "kernel" doesn't need to be particularly involved in this. User-space apps can talk to user-space suspend manager and when an appropriate time to suspend arrives the user-space suspend manager can initiate a suspend.

wakelocks as a feature are perfectly fine. However they don't need to be a kernel feature.

(well .. to be accurate there does need to be a way for kernel modules such as drivers to hold-off suspend for a while, but we have that - they are called wakeup-sources. Userspace uses a separate mechanism that is more fitting for user-space).

(and to be completely fair - there is a school of thought which says that we shouldn't be using suspend at all and so wakelocks aren't fine no matter where they are implemented. Adherents of this school believe that when a tab isn't visible the animation on it should stop and then when nothing is using CPU the CPU should reduce power usage. It is hard to argue against this in terms of elegance of the design. Whether it is practical is somewhat easier to argue against).

Wakelocks

brendan_wright — Thu, 22 Dec 2011 23:46:27 +0000

Because to save power and battery life, the OS needs to know that it can quickly suspend after user input has ceased even though 5 browser tabs are busy doing constant updates of moving HTML ads. But it also needs to know that it can't susped the CPU when it's busy rendering a 3D scene, or the screen when a movie is playing. Linux should have best in class power managment, and to do that it needs more information from userspace apps.

ashmem

cmccabe — Thu, 22 Dec 2011 18:35:37 +0000

I think we're going to have to agree to disagree. I think passing file descriptors via sockets is elegant.

Wakelocks

dlang — Thu, 22 Dec 2011 17:20:53 +0000

wakelocks are still not needed. It's possible to suspend your linux or windows desktop with programs running without wakelocks, so why would they be needed on android for this case?

ashmem

Cyberax — Thu, 22 Dec 2011 13:02:23 +0000

Different users must be able to share the shared memory segment, so simple mapping doesn't work. Passing file descriptors using sockets might work, but it's ugly.

Wakelocks

etienne — Thu, 22 Dec 2011 11:53:12 +0000

You have still added something about wavelocks in this article:
> Android's interesting approach to application management. In the Android world, applications never choose to exit. They hang around until memory gets tight, at which point kernel starts to kill them off.

So the developpers of highly interactive application may want to consider the case that, even if they want to respond immediately to user requests, if they did not see a user request for the last 20 days, they may enter a "power saving" state by themself - unless that is done, wavelocks seem usefull.

Bringing Android closer to the mainline

swetland — Thu, 22 Dec 2011 04:32:42 +0000

Nope. And we have made changes like that in the past, often as kernel subsystems evolve to provide functionality we'd previously implemented in an ad-hoc manner.

We're not interested in completely gutting and rebuilding the Android userspace, no matter how many times people suggest "oh you would just rewrite x, y, and z and then it would do something almost like w".

We have existing, proven, working code that's been in the field for four plus years and gets the job done. Not breaking the existing system is important. A lot of the proposed "oh just replace x with y" where y is something not-really-the-same or something reimplemented-as-a-userspace-daemon would require quite a lot of work to validate and hunt down regressions. Work that is hard to justify given everything else we need to do to keep moving the platform forward.

That said, working towards unified interfaces and the ability to run Android on "out of the box" mainline kernels is definitely a goal we're interested in.

Bringing Android closer to the mainline

tbird20d — Wed, 21 Dec 2011 18:11:11 +0000

Is it out of the question for Android to transition some kernel interfaces to more generic ones in the future? Most programs end users care about are written either for Dalvik or using NDK and shouldn't care much about wakelocks or logging for example.

See the mailing list for some discussion of this. In general, it is conceivable to transition to more generic interfaces. However, for each one you have to evaluate what the impact is to existing code throughout user space. For example, most programs doing logging are not directly making Linux system calls, and there is likely some flexibility there to change the interface if needed. The question, IMHO, is how much change is really needed.

Wakelocks

corbet — Wed, 21 Dec 2011 14:26:29 +0000

We'll always listen to requests, but, honestly, I've written so much about wakelocks/suspend blockers that I can't really imagine what I could add to that discussion at this point. Nothing really has changed on that front for a while now.

ashmem

swetland — Wed, 21 Dec 2011 12:37:57 +0000

Yup -- that was the main limitation (not being able to get an fd to pass to another process for mapping) that ashmem was working around.

It also (though this feature came later) has functionality which allows a process to indicate ranges of pages which are not needed and may be reclaimed by the kernel in the event of a low memory situation. The idea being that using this to mark caches, etc, gives the kernel the ability to reclaim some pages as an alternative to invoking the low memory killer to obtain more memory in a pinch.

ashmem

josh — Wed, 21 Dec 2011 09:51:33 +0000

Linux already supports anonymous shared memory; you can call mmap with MAP_SHARED|MAP_ANONYMOUS to get an anonymous shared region. Child processes will inherit that region. The only difference: you can't get a file descriptor for the region, and thus you can't pass it around like a file descriptor, such as through a pipe. If ashmem works like a natural extension of that, allowing the association of a file descriptor with the mapping to allow passing it around, then it seems like a very reasonable addition.

In theory, a trusted process could serve the same function in userspace, by using the new cross-memory attach (CMA) mechanism. Processes could ask the trusted process to create a new piece of anonymous memory for them to use, and multiple processes could attach to that memory. (The permission issues would prove difficult to deal with, but not impossible; the trusted process would need to manage a set of cooperating processes with various credentials.) However, I think it makes sense to handle this in a much more natural way through the kernel, by associating a file descriptor with anonymous shared memory.

Bringing Android closer to the mainline

job — Wed, 21 Dec 2011 09:30:07 +0000

/sys/class

ebiederm — Wed, 21 Dec 2011 07:30:46 +0000

Internally classes in sysfs may go away but at this point the /sys/class directory is part of the ABI and so /sys/class can not be practically removed.