See, NOW it makes sense... [LWN.net]

See, NOW it makes sense...

Posted Feb 4, 2010 11:36 UTC (Thu) by kragil (guest, #34373) [Link] (1 responses)

That answer just gave me an idea:

How about giving contributors not employed by Google or Open Handset members
a free Google phone once they have a certain number of patches in the
Android stack.

It is easy to do (just grep some SCM output, or just put "providing proof"
burden on the applicants) and the number of patches needed can be set so
that certain budget limits are not exceeded.

Kernel hackers would certainly be ledgeable and if they bother to apply for
a phone they might actually contribute patches.

Disclaimer: I certainly wouldn't qualify.

See, NOW it makes sense...

Posted Feb 4, 2010 14:12 UTC (Thu) by gregkh (subscriber, #8) [Link]

Google has been very generous in giving out free phones to lots of kernel developers in the past, so this isn't really an issue.

See, NOW it makes sense...

Posted Feb 4, 2010 14:16 UTC (Thu) by mjg59 (subscriber, #23239) [Link] (7 responses)

Like I said, spending time on this would be conditional on some assurance that it wouldn't just be rejected out of hand. I'm reasonably sure that this can be implemented without any changes to applications being required, but it may need some (fairly minor) alterations to the underlying Android stack.

See, NOW it makes sense...

Posted Feb 5, 2010 19:38 UTC (Fri) by jejb (subscriber, #6654) [Link]

> Like I said, spending time on this would be conditional on some assurance
> that it wouldn't just be rejected out of hand. I'm reasonably sure that
> this can be implemented without any changes to applications being
> required, but it may need some (fairly minor) alterations to the
> underlying Android stack.

If google wishes to offer fair terms for this type of challenge, count me in. I've already done quite a bit of android hacking with the G1 Qualcomm gave me. It looks to me like the move off wakelocks (and possibly binders) can be contained to some small glue changes in bionic and some of the java classes.

I'd suggest reasonable terms be that if we get this working acceptably (meaning comparable battery life to the existing 2.6.29 implementation), Google would commit to moving to the upstream kernel we produce and integrating the necessary changes (whether ours or rewritten ones) into the android user code.

See, NOW it makes sense...

Posted Feb 7, 2010 9:11 UTC (Sun) by swetland (guest, #63414) [Link] (5 responses)

I'd love some assurance that *our* patches wouldn't just be rejected out of hand too.

One thing we are working on doing is getting the external open tree to the point where it's trivial for somebody to check it out and do a full from-source build for Nexus One (which is completely unlockable and reflashable, as shipped from the factory). This then gives everyone some common ground to test patches against, and perhaps measure power consumption of various builds across the normal range of testcases.

At the moment the remaining issues are some build system cleanup, and sorting out redistribution of the handful of userspace proprietary binary modules (roughly 10 libraries/binaries -- fewer, if I'm reading the wiki right, than in a N900).

I can't promise you that we'd absolutely take a change -- it'd have to be verified to actually be better (or at least no worse), across the board, on power management, and that there were no regressions in stability or correctness of any driver. We spent quite a lot of time measuring power usage both in lab and field trial environments -- changes that fundamentally alter how this works get a lot of scrutiny.

See, NOW it makes sense...

Posted Feb 7, 2010 19:07 UTC (Sun) by mjg59 (subscriber, #23239) [Link] (2 responses)

So from my perspective, the three major sticking points are wakelocks, the early suspend/late resume code and the binder. We can argue over whether the binder code strictly needs to be in the kernel, but it's not something that has any real influence over the platform code. If we were in a world where the only significant difference between the mainline kernel and the Android one was binder support, I think that'd be the least of our problems.

The early suspend infrastructure is more interesting. I think it's fair to say that it's a somewhat hacky solution to the problem (take msm_fb - early_suspend and earlier_suspend?), but the problem it's solving is certainly legitimate. Rafael's been working on a mechanism for providing a power management dependency tree that's orthogonal to the device tree. I'd hope that this would provide the necessary functionality that we could do without early suspend, but it'd be great to talk about that.

Wakelocks are obviously the most controversial aspect of this. I think the main problem is that there's never been a terribly convincing answer as to why wakelocks are the appropriate solution to the problem. The upstream omap tree's approach has been to implement core suspend as a cpuidle state with appropriate latency restrictions. That fits in nicely with how other architectures manage this kind of functionality, so I think the onus is really on Google to explain why wakelocks provide more functionality than something that makes use of existing kernel code.

I appreciate that the "Please throw all of this out and rewrite it from scratch" attitude is a problem, and I'd entirely understand if Google don't feel that the work required to rework all of this in a way that upstream approves of would be justifiable. On the other hand, Android's absence from various forums (the collaboration summit, LPC, Linuxcon, the power management mini-summits and so on) makes it much harder for us to appreciate why this difference of opinion exists. It's true that the initial response to wakelocks was pretty universally negative without a lot of effort being spent on figuring out alternative solutions that would be mutually acceptable, and I'm certainly sorry that it happened that way - but at this point I suspect that we've been largely talking past each other for the best part of a year, and it would be good to spend some time figuring out why we have this fundamental disagreement before having another round of acrimonious debate culminating in another pile of code that goes nowhere.

See, NOW it makes sense...

Posted Feb 8, 2010 5:56 UTC (Mon) by swetland (guest, #63414) [Link] (1 responses)

Yeah, I consider the binder a side-issue. In theory one could replace it entirely, if one were willing to write a bunch of userspace code that accomplished everything userspace needs from it, etc, etc, and it is self-contained and no other drivers depend on it.

I thought we had actually explained a number of situations where wakelocks provide value beyond existing solutions -- I'll have to dig through the linux-pm threads and try to extract some stuff from there. One thing we're looking to do is put together a clean writeup of the "why" behind all this as a basis for discussion.

I don't see how retention from idle (which we do as well) fully solves the problem -- since you can easily have situations, in an environment where you can't control every process/thread (300+ threads in a typical android system at boot -- yeah, kinda crazy, but it is what it is). Going to suspend means we don't need to worry about non-optimal software with polling/periodic behaviors causing the CPU to wake up more often than ideal. Also, you still have situations in which you *can't* go to full retention in idle because of a driver being busy (which is what idlelocks -- just another class of wakelock) are about.

I think it would be extremely valuable to get some folks together somewhere and try to hash some of this out. In general we are not opposed to slightly different ways of doing things as long as we accomplish what we need to accomplish power-wise. I mean, Arve's already respun the wakelock API (now suspend_blocker, no?) several times in response to various feedback.

I'd love to engage in a forum where the goal is to solve common problems rather than assign blame -- Greg's blogpost implying that he's giving a CELF keynote about how we suck does not create much enthusiasm on our side for attending, for example.

We know that some people would have been happier if we worked out a design for all of this before 1.0 shipped, but that didn't happen, and arguing about how practical it would have been or not doesn't really gain anybody anything. Figuring out how to collaborate on things so that future issues get sorted out earlier and more smoothly seems entirely worthwhile though.

So when/where is the next time/place that we could meet up with the other interest parties on the linux-pm front and talk about these things?

See, NOW it makes sense...

Posted Feb 8, 2010 6:34 UTC (Mon) by mjg59 (subscriber, #23239) [Link]

Right, it's absolutely true that the wakelock approach simplifies the userspace implementation,
but we could take a different view of things - at the most aggressive level, you could keep a
userspace wakelock implementation and use the uswsusp interface to fire off the process
freezer when all of them are released. All we'd need then would be a mechanism to provide a
list of PIDs that you don't want frozen.

The drivers side of thing is more interesting, and I think that's something that could potentially
be implemented using Rafael's runtime suspend work. As long as the dependencies are
expressed (and I admit that doing so is problematic right now), a busy driver can do a
pm_runtime_get() and then a pm_runtime_put() when it's idle again. Letting that information flow
up the tree could then allow the platform code to inhibit entry to the deeper states. TI seem to
do this in a more hacky way by simply using the systemwide busmastering flag, which cpuidle will
then use to limit idle to wfi-type states. It's not elegant, but it works.

The next linux pm summit is planned for Boston in August, which is kind of far away. There's the
collab summit in SF in April which isn't generally a highly technical conference but is co-hosted
with CELF this year. I suspect we could probably sort something out for that, if there's interest,
and it's the kind of thing that the Linux Foundation would probably love to make happen.

See, NOW it makes sense...

Posted Feb 7, 2010 23:54 UTC (Sun) by kragil (guest, #34373) [Link] (1 responses)

I see another bug here .. why aren't all Googlers automatically LWN
subscribers? Do it like Canonical et al.

Chris? Fix that :)

Our editor is really cheap for big corps that have revenues like MS has
profits.

See, NOW it makes sense...

Posted Feb 8, 2010 5:59 UTC (Mon) by cdibona (guest, #13739) [Link]

Any googler that wants an account can have one. We pay for it out of our
budget. Been like that for a number of years.

See, NOW it makes sense...

Posted Feb 5, 2010 19:43 UTC (Fri) by jejb (subscriber, #6654) [Link]

OK, so to move this to a positive tone, We (that's me, mjg59 and any other developers who join) will accept a challenge to move android to the vanilla kernel:

http://lwn.net/Articles/373173/

Are our terms acceptable, or would you like to add other conditions?