Weekly Edition
Return to the Announcements page
| |||||||||||||
Greg Kroah-Hartman: Android and the Linux kernel community
Kernel hacker Greg Kroah-Hartman looks at the problems with Android's kernel modifications, which aren't in the mainline—nor headed that way. He does hold out hope that the situation will eventually change, as well as offering his help to get there. "Now branches in the Linux kernel source tree are fine and they happen with every distro release. But this is much worse. Because Google doesn't have their code merged into the mainline, these companies creating drivers and platform code are locked out from ever contributing it back to the kernel community. The kernel community has for years been telling these companies to get their code merged, so that they can take advantage of the security fixes, and handle the rapid API churn automatically. And these companies have listened, as is shown by the larger number of companies contributing to the kernel every release.
[...]
But now they are stuck. Companies with Android-specific platform and drivers can not contribute upstream, which causes these companies a much larger maintenance and development cycle."
(Log in to post comments)
Greg Kroah-Hartman: Android and the Linux kernel community Posted Feb 3, 2010 3:08 UTC (Wed) by fuhchee (guest, #40059) [Link]
"... Companies with Android-specific platform and drivers can not contribute upstream, which causes these companies a much larger maintenance and development cycle ..."
Have any of these companies supplied anecdotal or better evidence of this greater burden they undertook? What justifies the sense that they are interested in keeping up with upstream linux developments for their phones?
Greg Kroah-Hartman: Android and the Linux kernel community Posted Feb 3, 2010 3:52 UTC (Wed) by gregkh (subscriber, #8) [Link]
Yes, a number of companies have approached the kernel development
community to try to get us to help solve this problem. It is causing them a lot of grief, as I tried to explain.
You know things are bad when a hardware company comes and asks the
Greg Kroah-Hartman: Android and the Linux kernel community Posted Feb 3, 2010 17:28 UTC (Wed) by vblum (guest, #1151) [Link]
Hm, having read your blog article (and not the rest, in particular not any other details on the earlier discussions of API changes), and also having read Chris DB's comment further below, I have to say that Google perhaps has a point.
... with an investment as large as Google's in Android, they have to be allowed to make and retain their own design decisions, at least for their own use.
For example, the "Android userspace logic" would seem a difficult thing to legislate for anyone but Google, unless the necessary changes are minor. Likewise, if they find a new lock type necessary for their work, it would seem a tall order to have them take it out now (and deal with the resulting breakage on their users' phone ...)
So is there really no compromise possible in another way? (for example - only drivers needed for Android get to use the Android-specific infrastructure?) That at least would keep the overall code base closer to the mainline than with an outright fork ... and well, perhaps some day someone else actually finds that framebuffer infrastructure unexpectedly helpful after all?)
Greg Kroah-Hartman: Android and the Linux kernel community Posted Feb 3, 2010 19:34 UTC (Wed) by dlang (✭ supporter ✭, #313) [Link]
no, Google created some new core kernel infrastructure. For it to work (and for the drivers that depend on it to work) it needs to be implemented everywhere.
Kernel developers (including other embedded developers who have achieved good power savings modes) don't believe that the Android way of doing things is good.
Greg Kroah-Hartman: Android and the Linux kernel community Posted Feb 3, 2010 3:52 UTC (Wed) by dowdle (subscriber, #659) [Link]
If you would have read the fine article, Greg mentioned that he had gotten a number of questions from developers as a result of the removal of the Google code... and that a number of companies affected by the removal were already working on changing their code to get it into the mainline since Google has dropped the ball.
Greg Kroah-Hartman: Android and the Linux kernel community Posted Feb 3, 2010 10:11 UTC (Wed) by imcdnzl (subscriber, #28899) [Link] It is interesting to watch this... and contrast with Symbian where we are moving to a fully open environment, including the planning stages. It is a slightly different model though being EPL, rather than GPL. Keep an eye on http://countdown.symbian.org as this will signify something important soon... Disclaimer: I was a Linux kernel developer (DCCP), and now work for Symbian as Head of IT.
Greg Kroah-Hartman: Android and the Linux kernel community Posted Feb 4, 2010 0:49 UTC (Thu) by Doogie (guest, #59626) [Link]
Where can I download the source code to Symbian without having to register first?
Greg Kroah-Hartman: Android and the Linux kernel community Posted Feb 4, 2010 6:34 UTC (Thu) by imcdnzl (subscriber, #28899) [Link]
Greg Kroah-Hartman: Android and the Linux kernel community Posted Feb 3, 2010 17:05 UTC (Wed) by cdibona (subscriber, #13739) [Link]
I think if the Android kernel were important enough to the mainline, then
this wouldn't be a problem. The reality is that the mainline doesn't want the code, so a fork is a normal response to this. The numerous ARM/SoC forks bear that logic out.
For the work we do on our non-mobile systems (our production kernels and the
Also, this whole thing stinks of people not liking Forking. Forking is
Greg Kroah-Hartman: Android and the Linux kernel community Posted Feb 3, 2010 17:16 UTC (Wed) by dlang (✭ supporter ✭, #313) [Link]
I disagree, if the kernel forked for each new type of system we would have dozens of different versions of the kernel
desktop
and most of these would also be forked for different processors
Linux is what it is because it has integrated all these needs into one codebase, so when someone adds an enhancement (say a filesystem) to the kernel who only cares about x86 desktop use, it also is available to all the other uses as well.
This means that the bar for adding new stuff to the kernel is fairly high as it needs to be done in a way that will work for everything (or at the very least, not hurt for anything)
The kernel developers would very much like to integrate Android support into the kernel, but that doesn't mean that they will take any code thrown their way and apply it. The google Android team worked in a vaccum, they didn't talk to the kernel team until they were finished, at that point the kernel team responded with 'what about doing it this way' and 'how about using this feature that already exists to solve this problem', the Android team has dropped the ball and not worked to either change the Android code or to justify why there is no better way.
I will agree that the _ability_ to fork is important, it keeps everyone honest, but actual doing a fork is a sign of failure on someone's part, in this case it seems to be clearly a failure on the part of the Android team
Greg Kroah-Hartman: Android and the Linux kernel community Posted Feb 3, 2010 18:26 UTC (Wed) by cdibona (subscriber, #13739) [Link]
If you don't like how we architected android, don't complain that the code
isn't being mainlined. You can't have your cake and eat it too. We get it, you don't like how we put together the kernel for android. We're okay with that. Other companies coming to you for help is fine, too, you can choose who you help. Others might see that as an opportunity, but whatever.
That you say (multiple times) that we're working in a vacuum is simply not
Drag: The point of the arm forks comment was that there is clearly
To sum up: I think that Android is an unusual use of Linux that doesn't
Greg Kroah-Hartman: Android and the Linux kernel community Posted Feb 3, 2010 19:05 UTC (Wed) by mjg59 (subscriber, #23239) [Link]
The code could be mainlined if Google were willing to consider that their wakelock approach was
suboptimal and adapt to a more reasonable one. This could be done without breaking existing userspace interfaces.
I'm not aware of any significant interaction between the Android project and the kernel
A little over 18 months ago at LRL USA, Robert Love talked about how Android were keen to
I can't find anything to support your assertion that this has been a long term process - the major
Greg Kroah-Hartman: Android and the Linux kernel community Posted Feb 3, 2010 21:39 UTC (Wed) by dcbw (guest, #50562) [Link]
So just because it comes out of Google, and because you built it, all the technical decisions that Google made without input from non-Google developers are perfect and should just be accepted?
That's seriously arrogant.
Feedback is constructive. Perhaps wakelocks *aren't* the best solution for the problem you're trying to solve. Perhaps other kernel developers have more flexible solutions that solve Google's problem just as well as wakelocks, but solve other people's problems too.
I can't remember significant interaction between Google Android developers and mainline kernel developers before early 2009. Contrast that to Nokia who have been *all over* the kernel upstreaming their OMAP patches for the N8xx and N900.
If wakelocks and binder really are the best solutions to the problem for the Linux kernel, then it wouldn't be hard to justify them as such, right? But that hasn't happened.
Greg Kroah-Hartman: Android and the Linux kernel community Posted Feb 3, 2010 21:53 UTC (Wed) by cdibona (subscriber, #13739) [Link]
No, that is exactly not what I am saying. See my other posts.
Greg Kroah-Hartman: Android and the Linux kernel community Posted Feb 3, 2010 21:59 UTC (Wed) by dcbw (guest, #50562) [Link]
"I think if the Android kernel were important enough to the mainline, then
this wouldn't be a problem."
i.e. just because millions of users are using it mainline should ignore technical considerations and just take the code? That seems pretty wrong...
Greg Kroah-Hartman: Android and the Linux kernel community Posted Feb 3, 2010 22:17 UTC (Wed) by cdibona (subscriber, #13739) [Link]
I don't see how that is wrong or arrogant. Let me break it down:
if: Android mattered more to the mainline maintainters.
Nowhere in there did I say we were that important to them.
And be careful flinging around numbers, we have many many millions of handsets, too. It's not about that. It's about what is best on one side for the mainline kernel (and androids mods may not be that) and what is best for the people with androids in their pockets, and indeed whether it matters to harmonize the two.
If you read arrogance in that, that wasn't the intent.
Greg Kroah-Hartman: Android and the Linux kernel community Posted Feb 3, 2010 22:27 UTC (Wed) by dcbw (guest, #50562) [Link]
I actually did mean "millions of Android users", not millions of mainline kernel users.
The arrogance comes from expecting the kernel developers to ignore technical considerations just because there are millions of users of your code.
While having millions of users may mean it works, it may not be the best solution for the problem, for upstream *or* for your users. You may find that the solutions proposed by the kernel developers also make your life easier through easier maintenance of the code and cleaner subsystems when you have to enhance them in the future.
But you have to be /open/ to the possibility that wakelocks may not be the best solution to the problem, instead of proposing that kernel developers just accept them as-is even though they are (in the opinion of some very knowledgable upstream developers) technically faulty.
Greg Kroah-Hartman: Android and the Linux kernel community Posted Feb 4, 2010 19:25 UTC (Thu) by rahvin (subscriber, #16953) [Link]
I'm glad the Android developer posted. He's conveyed something I didn't realize, that is that Google and their developers believe they are right regardless of what others think. It sounds like this might be a corporate problem in that they truly believe they are right and everyone else is wrong.
I don't think anyone would argue that in some instances it's better to be out of tree than in, but it's the exception, not the rule. Part of getting into mainline is listening to other people when they tell you your code is garbage and your solution is the wrong one and that a better one exists. This is especially true when the other people doing the same kind of development (embedded in this case) tell you that your approach is the wrong one as otherwise you might have an argument that what you propose is an embedded specific exception.
It's an eyeopener to me to see how arrogant the Google developers are about their code. It explains their toss it over the wall attitude precisely. I have to wonder how long Google will be able to maintain this attitude with all the other players in the Open Handset Alliance, and most particularly what will happen when Google gets tired of Android and abandons it. It's unfortunate this attitude exists, while the corporation is interested in Android it will raise their costs significantly and when they abandon it if others haven't forked the code it will die. Rather unfortunate, but as I said, it's probably a corporate problem that in the long term will see most Google code abandoned, rather unfortunate but I guess that's what happens when you can't play with others or take constructive criticism. There is an interesting sociological question on where such corporate attitudes develop and whether they are related to some demographic figure, such as average age, of the Google developers plays a role.
Greg Kroah-Hartman: Android and the Linux kernel community Posted Oct 3, 2011 18:02 UTC (Mon) by tbird20d (subscriber, #1901) [Link]
But lots of embedded developers, me included, think that Android has some nice stuff, that is being blocked from mainline by stubborn-ness, ignorance and pride, rather than any substantive technical issues.
The 'they come to us' attitude Posted Feb 4, 2010 4:42 UTC (Thu) by eparis123 (guest, #59739) [Link] then: We'd get more attention from them .. The linux kernel development model (or any large FOSS development model thereof) depends on initiation from the patches authors, not from 'them', the community. I'm sorry if this sound rude, but Al Viro once commented here as I remember that the community perceives the 'they come to us' attitude as a sign of arrogant developers. .. or they'd change their practices. You don't really join a community by asking it to change its practices firsthand! I guess this is really obvious
"If Android code was important to the mainline kernel..." Posted Dec 9, 2010 10:37 UTC (Thu) by dneary (subscriber, #55185) [Link]
Replying to a really old comment... don't know if that's good or bad LWN etiquette.
To turn this phrase on its head, one could argue that:
If getting code upstream was important to Android developers:
No-one likes to maintain someone else's abandoned code.
I only bring this up now because of a recent announcement which reads like a regression to me: http://source.android.com/source/code-lines.html (that said, it has the advantage of clarifying expectations, which I applaud).
Cheers,
Greg Kroah-Hartman: Android and the Linux kernel community Posted Feb 3, 2010 19:30 UTC (Wed) by dlang (✭ supporter ✭, #313) [Link]
actually, I need to clarify my last statement.
temporary forks are definantly common in linux development. The key is that they are temporary with the intent being to merge them as the ideas that caused them to be created are tested.
Every developer who submits patches to linux is technically doing this, but there is a long track record of larger forks. The RT tree is a perfect example of this, they forked and did a lot of work, and are now getting that work merged back in.
so forking is bad, but fork + merge is the bread and butter of kernel development
Greg Kroah-Hartman: Android and the Linux kernel community Posted Feb 4, 2010 14:44 UTC (Thu) by nevets (subscriber, #11875) [Link]
I am one of the RT developers and even maintained the RT tree for a while in the past.
When we first started working on -rt, it was not too different from Android. That is, we were working in a niche market. We did not even know that mainline needed anything we were doing. At the time it was more R&D.
Maintaining a tree outside of mainline is extremely difficult. As -rt developed, we realized that it has advantages for mainline. But more importantly for us, the more we got into mainline the easier our job would be.
Mainline was a mess with respect to Real-Time. We started sending in patches to clean it up. We started combining more spin_lock() and local_irq_disable() into spin_lock_irq() which helped mainline annotate what locks were being used in interrupt context and which were not.
hrtimers, generic timers, generic interrupts, lockdep, ftrace, mutex, pi futex, and a list of others all came from this -rt work.
Once we realized that -rt was worth the effort, we started working hard to get all of it into mainline. If we just passed the patch over to the mainline developers and said, here it is, take it or leave it, they would have laughed in our faces. Ever single enhancement that we had in -rt, we had to rewrite 2 or 3 times before it was acceptable for mainline.
Having your code in mainline will make things much easier in the long run. Believe me, it can be hell to constantly keep up with mainline. If you look at your work and ask yourself, how could this help everyone not just our little niche, you will see better solutions that will not only help mainline, but will also help you. Every time we did a rewrite, the end result was always something better for the RT tree.
Listen to the mainline developers, they may actually make your product better.
Greg Kroah-Hartman: Android and the Linux kernel community Posted Feb 3, 2010 17:50 UTC (Wed) by drag (subscriber, #31333) [Link] The numerous ARM/SoC forks bear that logic out.The numerous ARM/SoC forks bear out exactly why this is such a horrific idea. As somebody who has worked on porting Linux to a older ARM SoC device having support for most of the hardware already integrated into the kernel was invaluable. Most of these devices are custom, but are based on reference platforms. And having support for those reference platforms in the kernel means that people can relatively quickly and easily port _MODERN_ versions of Linux to new systems. That way you get all the benefits of all the development that has gone in the last few years. The kernel build system can take into account everything from device addressing, memory addressing, execute in place, building the bootloader into the kernel image and all that happy horseshit you have to go through on a 'legacy free' SoC design. Stuff that you don't have to deal with on x86 due to the standardization. If that code for the bootloader, kernel, memory mappings, and devices never got into upstream copies of the kernel/uboot/etc then what that means is that your forced to hunt around for patches to random versions of software... much of which is incompatible with each other and all of which will never work with any modern versions of software that was patched. That is a development nightmare, maintenance nightmare, and ruins much of the reason for using a open source system in the first place. If I have to develop everything from scratch then that means that developing for Linux is usually going to be as or even more expensive then doing it for something like Windows CE. It's the difference between looking up hardware documentation, editing 3-4 files, and compiling and having something working in less then a week versus spending months and tying up multiple developers just to get to the point were a device is bootable. What google is doing, by not working closely with kernel developers, is self-defeating. And kernel developers are right in rejecting radical modifications to the kernel that was done without their input and without the support of original hackers. We are going to need something better then just random code dumps to keep having nice kernel.
Greg Kroah-Hartman: Android and the Linux kernel community Posted Feb 3, 2010 18:09 UTC (Wed) by mjg59 (subscriber, #23239) [Link]
The reason mainline hasn't accepted the wakelock infrastructure is that Google has still failed to
demonstrate why it's necessary. Almost identical benefits can be obtained using the kernel's existing range timer functonality, which has the added bonus of removing the need for the strong userspace/kernel tying that Android requires right now.
Of course, Google's a big enough company that maintaining a fork isn't a significant problem for
Greg Kroah-Hartman: Android and the Linux kernel community Posted Feb 3, 2010 18:09 UTC (Wed) by gregkh (subscriber, #8) [Link]
> The reality is that the mainline doesn't want
> the code, so a fork is a normal response to this.
That is not true. mainline does not want the code "as-is" yes, because
Forking is fine, as long as you have the resources and ability to maintain
In short, you are not working in a vacuum here.
Greg Kroah-Hartman: Android and the Linux kernel community Posted Feb 3, 2010 19:33 UTC (Wed) by morrildl (guest, #63330) [Link]
> But note that you now have made other companies
> dependent on that fork, which is causing them grief
Since I am the Android Open-Source and Compatibility lead, I get a lot of
Regardless, please do feel free to forward these inquiries to me, since you
See, NOW it makes sense... Posted Feb 3, 2010 20:19 UTC (Wed) by Epicanis (guest, #62805) [Link] "please do feel free to forward these inquiries to me, since you shouldn't have to deal with them."Maybe I'm reading too much into this (especially since I'm not a "kernel developer" by even the most ridiculous of standards) but what I'm getting from this is that you really DON'T want to work with mainline kernel developers. Perhaps Google isn't comfortable with the compromising and reduced control that would necessarily be involved in sharing responsibility with the mainline developers? While Greg KH presumably is interested in devices working on all Linux platforms, Google presumably feels "who cares about the rest, so long as it works on Google® phones running Android®". That, at least, is what this response implies to me. (That'd be understandable, but it might save a lot of pointless arguing if Google would just come out and say so explicitly if that's the case...)
See, NOW it makes sense... Posted Feb 3, 2010 20:28 UTC (Wed) by dlang (✭ supporter ✭, #313) [Link]
note that such a policy would seem to contradict the google opensource policy that made the news a few weeks ago. That policy stated that every google opensouce project should be working to get their work upstream.
See, NOW it makes sense... Posted Feb 3, 2010 20:58 UTC (Wed) by morrildl (guest, #63330) [Link]
> Maybe I'm reading too much into this
You are.
> Perhaps Google isn't comfortable with the compromising and reduced
Wow, what is it with Linux people automatically jumping on the "control"
It's true that Android does a few unusual things in the kernel, and it's
For our part, meanwhile, we have working code that's running on millions of
And anyway, what's the hurry? Are we trapped in some hellish game where the
See, NOW it makes sense... Posted Feb 3, 2010 21:07 UTC (Wed) by mjg59 (subscriber, #23239) [Link]
Android runs on hardware where wakelocks are unnecessary. Retention mode is generally
equivalent to system suspend in terms of power, so all you need is for the kernel to stop scheduling tasks. There's no "new paradigm" here, there's simply an inelegant mass of code that ties your userspace to your kernel, buys you nothing in the process and prevents your code from going upstream.
Like I said, this could be rearchitected without breaking the existing userspace interface. It'd even
See, NOW it makes sense... Posted Feb 3, 2010 21:14 UTC (Wed) by dberlin (subscriber, #24694) [Link]
"Android runs on hardware where wakelocks are unnecessary. "
Great, sounds like you are volunteering to actually remove them and show that this is the case, right?
See, NOW it makes sense... Posted Feb 3, 2010 21:21 UTC (Wed) by mjg59 (subscriber, #23239) [Link]
If I had (a) hardware, and (b) some indication that Android upstream would accept it, sure.
See, NOW it makes sense... Posted Feb 4, 2010 7:31 UTC (Thu) by cdibona (subscriber, #13739) [Link]
That's easily solved, Matt, just email me. :-)
See, NOW it makes sense... Posted Feb 4, 2010 11:36 UTC (Thu) by kragil (guest, #34373) [Link]
That answer just gave me an idea:
How about giving contributors not employed by Google or Open Handset members
It is easy to do (just grep some SCM output, or just put "providing proof"
Kernel hackers would certainly be ledgeable and if they bother to apply for
Disclaimer: I certainly wouldn't qualify.
See, NOW it makes sense... Posted Feb 4, 2010 14:12 UTC (Thu) by gregkh (subscriber, #8) [Link]
Google has been very generous in giving out free phones to lots of kernel developers in the past, so this isn't really an issue.
See, NOW it makes sense... Posted Feb 4, 2010 14:16 UTC (Thu) by mjg59 (subscriber, #23239) [Link]
Like I said, spending time on this would be conditional on some assurance that it wouldn't just be rejected out of hand. I'm reasonably sure that this can be implemented without any changes to applications being required, but it may need some (fairly minor) alterations to the underlying Android stack.
See, NOW it makes sense... Posted Feb 5, 2010 19:38 UTC (Fri) by jejb (subscriber, #6654) [Link]
> Like I said, spending time on this would be conditional on some assurance
> that it wouldn't just be rejected out of hand. I'm reasonably sure that > this can be implemented without any changes to applications being > required, but it may need some (fairly minor) alterations to the > underlying Android stack.
If google wishes to offer fair terms for this type of challenge, count me in. I've already done quite a bit of android hacking with the G1 Qualcomm gave me. It looks to me like the move off wakelocks (and possibly binders) can be contained to some small glue changes in bionic and some of the java classes.
I'd suggest reasonable terms be that if we get this working acceptably (meaning comparable battery life to the existing 2.6.29 implementation), Google would commit to moving to the upstream kernel we produce and integrating the necessary changes (whether ours or rewritten ones) into the android user code.
See, NOW it makes sense... Posted Feb 7, 2010 9:11 UTC (Sun) by swetland (subscriber, #63414) [Link]
I'd love some assurance that *our* patches wouldn't just be rejected out of hand too.
One thing we are working on doing is getting the external open tree to the point where it's trivial for somebody to check it out and do a full from-source build for Nexus One (which is completely unlockable and reflashable, as shipped from the factory). This then gives everyone some common ground to test patches against, and perhaps measure power consumption of various builds across the normal range of testcases.
At the moment the remaining issues are some build system cleanup, and sorting out redistribution of the handful of userspace proprietary binary modules (roughly 10 libraries/binaries -- fewer, if I'm reading the wiki right, than in a N900).
I can't promise you that we'd absolutely take a change -- it'd have to be verified to actually be better (or at least no worse), across the board, on power management, and that there were no regressions in stability or correctness of any driver. We spent quite a lot of time measuring power usage both in lab and field trial environments -- changes that fundamentally alter how this works get a lot of scrutiny.
See, NOW it makes sense... Posted Feb 7, 2010 19:07 UTC (Sun) by mjg59 (subscriber, #23239) [Link]
So from my perspective, the three major sticking points are wakelocks, the early suspend/late resume code and the binder. We can argue over whether the binder code strictly needs to be in the kernel, but it's not something that has any real influence over the platform code. If we were in a world where the only significant difference between the mainline kernel and the Android one was binder support, I think that'd be the least of our problems.
The early suspend infrastructure is more interesting. I think it's fair to say that it's a somewhat hacky solution to the problem (take msm_fb - early_suspend and earlier_suspend?), but the problem it's solving is certainly legitimate. Rafael's been working on a mechanism for providing a power management dependency tree that's orthogonal to the device tree. I'd hope that this would provide the necessary functionality that we could do without early suspend, but it'd be great to talk about that.
Wakelocks are obviously the most controversial aspect of this. I think the main problem is that there's never been a terribly convincing answer as to why wakelocks are the appropriate solution to the problem. The upstream omap tree's approach has been to implement core suspend as a cpuidle state with appropriate latency restrictions. That fits in nicely with how other architectures manage this kind of functionality, so I think the onus is really on Google to explain why wakelocks provide more functionality than something that makes use of existing kernel code.
I appreciate that the "Please throw all of this out and rewrite it from scratch" attitude is a problem, and I'd entirely understand if Google don't feel that the work required to rework all of this in a way that upstream approves of would be justifiable. On the other hand, Android's absence from various forums (the collaboration summit, LPC, Linuxcon, the power management mini-summits and so on) makes it much harder for us to appreciate why this difference of opinion exists. It's true that the initial response to wakelocks was pretty universally negative without a lot of effort being spent on figuring out alternative solutions that would be mutually acceptable, and I'm certainly sorry that it happened that way - but at this point I suspect that we've been largely talking past each other for the best part of a year, and it would be good to spend some time figuring out why we have this fundamental disagreement before having another round of acrimonious debate culminating in another pile of code that goes nowhere.
See, NOW it makes sense... Posted Feb 8, 2010 5:56 UTC (Mon) by swetland (subscriber, #63414) [Link]
Yeah, I consider the binder a side-issue. In theory one could replace it entirely, if one were willing to write a bunch of userspace code that accomplished everything userspace needs from it, etc, etc, and it is self-contained and no other drivers depend on it.
I thought we had actually explained a number of situations where wakelocks provide value beyond existing solutions -- I'll have to dig through the linux-pm threads and try to extract some stuff from there. One thing we're looking to do is put together a clean writeup of the "why" behind all this as a basis for discussion.
I don't see how retention from idle (which we do as well) fully solves the problem -- since you can easily have situations, in an environment where you can't control every process/thread (300+ threads in a typical android system at boot -- yeah, kinda crazy, but it is what it is). Going to suspend means we don't need to worry about non-optimal software with polling/periodic behaviors causing the CPU to wake up more often than ideal. Also, you still have situations in which you *can't* go to full retention in idle because of a driver being busy (which is what idlelocks -- just another class of wakelock) are about.
I think it would be extremely valuable to get some folks together somewhere and try to hash some of this out. In general we are not opposed to slightly different ways of doing things as long as we accomplish what we need to accomplish power-wise. I mean, Arve's already respun the wakelock API (now suspend_blocker, no?) several times in response to various feedback.
I'd love to engage in a forum where the goal is to solve common problems rather than assign blame -- Greg's blogpost implying that he's giving a CELF keynote about how we suck does not create much enthusiasm on our side for attending, for example.
We know that some people would have been happier if we worked out a design for all of this before 1.0 shipped, but that didn't happen, and arguing about how practical it would have been or not doesn't really gain anybody anything. Figuring out how to collaborate on things so that future issues get sorted out earlier and more smoothly seems entirely worthwhile though.
So when/where is the next time/place that we could meet up with the other interest parties on the linux-pm front and talk about these things?
See, NOW it makes sense... Posted Feb 8, 2010 6:34 UTC (Mon) by mjg59 (subscriber, #23239) [Link]
Right, it's absolutely true that the wakelock approach simplifies the userspace implementation,
but we could take a different view of things - at the most aggressive level, you could keep a userspace wakelock implementation and use the uswsusp interface to fire off the process freezer when all of them are released. All we'd need then would be a mechanism to provide a list of PIDs that you don't want frozen.
The drivers side of thing is more interesting, and I think that's something that could potentially
The next linux pm summit is planned for Boston in August, which is kind of far away. There's the
See, NOW it makes sense... Posted Feb 7, 2010 23:54 UTC (Sun) by kragil (guest, #34373) [Link]
I see another bug here .. why aren't all Googlers automatically LWN
subscribers? Do it like Canonical et al.
Chris? Fix that :)
Our editor is really cheap for big corps that have revenues like MS has
See, NOW it makes sense... Posted Feb 8, 2010 5:59 UTC (Mon) by cdibona (subscriber, #13739) [Link]
Any googler that wants an account can have one. We pay for it out of our
budget. Been like that for a number of years.
See, NOW it makes sense... Posted Feb 5, 2010 19:43 UTC (Fri) by jejb (subscriber, #6654) [Link]
OK, so to move this to a positive tone, We (that's me, mjg59 and any other developers who join) will accept a challenge to move android to the vanilla kernel:
http://lwn.net/Articles/373173/
Are our terms acceptable, or would you like to add other conditions?
See, NOW it makes sense... Posted Feb 3, 2010 21:23 UTC (Wed) by dlang (✭ supporter ✭, #313) [Link]
well, assuming that phone hardware is similar, the Nokia phones would appear to be proof that you don't need wavelocks to get good battery life.
See, NOW it makes sense... Posted Feb 3, 2010 21:46 UTC (Wed) by jebba (✭ supporter ✭, #4439) [Link]
...but the userspace code to recharge the battery is completely closed...
See, NOW it makes sense... Posted Feb 3, 2010 21:48 UTC (Wed) by mjg59 (subscriber, #23239) [Link]
Which is entirely orthogonal to runtime power consumption.
See, NOW it makes sense... Posted Feb 3, 2010 21:49 UTC (Wed) by dcbw (guest, #50562) [Link]
The N900 uses TI OMAP processors and Nokia radios, which is different than Android phones which typically use integrated Qualcomm chipsets and radios.
But they both have to solve the same problems (power management) and the fact that Nokia seems to get good battery life out of their devices appears at first glance to be a counterpoint to the necessity of wakelocks.
See, NOW it makes sense... Posted Feb 3, 2010 21:55 UTC (Wed) by dlang (✭ supporter ✭, #313) [Link]
I wasn't aware that android was supposed to be tied to a specific chipset. I was under the impression that it was intended to be used on a wide variety of devices.
is there anything that would prevent android from being run on the nokia phones, or the nokia software from running on android phones (other than the fact that some drivers are only in the android fork and the android fork does not include all the drivers in the main kernel)?
See, NOW it makes sense... Posted Feb 3, 2010 22:04 UTC (Wed) by dcbw (guest, #50562) [Link]
Android *isn't* tied to a specific chipset. But when Google was creating Android they built it for the reference design (HTC Dream which used a Qualcomm MSM7201A SoC). And Google doesn't appear to be interested in porting Android to other SoCs like TI's OMAP; they appear to be entirely focused on Qualcomm chipsets where they can leverage their existing investment.
As a company building phones, would you rather just use the same hardware Google build Android for, or would you rather invest a ton of time into porting Android to another SoC and write completely new SDIO, radio, power-management, etc drivers while your competitors come out with 3 new phones?
It makes a ton of sense that (AFAIK) nobody has come out with a non-Qualcomm Android phone, since to do so would require a significant investment. If you use Qualcomm MSM7K or MSM8K (snapdragon) platforms, then Google has already made that investment for you, for the most part.
See, NOW it makes sense... Posted Feb 3, 2010 22:10 UTC (Wed) by mjg59 (subscriber, #23239) [Link]
Not phones, but the Nook is android on a Samsung platform.
See, NOW it makes sense... Posted Feb 3, 2010 22:38 UTC (Wed) by dcbw (guest, #50562) [Link]
Interesting; though the scope there is quite a bit smaller; no real radio management, low-latency audio paths, fast 3D graphics, are required which makes the driver bring-up less intensive. Also the hardware refresh cycle for these is probably a lot slower than a phone, so you can recoup a lot more of your investment.
See, NOW it makes sense... Posted Feb 4, 2010 11:24 UTC (Thu) by broonie (subscriber, #7078) [Link]
Thing is, Linux has a pretty good mainline stack for all of these things (possibly excepting 3D) - to the extent that people play with mainline a lot of this stuff is available off the shelf.
See, NOW it makes sense... Posted Feb 4, 2010 11:35 UTC (Thu) by SimonKagstrom (subscriber, #49801) [Link]
The Samsung Spica Android phone is also built on their own platform (6410 I
believe).
See, NOW it makes sense... Posted Feb 3, 2010 22:13 UTC (Wed) by dlang (✭ supporter ✭, #313) [Link]
so all the vendors working on android based e-readers, netbooks, etc should not be doing so, they should be using some other linux variant ans android only cares about phones with the current generation of qualcomm chipsets.
this also says that the next generation of qualcom chipsets may need a different OS as android is not working to be multi-platform.
See, NOW it makes sense... Posted Feb 3, 2010 22:49 UTC (Wed) by dcbw (guest, #50562) [Link]
I never said anything of the sort. Netbook and ebook reader makers can certainly use Android.
What I *did* say was that if you move away from the Qualcomm-based platform, you are going to be investing a *lot* more into the driver and architecture bring-up. But that could very well be a way of differentiating yourself in the market. It's a cost/benefit analysis and it appears that ebook reader makers have decided that over the long run they will make more money than it cost to port Android to a new architecture.
But the fact that almost all the phones running Android are doing so on Qualcomm 7k or 8k chipsets shows that most *phone* companies appear to be maximizing their investment by leveraging the work that Google has already done. Phone product cycles are also a lot faster than ebook reader product cycles, and phones have a lot more hardware in them (which requires more/different drivers) than ebook readers.
And just because Qualcomm comes out with a new platform doesn't mean your investment is worthless. Often new iterations of a chipset will contain only tweaks of the older hardware; they rarely rearchitect the entire system because that simply trashes all the money you've invested in the software so far. Which isn't smart.
For example, Android on the MSM8k can use mostly the same radio drivers (qmi), shared memory drivers (smd), network device (rmnet), etc that the MSM7k used. The GPU probably required a significantly modified driver, but at least you didn't have to rewrite *all* the drivers just to move from MSM7K -> MSM8K.
See, NOW it makes sense... Posted Feb 3, 2010 23:08 UTC (Wed) by cdibona (subscriber, #13739) [Link]
As an aside, the Verizon/Moto Driod runs on a TI OMAP. So...
See, NOW it makes sense... Posted Feb 3, 2010 23:59 UTC (Wed) by dcbw (guest, #50562) [Link]
Right you are. Quick question, I see a lot of @android.com on the OMAP patches. Who spearheaded the Android OMAP efforts here? Google, TI, Motorola, combination of all?
See, NOW it makes sense... Posted Feb 4, 2010 7:33 UTC (Thu) by cdibona (subscriber, #13739) [Link]
A combination. It's always a mix, from what I understand.
See, NOW it makes sense... Posted Feb 4, 2010 0:32 UTC (Thu) by mjg59 (subscriber, #23239) [Link]
OMAP, of course, being a platform that achieves good battery life without wakelocks.
See, NOW it makes sense... Posted Feb 4, 2010 19:51 UTC (Thu) by rahvin (subscriber, #16953) [Link] As a company building phones, would you rather just use the same hardware Google build Android for, or would you rather invest a ton of time into porting Android to another SoC and write completely new SDIO, radio, power-management, etc drivers while your competitors come out with 3 new phones?I would rather see you upstream all your changes (including userspace), then TI or other phone makers can upstream and GPL all their changes (including userspace tools) and in the long run Android runs on anything that the Kernel runs on. There is a reason it's called a community. Rather that build everything yourself why not get upstream so that all the work others are doing can be used. That way if some new SOC maker comes to Google and says "we want you to use our chip" you simply tell them to get "X" into the Kernel and then you aren't dependent on one supplier/IP platform. I think you guys have fully missed the point of FOSS.
See, NOW it makes sense... Posted Feb 3, 2010 21:56 UTC (Wed) by dcbw (guest, #50562) [Link]
You're assuming wakelocks are the best solution to the problem. They may be the solution that Google came up with, but just because Google came up with it doesn't mean it's the best. And the kernel developers who reviewed the wakelock patches appear to disagree that it's the best solution.
So why didn't Google initiate discussion of advanced mobile power management techniques during the initial development of Android in 2007/2008, when everyone could come together and agree on a good design *before* you started shipping millions of products?
Of course if you have enough engineers you can forward-port until the cows come home, but that's not a great use of engineering talent...
On one side this is Linux developers appearing to whine that Google didn't do things the way they "should have been done", and on the other side it's Google appearing to be arrogant by expecting that every technical decision they made is the best one and that everyone else should see that and accept it as-is with great fanfare and many thanks.
See, NOW it makes sense... Posted Feb 3, 2010 23:19 UTC (Wed) by Epicanis (guest, #62805) [Link] Wow, what is it with Linux people automatically jumping on the "control" meme? This is purely a pragmatic thing.Control is "a pragmatic thing" - at least from the point of view of the control-holder. The point is, it's often seen as easier and more easily profitable to retain control over something than to let someone else have influence over one's business plans. The reason "Linux people" (I take it you are NOT a "Linux person"?) tend to be sensitive to this is because of how often it becomes a roadblock. Proprietary data formats, proprietary video and audio codecs, software patents, digital restrictions, proprietary hardware, trademark and copyright licensing... all there to retain control. Apple's profitability seems like a pretty good demonstration of how iron-fisted control can be pragmatic. I recognize that if it were the case that Google just doesn't want to deal with working out compromises with mainline kernel developers that this would probably be a purely pragmatic decision, perhaps on the assumption that not having to do so would decrease delays in Android development, and perhaps, like Apple, make it easier to control the perception of the Android brand (reducing the chance that a change somewhere in the kernel that Google doesn't operate on inadvertently causing performance problems for Android, for example). Obviously not everyone agrees that this is the best course of action in the long run, otherwise there wouldn't be all the arguing and occasional light snark that we're seeing over this. It doesn't help that Google occasionally does things that make people question its attitude towards Linux - Google often seems reluctant to port their standalone applications to Linux, for example. Admittedly this doesn't affect me as much, since the one Google application that I really like to use has maintained pretty good Linux support...so far. I just hope Marble develops fast enough to take over before Google decides to quietly drop development on the Linux versions of Google Earth... tl;dr - Google's motives likely are pragmatic rather than malicious, but from this discussion they really don't seem to like having to deal with mainline Linux development and a lot of people seem concerned about how this might cause problems for users of mainline Linux if Google decides to walk away.
See, NOW it makes sense... Posted Feb 3, 2010 23:59 UTC (Wed) by kragil (guest, #34373) [Link]
Insightful comment. 100% agree.
Might I add a quote from Aaron Seigos recent blog post about the N900:
"I've seen this played out many times while I've knocked around the
There is no successful, closed ecosystem monopoly in the devices world yet.
See, NOW it makes sense... Posted Feb 4, 2010 1:25 UTC (Thu) by Doogie (guest, #59626) [Link]
I think you hit the nail on the head about the control issue. Two of the most critical components of Android (linux kernel, dalvik vm) are now effectively incompatible forks of the (currently) more common implementations of the original technologies. Once there are a couple of hundred million Android devices out there, who's to say that the Google controlled branch of the Linux kernel isn't the "standard" one and Torvalds' isn't the incompatible fork? Just sayin' :)
Please excuse one more followup... Posted Feb 4, 2010 1:12 UTC (Thu) by Epicanis (guest, #62805) [Link] It's probably really strange that I feel compelled to post here - especially more than once. I'm not a developer nor do I even know any developers personally, I'm not wealthy enough to own Google stock, I don't own an Android-based device (although I would be happy to get one, I must confess I'd rather have something maemo-based...), and whether or not "wakelocks" ever get implemented in the Linux kernel almost certainly has zero direct effect on me. I think the only link I have to anyone with a direct stake in all this is that Greg KH maintains the staging tree where the only currently-working driver for my netbook's wireless network chipset resides. Why the heck do I care so much?... "For our part, meanwhile, we have working code that's running on millions of devices, and we (also quite reasonably) are not eager to do extensive modifications to it simply for the goal of upstreaming it."There. That (I think) is the entire point around which the argument turns. The rest of the PR fluff, hyperbole, and accusations flying in both directions seem to be obscuring this. (the following paragraph is based on my potentially-faulty understading of the Android development timeline - corrections welcome.) I understand Google doing the early development of Android in relative secrecy - it was probably the only way to get it working and have it out in the market before a surprised Apple/Microsoft/Etc. could stall it with patent lawsuits and whatnot. The customizations apparently surprised the mainline Linux developers too, though. "Here it is, mainline developers. Take it or leave it, but it's too late to change anything now that it's out." "What? What's all this? It looks like it has some problems." "You're leaving it then? Fine." I understand the business reasons that I assume Google had for doing things this way. It's the way that it's being presented that seems to be stoking the fires. And anyway, what's the hurry? Are we trapped in some hellish game where the goblins will devour us if we don't meet our quota for adding upstream code?There's the presentation thing again. If "us" and "our" are read to mean "Google", it reads an awful lot like "And anyway, what are you Linux people gonna do to us it if we don't get our code upstream? Nothing, that's what!" I'm guessing you just meant to convey that some code living independent of the mainline Linux kernel with no intention of ever being in the mainline kernel isn't the proverbial end of the world. I agree, actually - but it's still not unimportant. Since we have working code, and it's not "less open-source" just because it lives in a different repo, why is this even an issue?Here's my hypothesis, developed with all of the insightfulness and wisdom of a Random Pseudonymous Commenter: Google is a huge, wealthy, influential corporation who is known to use a lot of Linux and therefore presumably has some sense of its worth, but now has split off its own "Android Linux" kernel, with some incompatibilities with the mainline kernel. There seems to be some fear that Big Powerful Google will attract more device-makers to write Linux drivers (which is a great thing), but said drivers will be written specifically for "Android Linux" and not everybody-else Linux (which is not so great) because who wants to, for example, write and maintain "wakelocks" and "no-wakelocks" versions of their drivers separately? This doesn't seem like an unreasonable prediction if Google is really dumping mainline Linux ("but we can still be friends, and that's okay..."). Sure, even if Google ends up just "throwing code over the wall" where mainline developers can sift through it and work on the task of translating the drivers to work on everybody-else-Linux, that IS much better than nothing at all. But it kind of hurts compared to what a more unified collaboration might accomplish. And after all this, I'm STILL not entirely sure why I, personally, even care, but I do...
Please excuse one more followup... Posted Feb 4, 2010 6:31 UTC (Thu) by mogul (subscriber, #3163) [Link]
Well spoken. I'm in the same situation with respect to personal involvement (and haven't even posted comments on LWN despite years of reading), and am bothered by the same dismissive elements in the posts above.
Fragmentation Posted Feb 4, 2010 20:32 UTC (Thu) by andypep (subscriber, #33588) [Link]
I agree with you, and I'm possibly even less concerned at an intimate level than you are.
What bugs me is the uneasy feeling that Linux may fail in the same way that Unix did - by being developed independently by a number of organisations that lead to a lack of standardisation and fragmentation. The GPL was the best bet to try to stop this happening by insisting on passing the modifications back into the pot, but this has to be de in a timely and considered manner if it is to continue to work that way. Previous forks have sorted themselves out in the market place - but it looks here like the kernel has developed two large separate markets that will go their different ways.
That bothers me in the medium to long term.
"Linux people" Posted Feb 4, 2010 4:21 UTC (Thu) by Epicanis (guest, #62805) [Link] I swear that unless someone indicates they want me to reply to something I'm going to digitally shut up here after this post, but that "Linux people" comment is bugging me. "Wow, what is it with Linux people[...]" You're the "Android Open Source and Compatibility Lead" and obviously have some role in Android development that puts you in a place to understand at least some of the technicalities of Linux kernel development - certainly more than me, at least. Despite this, you've just gone to the effort of coming here into the midst of a bunch of "Linux people" worried about potential compatibility issues between mainline Linux and Android Linux and implicitly declared yourself "not a Linux person". I don't get it. The "[Linux|Open Source] people are impractical for caring about compatibility and licensing and patents and stuff that consumers don't care about" meme seems to pop up a lot lately. Most of the time it's just obvious annoying trolling, or sometimes a blogger or "pod"caster who appears to think that saying so will buy them more "mainstream" credibility, but seeing even small signs of it from the "Android Open Source and Compatibility Lead" is vaguely disturbing. Please tell me this is just a matter of "posting while ticked off" and not really the way Google feels about things...
I'm worried. Posted Feb 4, 2010 15:02 UTC (Thu) by mgross (subscriber, #38112) [Link]
The wake lock patch isn't too objectionable "in theory" when it stayed as an alternate way for calling pm_suspend, but changing the semantics of /sys/power/state and the addition of the /sys/power/wait_fp_* file nodes to signal the surface flinger to start/stop with the drawing *is* nasty, and represents a significant fork.
I would argue that these are the most dangerous changes. With this a kernel for android is not compatible with a gnu/linux root file system (for power management).
So it was bad enough that an android root file system couldn't run on a normal linux kernel (missing binder, early suspend fb, wakelocks) but now if you run an android kernel with a gnu/linux user space you can't suspend.
Also, if you pull the omap or msm kernel git trees and do a git format-patch v2.6.29 (their baseline versions), you'll see 7240 patches for omap and 1083 for msm
Now grep those patches for wakelock and early suspend to see how may places the wake locks are finding themselves. Thus showing how wrong my "in theory" comment above was.
From a kernel developer perspective I fear we've took too long to "get it". This is a big forking point. Something needs to be done or most future Android devices will be stuck on a 2.6.29 base using old tool chains, with incompatible driver enabling for Si's device vendors and integrators to support.
its not just a problem for Linux, it will more than double the cost of enabling and support for platform enabling to for the Si's and device manufacturers.
It adds economic pressure to the device ecosystem. Making it more expensive to develop and sell new hardware. (wait, that is good for my stock.... just kidding...a little)
I have no worries. Posted Feb 7, 2010 8:59 UTC (Sun) by swetland (subscriber, #63414) [Link]
We've already moved to 2.6.32 (final cleaned up branches should be online this week) and will continue to track and rebase up to newer kernels, as we have since we started back with 2.6.14.
Arve is planning on once again reviving his push of wakelock patches to linux-pm (I guess this'll be rewrite #5) -- the last one stalled out after somebody told us we need to use read/write instead of ioctl()s and it was unclear how hard a requirement this was and if it really was an improvement.
The version he's been cleaning up for going-to-mainline discussion is not fully folded into our tree, since it keeps getting reworked based on feedback on the linux-pm list. We'll stick with what we have now until we can sort out something that everyone's happy (more or less) with.
I think if we can find some common ground on wakelocks, the rest really becomes a non-issue. The SoC specific code then is no different than any other kernel code, and the android generic drivers (binder, ashmem, logger, etc) have always been standalone components that, if necessary, could be maintained as a sidecar patchset.
Some things like the binder could be replaced -- we'd need somebody to completely replace the userspace infrastructure that depends on it, in a way that doesn't cause a performance or stability regression, but if somebody *had* such a set of patches and they survived automated and field stress testing, we could seriously talk about dropping the binder driver entirely.
- Brian
Greg Kroah-Hartman: Android and the Linux kernel community Posted Feb 3, 2010 18:34 UTC (Wed) by kragil (guest, #34373) [Link]
Sorry, but that was a not the insightful response I might have expected from Googles open source dude.
Forks can be very harmful to communities and a lot of companies in the Android ecosystem obviously feel the pain. Building ontop of Linus kernel is the better solution in the long run because it makes building and supporting an Android product easier. Just look at xda-developers how much work it is to adapt their WinMo hacks to new Android kernel releases, that would be much easier if the changes were upstream.
But judging from your response Google wants to maintain a fork for no good reason and Googles open source dude does not really understand some of the benefits of open source (singing the "Forks are not bad"-mantra does not cut it)
Greg Kroah-Hartman: Android and the Linux kernel community Posted Feb 3, 2010 18:40 UTC (Wed) by cdibona (subscriber, #13739) [Link]
Forks aren't always great, but I honestly don't think of forks as being a bad thing and I've tried to instill in Google the same ethic.
In fact, I'd say that the various forks of Linux, and how the Linux maintainers have roped back in some forks (and let others go on their merry way) is what made the Linux kernel great and not just a BSD rehash.
Also, expecting to port Android to a new SoC to be super easy is unrealistic. I have nothing but admiration for those that try and succeed, but porting android to a new platform is hard right now.
Greg Kroah-Hartman: Android and the Linux kernel community Posted Feb 3, 2010 19:29 UTC (Wed) by kragil (guest, #34373) [Link]
Thanks for answering. If it is the case that porting and updating Android is hard then IMHO Google should do everything to make the process easier. The whole purpose of Android is bringing secure internet to more people, isn't it? Then reducing the cost of building those devices should be one of Googles priorities.
BTW although I sound very critical my next Phone will be Android or Maemo and I hope to get a Pixel Qi Android tablet soon. But I hope I will always get an uptodate version of Android on every device. Androids track record in that regard is not stellar. That is another thing upstream friendly development might improve.
Greg Kroah-Hartman: Android and the Linux kernel community Posted Feb 3, 2010 19:33 UTC (Wed) by cdibona (subscriber, #13739) [Link]
I agree the process should be easier, but I'd say it is easier today than it
was 2+ years ago. But some of these new system on chips are pretty rough stuff sometimes.
There are some interesting cheaper androids out of HUWEI and some other
It's okay to be critical :-)
Greg Kroah-Hartman: Android and the Linux kernel community Posted Feb 3, 2010 19:40 UTC (Wed) by dlang (✭ supporter ✭, #313) [Link]
you seem to be contradicting yourself here.
you say that you don't think that forks are bad, but then you say that the linux way of eliminating forks is what has made it better than the *BSDs.
If long-term forks are acceptable, than the *BSDs should be the perfect example to point to as a measure of how they are so successful.
Linux has not had any long-term forks, it came close in the 2.4/2.5 days with Redhat, but even at it's worst it didn't get close to what a separate Android kernel would be or what the different *BSD kernels are.
Greg Kroah-Hartman: Android and the Linux kernel community Posted Feb 4, 2010 7:14 UTC (Thu) by rahulsundaram (subscriber, #21946) [Link]
Afaik Red Hat carefully backported features to the EL3 kernel after they
have already been mainlined in the development branch of Linux That's quite different from what Google is doing and their replies don't improve the situation
Greg Kroah-Hartman: Android and the Linux kernel community Posted Feb 4, 2010 7:37 UTC (Thu) by cdibona (subscriber, #13739) [Link]
Well, if anything, I was probably unfair to the BSDs, there are some solid , incredibly useful forks of bsd.... OpenBSD comes to mind (but that's gonna start a centithread if I bring that up too much) as does FreeBSD. They're both healthy and have solid direction.
I also happen to think that if you look back on this a few years or four from now, this will all wash out. By 2.8 maybe? :-)
Greg Kroah-Hartman: Android and the Linux kernel community Posted Feb 8, 2010 7:19 UTC (Mon) by rgmoore (✭ supporter ✭, #75) [Link] you say that you don't think that forks are bad, but then you say that the linux way of eliminating forks is what has made it better than the *BSDs.Part of the reason that Linux works well is because it lets people with good ideas go off to develop them for a while but always with the idea that those good ideas will be brought back into mainline if/when they've proven their worth. That freedom to develop radical new ideas is why the ability to fork is so important. The problem that the BSDs have had is that they haven't done as well with the merging back into mainline part. They've allowed their forks to become permanent splits instead of temporary development projects.
Greg Kroah-Hartman: Android and the Linux kernel community Posted Feb 4, 2010 15:34 UTC (Thu) by rich0 (guest, #55509) [Link] Forks aren't always great, but I honestly don't think of forks as being a bad thing and I've tried to instill in Google the same ethic.The big problem with this sort of approach is that projects on the scale of something like Chrome or Android can turn into entities unto themselves with almost no connection to the projects they were derived from. The last time I downloaded the chromium source the tarball was around 0.5GB - almost all of it from forked dependencies. The resulting build is unnecessarily long, and it ends up losing any benefit from upstream improvements in any of those dependencies. If zlib or whatever fixes a security flaw I need to download and rebuild another 0.5GB tarball to incorporate the fix. A company the size of Google can sustain this, I guess, but it makes the resulting project unwieldy. There may not ever be a community-based version of chromium as a result, just as there is no practical community- based version of android out there now (I'd define community-based as a fork with no corporate sponsorship). On the other hand, that need-not remain the situation permanently, and there are signs that Google has been trying to integrate with upstream/etc (but they've dug themselves into a bit of a hole). Sure, forks aren't always a bad thing, but they can be counterproductive and they can lead to isolation. When there is no reason to persist with a fork it behooves us all to work together if we truly value the open source spirit.
Greg Kroah-Hartman: Android and the Linux kernel community Posted Feb 3, 2010 22:55 UTC (Wed) by cmccabe (subscriber, #60281) [Link]
First, let me say that I have a G1 and I've been happy with it.
I think that Google has really built something amazing in Android, reimagining a lot of things like process lifecycle in a way that makes sense in the mobile world.
> For the work we do on our non-mobile systems (our production kernels and
This is prima facie false. Linux has been successful in a lot of consumer devices, like set-top boxes, e-readers, network routers, and other devices, without forking. All of those devices need power management.
Personally, I think that the reason why Android has been successful, where things like the Neo failed, is because of the technical and user-interface decisions they made in user-space. Manufacturers have also been willing to jump on board with Google because they know that Google will be around in a few years.
I guess if you decide to fork the kernel, it means that there will be a lot more work for kernel hackers at Google and at certain manufacturers. In addition to the 10% time you can add the 50% "backporting patches from Linus' tree" time. I just feel sad that those people won't be working on something more productive than reconciling two trees which really needn't have been separate.
Greg Kroah-Hartman: Android and the Linux kernel community Posted Feb 4, 2010 6:32 UTC (Thu) by error27 (subscriber, #8346) [Link]
Chris that's a shockingly wrong response. Also it was full of crap and flame bait. It's like a gas leak at a waste treatment plant.
Greg Kroah-Hartman: Android and the Linux kernel community Posted Feb 4, 2010 11:48 UTC (Thu) by daniels (subscriber, #16193) [Link]
Sure, there are quite a few ARM/SoC forks, and there are a lot of very unhappy users facing a lot of problems that will probably never be solved, or will be solved with extensive backporting.
There are also (and check linux-arm if you don't believe me) a lot of SoCs that work flawlessly, out of the box, with stock kernels. The OMAP series is used quite extensively, and the linux-omap tree is very active and frequently merged back into the mainline; I believe that linux-2.6 works out of the box on the N900, with no particular disadvantage, and certainly no regressions in power management.
Greg Kroah-Hartman: Android and the Linux kernel community Posted Feb 7, 2010 9:17 UTC (Sun) by swetland (subscriber, #63414) [Link]
It took a few years for the omap2/omap3 stuff to actually get to mainline and stabilize. I've been watching that process since we started on Android in 2005. I fully believe we'll get there with other SoC support and perhaps even with some of the generic Android drivers, but it's not going to happen overnight.
Greg Kroah-Hartman: Android and the Linux kernel community Posted Feb 3, 2010 18:32 UTC (Wed) by mfuzzey (subscriber, #57966) [Link]
Hum just thinking out loud here but seeing as Android itself is open source how hard would it be to fork android (or probably just the interface between the kernel and the android API) so that it would run on a vanilla kernel?
As Greg mentions it should be possible to support the standard kernel interfaces without touching the high level APIs and breaking all the apps out there.
Greg Kroah-Hartman: Android and the Linux kernel community Posted Feb 3, 2010 18:54 UTC (Wed) by cdibona (subscriber, #13739) [Link]
It's totally possible. I've actually been surprised that no one has popped
android on top of other kernels, but maybe they have and we've just not see it ship yet...
Greg Kroah-Hartman: Android and the Linux kernel community Posted Feb 5, 2010 6:56 UTC (Fri) by suzhe (guest, #63379) [Link] Something like these?Android on Ubuntu Android on Moblin
Greg Kroah-Hartman: Android and the Linux kernel community Posted Feb 3, 2010 18:57 UTC (Wed) by dberlin (subscriber, #24694) [Link]
Given the number of ARM forks relative to everything else, it is very hard to
believe the main problem here is anything but a serious disconnect between what the mainline kernel people believe the ARM folks need and what the ARM folks believe they need. It is very hard to see how this should be blamed entirely on the folks doing the embedded work, as easy as it is to say "hey our process works for everyone else". In reality, it's closer to "hey we've browbeaten everyone else into working with our process".
Things like having to continually churn API's, rewrite things in ways that
So i'd bet the burden of having to integrate with mainline is probably much
At least to me, complaining that they are making it so others can't
Greg Kroah-Hartman: Android and the Linux kernel community Posted Feb 3, 2010 19:47 UTC (Wed) by dlang (✭ supporter ✭, #313) [Link]
The ARM people do not like the fact that they have every board ever produced maintained as if it was a separate cpu architecture.
They have a solution in mind for this (a config table to separate the system config from the kernel architecture) and are moving on it.
the proliferation of ARM sub architectures is a major problem.
in the x86 world, imagine that every motherboard was a separate architecture, with devices at different addresses and supporting different features. This actually happened back in the 8086 and 80286 days, but the marketplace demanded compatibility so that things developed for one system could run on others with similar or better capabilities.
in the embedded space, everyone starts off with the idea that the product is a one-off, but linux has now been used long enough that companies are starting to come out with 2nd and 3rd generation products, which take the same basic design, update to current chips, and add new features. In this process they are realizing that there is value in having things standardized, even if it that standardization is just having a standard way to define the non-standard ways that standard components are connected to the system.
Greg Kroah-Hartman: Android and the Linux kernel community Posted Feb 3, 2010 21:04 UTC (Wed) by morrildl (guest, #63330) [Link]
> the proliferation of ARM sub architectures is a major problem.
Boy howdy, you can say THAT again.
Greg Kroah-Hartman: Android and the Linux kernel community Posted Feb 4, 2010 7:38 UTC (Thu) by cdibona (subscriber, #13739) [Link]
And again. And again. And again, for many years to come.
Greg Kroah-Hartman: Android and the Linux kernel community Posted Feb 4, 2010 20:23 UTC (Thu) by rahvin (subscriber, #16953) [Link]
But the beauty of getting it all into the Mainline kernel is that switching vendors becomes trivial. And the best part is that you can tell the hardware vendor to do the work to get it into the kernel before you ever even consider using the hardware. Not only can the vendor produce better code for their hardware, but once it's all upstream it's trivial to support if you are using mainline Kernels.
It's a seriously big mistake Google is making by forking. The long term effort will grow exponentially over time until the effort to maintain will exceed the benefit at which point the company will dump the project. If you don't believe me, look at Redhat's financial reports and see how many millions they spend to back port security fixes to kernel versions that are still in their long term support tree. Although the average phone isn't very old there are people that keep them until they break (I've seen 5-6 years old personally). RedHat spends millions keeping 3 year old kernels workable, when you have to support kernels that are that old you will see the real expense of the path you are taking. What I fear is that once you get to that point google will just decide it's not worth it anymore and abandon the whole thing. I actually think that's pretty likely personally.
Your posts make it apparent you don't realize the long term consequence of forking, particularly something as massive and important as the kernel. When Verizon or ATT sues Google for billions because someone used a known security vulnerability in an Andriod kernel that some cracker used to shut down the cellular system (and as a consequence someone couldn't dial 911 and then died) how will you defend the path you've chosen when you sit on the witness stand?
Some clarification on "the Android Kernel" Posted Feb 7, 2010 8:48 UTC (Sun) by swetland (subscriber, #63414) [Link]
There seems to be a lot of confusion here.
We maintain a set of patches on top of Linux, which we periodically rebase to the latest released Linux kernel. We've been doing this roughly every other kernel release since about 2.6.14. This week we're finalizing our move to 2.6.32 for the Android "Froyo" release, and we'll likely be on .33 or .34 for "Gingerbread".
These patches fall roughly into four buckets:
We've been publishing this code since fall of 2007, when the Android SDK was first released. We try to do almost all our work in the open (for generic drivers and SoC support), with only very product/board specific drivers being delayed until products are announced or shipped.
We certainly would love to get stuff upstream, but it's most useful if we can actually build upstream so it'll work with the Android userspace otherwise we're stucking maintaining patches for that anyway.
We realize, of course, that not everything we do is going to fit with the upstream view of the world, but I think this is a two way street -- being simply told "throw that all out and start over" is perhaps not the best way to encourage people to contribute.
- Brian
Some clarification on "the Android Kernel" Posted Feb 7, 2010 11:44 UTC (Sun) by rahulsundaram (subscriber, #21946) [Link]
It is your decision to ship with changes that has not been send upstream
before the product even ships and long after that as well and it is often the case that if there are userspace dependencies on such out of tree patches it will come back to bite you
Even now code is being dumped over all the wall in large chunks rather than
Some clarification on "the Android Kernel" Posted Feb 8, 2010 6:04 UTC (Mon) by swetland (subscriber, #63414) [Link]
Shipping great products will *always* be higher priority than getting full buy-in from upstream or getting all patches reviewed. There's no way we can tie our ship schedules to an external process like that and accomplish the hard deadlines involved in shipping successful consumer electronics / mobile devices. Realistically, we'll always ship based on at least a couple-month-old kernel due to the time involved in QA, regulatory testing, and qualification for these kinds of devices.
That said, keeping up to date with mainline is extremely important to us -- thus we've rebased many times between 2.6.14 (first internal kernel work) and 2.6.32 (latest), to track mainline, as opposed to sitting on some years-old release forever.
And the best way to keep up to date is just to be in the mainline kernel, so we want to get there and will work to get there, as we can, around other schedule, staffing, etc limitations that often make it difficult.
Some clarification on "the Android Kernel" Posted Feb 18, 2011 6:00 UTC (Fri) by tchalvak (guest, #72984) [Link]
Thank you for explaining the necessity of your approach. With a goal like android phones in mind, I expect it is certainly necessary to make some compromises, it's just a question of how short the turn-around time can be to compensate for compromises.
I felt compelled to create an account to comment on the unfortunate level of hostility and lack of willingness to start useful, civil dialog ppreviously shown in this thread.
Android (from a user perspective) is a breath of fresh air in the operating system ecosystem, and I eagerly await the day that it becomes integrated more fully with desktop linux, so that everyone gets the benefit of both. I expect willingness to work together on what may be difficult technical solutions is the only thing that will get us there.
--posted via my android phone
Greg Kroah-Hartman: Android and the Linux kernel community Posted Feb 3, 2010 23:48 UTC (Wed) by benh (guest, #43720) [Link]
Note that we solved that beautifully on PowerPC using the Open Firmware
device-tree model which we generalized to embedded platforms by disconnecting it from OF itself, and defining a flat representation (ie self-contained blob) that can be carried around from FW or wrapper to kernel.
It's a much more powerful approach than everything I've seen done so far for
I know some people are looking at porting to ARM today, we'll see how that
Greg Kroah-Hartman: Android and the Linux kernel community Posted Feb 4, 2010 0:15 UTC (Thu) by dlang (✭ supporter ✭, #313) [Link]
unfortunantly ARM didn't have a standardized firmware to build off of.
that being said, it sounds like what they are trying to move to is very similar, just without the depdnancy on firmware (a device tree that can be passed in as a boot parameter)
Greg Kroah-Hartman: Android and the Linux kernel community Posted Feb 4, 2010 9:08 UTC (Thu) by mfuzzey (subscriber, #57966) [Link]
>in the x86 world, imagine that every motherboard was a separate architecture, with devices at different addresses and supporting different features.
I work a bit on ARM and I think the problem is somewhat exagerated. Sure it is complicated compared to x86 but without a common hardware architecture (which I don't see ever coming for embedded devices) its not likely to improve much.
>the proliferation of ARM sub architectures is a major problem.
If your SoC is already supported you only actually have to write one board specific C file, which is basically a set of platform data tables mapping memory addresses, IRQs etc. If the device tree stuff gets merged it will be even easier though not IMHO fundementally different.
>in the embedded space, everyone starts off with the idea that the product is a one-off, but linux has now been used long enough that companies are starting to come out with 2nd and 3rd generation products, which take the same basic design, update to current chips, and add new features.
If you're doing a one shot project with a short lifecycle it may not matter but if you either
One problem is that the kernel patch submission and review timing may be incompatible with your product release cycle. But that just means that you keep the patches in your local tree while waiting for them to be accepted upstream. This way your local patch stack should diminish over time.
Greg Kroah-Hartman: Android and the Linux kernel community Posted Feb 3, 2010 22:09 UTC (Wed) by dcbw (guest, #50562) [Link]
"Given the number of ARM forks relative to everything else, it is very hard to believe the main problem here is anything but a serious disconnect
between what the mainline kernel people believe the ARM folks need and what the ARM folks believe they need."
You seem to be lumping all "ARM folks" together. Nokia seems to be doing just fine (more or less) with their OMAP patches, getting core arch stuff and drivers upstream.
Greg Kroah-Hartman: Android and the Linux kernel community Posted Feb 4, 2010 10:51 UTC (Thu) by istoppa (guest, #25836) [Link]
Getting power management working decently on N900 has not been an easy task and trying to do it in a way that would be acceptable upstream has not made it easier.
However I must say that this has been mostly due to the fact that TI in the first place should have been much more active.
And it gets even more difficult to prove to upper management that going with the community is the right way, when a competitor comes out quickly by leveraging code that has taken time and money to get accepted.
I think the fact that the Android code has been kicked out finally proves the need to play by the rules also for big companies and that even if some bending backward might be needed in order to deliver products, eventually the open source activity cannot be omitted or indefinitely postponed.
Btw, usually interacting with the community also gives significant advantage in terms of getting advice from highly skilled people on how to deliver a better product, so it should rather be sought after, not avoided.
Said this, I have perceived the lack of better participation from Android as a big loss, since it would have been much more productive for all the sides to have open dialog and confrontation about something that, after all, should be taken as granted (good use time) and not become a differentiating feature (we already have userspace eyecandies for that).
Greg Kroah-Hartman: Android and the Linux kernel community Posted Feb 7, 2010 19:10 UTC (Sun) by vikrampandita (guest, #53239) [Link]
>However I must say that this has been mostly due to the fact that
Agree. There sure was slowness from TI, thats changing now:
* Kevin Hilman works with TI as PM maintainer getting most features
* Things improved since 2008 as k.org statistics show with @ti.com contributions:
TI realizes the value of upstream and has a long way to go,
Greg Kroah-Hartman: Android and the Linux kernel community Posted Feb 4, 2010 11:39 UTC (Thu) by broonie (subscriber, #7078) [Link]
Not really - it's a culture thing, originating from the chip vendors in the most part.
The overwhelming majority of the problem ARM faces is that most of the CPU vendors have traditionally seen Linux as a traditional embedded OS that you produce a BSP for with your own custom stuff in it and don't contribute to. This means that the core code required to boot the processor from one of these systems is non-mainline, and few of the companies building products have the time or resource to mainline it. The end result is that you end up with communities coalescing around the vendor BSPs which are often based on old kernels which make it difficult to share anything with mainline.
This is getting a lot better these days - many of the major SoC vendors are contributing their ports upstream.
I know how Google feel Posted Feb 3, 2010 21:29 UTC (Wed) by hipparchus2000 (guest, #63337) [Link]
I've used Linux a lot over the years, and the 2.6 kernel is the most unstable one to date.
Every time I update a machine, part of the hardware stops working. For example, I just updated the kernel on a laptop, and the PS2 trackpad stopped working. Why? As a result of this, I no longer use linux. This has got much worse from 2.4 to 2.6 kernel.
I really don't understand why modules can't just have a standard ABI like any other OS, and just
Well it's up to the Linux kernel guys. Either they want to keep ignoring everyone, or have the kernel
I know how Google feel Posted Feb 3, 2010 22:00 UTC (Wed) by dlang (✭ supporter ✭, #313) [Link]
the number of installations of linux (and ubuntu) keeps going up. it keeps being used in more places.
this is hard to reconcile with your 'linux is collapsing' claim.
I've also been using linux for a long time (since 1994) and I have been using the vanilla kernels the entire time (I would call them the kernel.org kernels, but I was using them before kernel.org existed)
In my opinion the current kernel releases are, on average, better than they have ever been. once in a while there are problem releases, but they are rare.
I've always thought free software was a good thing Posted Feb 3, 2010 22:51 UTC (Wed) by hipparchus2000 (guest, #63337) [Link]
I just don't think the linux kernel is well managed - how can a point release break drivers that
shouldn't need work: how old is PS2???
I think it's a poor decision not to have an ABI for drivers and this goes back to Linus when he first
I've always thought free software was a good thing Posted Feb 4, 2010 0:34 UTC (Thu) by Russ.Dill@gmail.com (subscriber, #52805) [Link]
Actually, its not PS/2. Its the embedded controller (EC) in your laptop running a bunch of firmware and doing ACPI things. Your EC just so happens to be a bit touchy. In an effort to fix problems on another set of machines with EC's that are touchy in a different way, it broke a small set of other machines including yours. Its fixed in one of the 2.6.32 stable releases. You have touchy hardware, cope.
Kernel API churn is a *good* thing Posted Feb 4, 2010 18:18 UTC (Thu) by prl (guest, #44893) [Link]
Really.
The Linux kernel occasionally does need to have a subsystem ripped out and re-implemented properly when an API is just plan WRONG (e.g. the Big Kernel Lock). And sometimes it becomes clear that a new API is needed (e.g. multiple linked list implementations reduced to one).
When an API change happens, every single use can be identified and ported. There'll be a transition phase and then the old API will be retired. Admittedly, in the case of the BKL, the transition phase is taking a decade or so, but its removal is undoubtedly a good thing.
There's no build up of crud. Yes, it makes out-of-tree maintenance harder; all the more reason to get ones code merged. And yes, it goes back to Linus and it's one of the best decisions about the development process that he ever made.
Sorry to hear you had a problem with picky hardware, but a regression like this is nothing at all to do with the kernel API.
I know how Google feel Posted Feb 4, 2010 5:47 UTC (Thu) by jmm82 (guest, #59425) [Link]
I first used Linux in 2000 and it took a computer scientist to set one up
at the time. It is almost to the point where my girlfriend can use it. Now I would say when I do a new install one or two things may break from year to year(I can normally fix it though), but the code is relatively stable.
Hardware support has definitely gotten better and hardware companies
In the embedded world it is common to fork a version of Linux and add
One reason I feel Android is getting so much attention is that millions of
The New Era of Big Company forks. Posted Feb 8, 2010 15:01 UTC (Mon) by bkuhn (subscriber, #58642) [Link] It seems to me we've entered a new era of FLOSS development. Big companies that are otherwise supportive of FLOSS in many situations (i.e., Google), can afford to maintain a public fork for as long as they wish. I've written a blog post about this problem.
|
|||||||||||||