Recent Features
| |||||||||||||
Android 4.1 in AOSPAndroid 4.1 in AOSPPosted Jul 10, 2012 22:05 UTC (Tue) by ajross (subscriber, #4563)In reply to: Android 4.1 in AOSP by karim Parent article: Android 4.1 in AOSP
Android desperately needs package management. The idea of building the whole thing at once just doesn't scale to projects this size. Simply parsing the makefiles alone (here's a fun trick: run make -d and count all the includes) is 30+ seconds on my laptop. My day job moved from MeeGo to Android; I never thought I'd say this, but I really miss RPM/yum/OBS...
(Log in to post comments)
Android 4.1 in AOSP Posted Jul 10, 2012 23:06 UTC (Tue) by karim (subscriber, #114) [Link]
Full JB build: 75min
Full ICS build: 55min Full GB build: 20min
All on same HW: quad core i7 w/ 8GB or RAM, RAID1 HDs.
FWIW, the packaging thing was attempted by Sony-Ericsson:
Android 4.1 in AOSP Posted Jul 10, 2012 23:35 UTC (Tue) by xorbe (subscriber, #3165) [Link]
So then, KC will take 90 minutes ...
Android 4.1 in AOSP Posted Jul 11, 2012 13:06 UTC (Wed) by markhb (guest, #1003) [Link]
I believe it's KLP (Key Lime Pie).
Android 4.1 in AOSP Posted Jul 11, 2012 13:13 UTC (Wed) by karim (subscriber, #114) [Link]
Indeed, KLP is what it's rumored to be.
Android 4.1 in AOSP Posted Jul 11, 2012 14:13 UTC (Wed) by mikemol (subscriber, #83507) [Link]
Are you doing anything with parallel make?
I haven't tried building Android, but I prefer "--jobs=2*${num_of_CPU_cores} --load=${num_of_CPU_cores}"
Works absolute wonders in any decent make-based build system.
Android 4.1 in AOSP Posted Jul 11, 2012 14:15 UTC (Wed) by karim (subscriber, #114) [Link]
This is -j16
The only escape is CCACHE. It'll cut those numbers by 50%. But I've seen it do funky stuff and I've learned to distrust it.
Android 4.1 in AOSP Posted Jul 11, 2012 14:29 UTC (Wed) by mikemol (subscriber, #83507) [Link]
On an eight-core (no SMT) system, I have "-j16 --load=8". I've found it works much better than a straight -j16 or "-j --load=8"; I keep my processors busy, and they're compiling instead of context-switching.
Android 4.1 in AOSP Posted Jul 11, 2012 14:31 UTC (Wed) by karim (subscriber, #114) [Link]
Interesting. I'll have to try that. Thanks for the heads-up.
Android 4.1 in AOSP Posted Jul 11, 2012 14:39 UTC (Wed) by mikemol (subscriber, #83507) [Link]
Load-based management is also very nice if you use distcc and gigabit ethernet; anything running on a remote box doesn't impact your local load. :)
Android 4.1 in AOSP Posted Jul 11, 2012 16:56 UTC (Wed) by rgfernandes (guest, #78238) [Link]
I have made some benchmarks.
Here in my work we have a server with 24 GB RAM and 2 Xeons with 6 cores each.
time make -j 24 -l 12
real 26m9.001s
time make -j 24
real 17m58.537s
time make -j 12
real 21m1.784s
Android 4.1 in AOSP Posted Jul 11, 2012 16:57 UTC (Wed) by rgfernandes (guest, #78238) [Link]
I forgot to say: Android 4.0.3
Android 4.1 in AOSP Posted Jul 11, 2012 17:07 UTC (Wed) by mikemol (subscriber, #83507) [Link]
Interesting. I wonder why the difference in behaviors between your system and mine. Cache size? I/O? Do you have I'll try to get back with some benchmarks of my own later this week.
FWIW, my rough stats are:
Android 4.1 in AOSP Posted Jul 11, 2012 17:32 UTC (Wed) by rgfernandes (guest, #78238) [Link]
The server use a RAID 5 with SCSI disks.
Maybe it is the disks.
Android 4.1 in AOSP Posted Jul 11, 2012 17:35 UTC (Wed) by karim (subscriber, #114) [Link]
Highly likely. That's why JBQ had been suggesting lots of RAM. You might as well have the entire tree in the fs cache in RAM ...
Android 4.1 in AOSP Posted Jul 11, 2012 17:35 UTC (Wed) by mikemol (subscriber, #83507) [Link]
Could be. I've got RAID-grade SATA disks, but I don't remember if I'm building on top of them right now or not. Either way, disk cache may mute the effect.
What do you have for RAM?
Android 4.1 in AOSP Posted Jul 12, 2012 19:30 UTC (Thu) by kugel (subscriber, #70540) [Link]
Is this make? My make has no --load parameter.
Android 4.1 in AOSP Posted Jul 12, 2012 19:33 UTC (Thu) by mikemol (subscriber, #83507) [Link]
Looks like I'm running GNU make 3.82-r1 on Gentoo. Which version of make are you running?
Android 4.1 in AOSP Posted Jul 12, 2012 20:18 UTC (Thu) by jimparis (subscriber, #38647) [Link]
It looks like the "-l LOAD" option was added to GNU Make version 3.28, which was released in 1988. So if you don't have it, you're probably not using GNU Make. :)
Android 4.1 in AOSP Posted Jul 12, 2012 20:02 UTC (Thu) by ABCD (subscriber, #53650) [Link] The parameter is actually --load-average, but GNU make (like most GNU software) allows you to shorten long options so long as they remain unambiguous.
Android 4.1 in AOSP Posted Jul 12, 2012 20:03 UTC (Thu) by mikemol (subscriber, #83507) [Link]
Good to know. I'll be sure to use the full-length form in the future, for clarity.
Android 4.1 in AOSP Posted Jul 11, 2012 16:21 UTC (Wed) by xxiao (subscriber, #9631) [Link]
Android 4.0.4 32 minutes
Android 4.1.1-r1 49 minutes.
About 50% longer.
Android 4.1 in AOSP Posted Jul 11, 2012 0:42 UTC (Wed) by cmccabe (guest, #60281) [Link]
It's been a while since I worked on Android. However, from what I remember, you definitely do not have to build the whole thing at once. There are various ways around it. adb shell will let you copy single files over to the device, and fastboot will let you boot to a new kernel you've transferred without updating everything else.
Anyway, the autoconf / automake phase of most single application builds on Linux can take upwards of 30 seconds, even on a new PC. So I don't really think Google is doing too badly there. Maybe it's time for a new PC.
Android 4.1 in AOSP Posted Jul 11, 2012 8:06 UTC (Wed) by rvfh (subscriber, #31018) [Link]
Yes indeed, out of memory:
source build/envsetup.sh
Android 4.1 in AOSP Posted Jul 11, 2012 15:35 UTC (Wed) by ajross (subscriber, #4563) [Link]
Except that if you start a new tree for any reason (and when doing integration work, this is routine) the only way to bootstrap it is with a full build of a 9+GB tree. Once built, you have some options (though frankly they're pretty poor in comparison to an RPM). But there's no escaping the huge bootstrap, and it's an immense pain that simply doesn't exist in the package management world.
Contrast this to a desktop distro, where if I want to cook up a special version of some package or another I can just do it in isolation (obviously with some caveats: changing runtime compatibility still requires full dependency builds, etc...). At no point does Fedora make make me build the whole world just to install an image with a new kernel or X server. Things like glibc and coreutils and gnome are already built (and signed!) and ready for use via a trivial download.
Android 4.1 in AOSP Posted Jul 11, 2012 17:01 UTC (Wed) by cmccabe (guest, #60281) [Link]
Well, Android doesn't make you build the whole world to install a new kernel either. As I mentioned, you can just build the kernel and then boot to it via fastboot. Similar solutions exist for the other components, too.
I used to keep around a bunch of trees checked out on different branches so that I could do things quickly without building everything from scratch. It's no different than any other software project, that way.
I guess the major difference between Android and, say, Red Hat is that RH has package management, whereas Android does not. So on Red Hat, it's more practical for you to have a build server creating all the RPMs and all the developers downloading that.
From a developer perspective, using someone else's build is problematic. Did they use the same CFLAGS and GCC version as you? How about the c++ compiler (Android has a bunch of C++)? It's easier just to build everything yourself and avoid all the "my X doesn't work with your Y" bugs that full-fledged Linux distributions deal with.
Android 4.1 in AOSP Posted Jul 11, 2012 18:57 UTC (Wed) by alankila (subscriber, #47141) [Link]
I'm personally pleased about the fact that android only manages end-user programs with its package manager, and also avoids the autotools suite. Android seems to do quite fine without either. However, it would be convenient to be able to build & deploy new binaries on device using the build system; maybe instead of generating files under out/ there should be a way to make it install directly to a device instead.
When developing some system software, I work out the files that are generated by the "mmm" command and then run a bunch of &&-combined shell commands to push the new files on device and restart the affected daemons so that they see the new copies of the files. It's good enough a solution for me in practice, but some of these steps could still be automated.
|
|||||||||||||