ARM and defconfig files [LWN.net]

By Jake Edge
June 16, 2010

The kernel tree for the ARM architecture is large and fairly complicated. Because of the large number of ARM system-on-chip (SoC) variants, as well as different versions of the ARM CPU itself, there is something of a combinatorial explosion occurring in the architecture tree. That, in turn, led to something of an explosion from Linus Torvalds as he is getting tired of "pointless churn" in the tree.

A pull request from Daniel Walker for some updates to arch/arm/mach-msm was the proximate cause of Torvalds's unhappiness, but it goes deeper than that. He responded to Walker's request, by pointing out a problem he sees with ARM:

There's something wrong with ARM development. The amount of pure noise in the patches is incredibly annoying. Right now, ARM is already (despite me not reacting to some of the flood) 55% of all arch/ changes since 2.6.34, and it's all pointless churn in

	arch/arm/configs/
	arch/arm/mach-xyz
	arch/arm/plat-blah

and at a certain point in the merge window I simply could not find it in me to care about it any more.

He goes on to note that the majority of the diffs are "mind-deadening" because they aren't sensibly readable by humans. He further analyzes the problem by comparing the sizes of the x86 and ARM trees, with the latter being some 800K lines of "code"—roughly three times the size of x86. Of that, 200K lines are default config (i.e. defconfig) files for 170+ different SoCs. To Torvalds, those files are "pure garbage".

In fact, he is "actually considering just getting rid of all the 'defconfig' files entirely". Each of those files represents the configuration choices someone made when building a kernel for a specific ARM SoC, but keeping them around is just a waste, he said:

And I suspect that it really is best to just remove the existing defconfig files. People can see them in the history to pick up what the heck they did, but no way will any sane model ever look even _remotely_ like them, so they really aren't a useful basis for going forward.

Another problem that Torvalds identified is the proliferation of platform-specific drivers, which could very likely be combined into shared drivers in the drivers/ tree or coalesced in other ways. Basically, "we need somebody who cares, and doesn't just mindlessly aggregate all the crud". Ben Dooks agreed that there is a problem, but that "many of the big company players have yet to really see the necessity" of combining drivers. He also noted that at least some of the defconfig files were being used in automated build testing, but did agree that there are older defconfigs that should be culled.

Dooks also had a longer description of the problems that ARM maintainers have in trying to support so many different SoCs, while also trying to reduce the size and complexity of the sub-architecture trees. Essentially, the maintainers are swamped and "until it hits these big companies in the pocket it [is] very difficult to get them to actually pay" for cleaning up the ARM tree and keeping it clean in the future.

Because Torvalds said that he was planning to remove the ARM (and other) defconfig files, ARM maintainer Russell King posted a warning to the linux-arm-kernel mailing list:

Linus doesn't appear to be listening to reason, so I see now this as a fait accompli. It'll [apparently] happen at the next merge window.

So, don't send anything which contains a defconfig file or updates to it. It's pointless.

That set off a separate discussion on that mailing list—King's and others' attempts to redirect it back to linux-kernel notwithstanding—about ways to reduce the amount of mostly redundant information carried around in the defconfig files. Ryan Mallon is in favor of proactively eliminating some defconfigs, while others discussed various ways to only keep the deltas between the config files for various SoCs.

Based on Torvalds's comments on linux-kernel, some kind of delta scheme is unlikely to fly. His main complaint is that the defconfig files are neither readable nor writable by humans, as they are generated by various tools. He made some specific suggestions of alternatives that would still allow the generation of those config files, using Kconfig files that are usable by humans.

Reducing the number of defconfigs, as Mallon suggested, may be helpful, but King at least is convinced that it doesn't go far enough. He believes that Torvalds has already made up his mind to remove the defconfigs in the next merge window and that the ARM community better be ready with something else:

I believe the only acceptable solution is to get an [alternative] method in place - no matter what it is - and remove the all but one of the defconfig files from the mainline kernel. _And_, most importantly, kautobuild needs to be fixed so that we still get build coverage.

The loss of kautobuild is a major concern here, and I believe it trumps everything else for the next merge window. Kautobuild is an extremely important resource that we simply can not afford to lose.

The discussion ranged from possible solutions to the immediate defconfig problem to the larger issue of reducing the duplication throughout the ARM trees. There is an effort underway to produce a single kernel that would support multiple ARM platforms for Ubuntu 10.10, which will likely help consolidate various sub-architectures. Given that Canonical is working closely with the newly formed Linaro organization—founded to simplify ARM Linux—there is reason to believe that things will get better.

Meanwhile, though, back on linux-kernel, Torvalds started a new thread to flesh out his ideas for a hierarchical collection of Kconfig files that would essentially take the place of the defconfigs. After some back and forth, Torvalds gave an example of exactly what he is suggesting:

Let's say that I want a x86 configuration that has USB enabled. I can basically _ask_ the Kconfig machinery to generate that with something like this:

- create a "Mykconfig" file:

	config MYCONFIG
		bool
		default y
		select USB

	source arch/x86/Kconfig

and then I just do

	KBUILD_KCONFIG=Mykconfig make allnoconfig

and look what appears in the .config file.

He goes on to describe a theoretical Kconfig.omap3_evm file that sets the specific requirements for that platform and then includes Kconfig.omap3. That file sets up whatever is required for the OMAP3 platform and includes Kconfig.arm. That would allow developers or tools like kautobuild to generate the necessary config files without having to carry them around in the kernel tree. Those Kconfig files would also be much more readable and any diffs would be understandable, which is important to Torvalds.

That solves a significant subset of the problem, but there is still a fly in the ointment: dependencies. In Torvalds's example, CONFIG_USB requires CONFIG_USB_SUPPORT, so that would need to be added to Mykconfig. Not accounting for dependencies will get you a kernel that doesn't build or, worse yet, won't run correctly. There are a number of possible solutions to the dependency problem, though, ranging from Catalin Marinas's patch to track unmet dependencies of options used in select statements to Vegard Nossum's summer of code project to add a satisfiability solver into the configuration editors (menuconfig, etc.).

It certainly seems likely that defconfig files will be removed from the kernel tree in the 2.6.36 merge window. Whether there is another solution—based on Torvalds's ideas or something else—to replace them is really up to the architecture teams as Torvalds is perfectly happy to move on without them. ARM, PowerPC, MIPS, and others all have lots of defconfig files, but unless he changes his mind, they won't in a few short months. They can keep maintaining those files in a separate repository somewhere, or find an acceptable method to generate them. While it may be painful in the short term, it will reduce the size of the kernel tree and make Torvalds's job easier, both of which are worth striving for.

(Log in to post comments)