the problem is that one person's 'bugfix' is another person's 'new feature'
especially when the bugfix can end up refactoring the code in the process.
yes, this is a big problem with Linux, but the rate of fixes (of all kinds) is the great strength of Linux. At this point nobody knows how to fix the weakness without giving up the strength. There are other OS groups (openBSD comes to mind) that seem like they follow the philosophy that you are advocating, but despite the fact that they had several years of a head start on Linux, their development models have caused them to be far less useful on current hardware. (and therefor any security benefits they may provide, far less useful)
I don't understand your comment about the kernel developers being unable to provide shared access to a single kernel image.
are you referring to the fact that there was a privilege escalation vulnerability on kernel.org? if so, any conclusions about what the problem was need to wait until we learn what happened. And in any case, the vast majority of the kernel developers were not involved in administering the systems (and note that it was several systems, not a single system)