The 2013 "State of X.Org" report

I don't understand how you could expect the X server to 'imitate' Linux's architecture of a complicated core and relatively simple drivers. An X server simply does a different job: it's not going to have that architecture any more than, say, a compiler would. You can't "just" totally rearchitect software without reimplementing it completely anyway (which is, surprise, what Wayland is doing).