Handset cohabitation: Ubuntu for Android

Sharing kernel driver with android/X would seem a rather complex solution. And depends on the goodwill of the proprietary code vendors. Would seemingly make more sense to create xorg-server-android which exposes whatever 2d-acceleration/opengl-es/video codecs the android phone already supports.

As bonus point you could then sell the xserver as an independent Android app in the market for the small group of users who want to run X11 apps remotely on android.