It does not seem infeasible to compile and load a driver whenever a new device is detected or feature accessed for the first time. A bit slow, but still faster than downloading and installing an (often bloated) Windows driver.
Then, a full kernel compilation starts in the background. On the next boot a module-less kernel boots again.
All this looks technically feasible but, why the effort? Which problem would this solve?