I personally find it really handy to be able to unload FAT support and load a version with different NLS junk configured, or unload my wireless driver and load a version build with debugging when my access point does something new and different. (At times, I've had problems where the only way I knew to get my wireless driver to power up the device again was to remove the module so that it forgot it knew about the device, but it's better now.) I've also been glad to be able to patch my hfsplus driver to handle my weird ipod without rebooting. It's also nice to be able to say "no" to drivers for hotpluggable things I may want at some point, knowing I can just build them as modules if I end up wanting them. It's definitely worth having a kernel that will boot without modules, but it's useful to have everything that the system doesn't require (but the users will want) as a module.