There is another reason why some people think that pushing crypto into the kernel is the right thing: It makes it easier to get a FIPS 140 validation.
One important topic there is to separate the code to handle the keys from the application. Using a validated library is not a real option because a process using a validated library still has full access to the code and data of that library. Thus it seems to them easier to define the crypto module boundary as the kernel mode - code and data living there is natuarally protected from any user process.
From a software engineering POV and also for easier auditing it would be far better to move the crypto code into a daemon which may then be validated. This will give a very nice and enforceable crypto module boundary. The claimed drawback is a performace penalty.
Of course this does not solve the general problem that the kernel has access to everything. But it offers a better migration path to some future (capability based) microkernel.