Kernel documentation with Sphinx, part 1: how we got here

long long way from Knuth's literate programming.

LP wasn't just about writing better comments. It also involved changing the order in which code was written so that ideas could be developed in an order that made sense to the human reader, often quite different to the order that the compiler wants.
This isn't just re-arranging function declarations. It might also mean writing a rough outline of a function with various "blanks", then filling in the blanks one by one after explaining them.

I think literate programing can work very well when the programmer fully understands the problem they are trying to solve and can then present it coherently as a lesson to the reader. A lesson which can be compiled and run to show that it works.
I don't think it works well at all for code which is being built by engineers who are coming to understand the problem as they go (most of us) and for whom the requirements change between the start and end of the project (though of course, that would never happen!).

I think that for code that is under development, having significant documentation in with the code is a mistake as it is very likely to become out of date quickly. Having documentation in with the code only makes sense (to me) once the code has stabilized. Then there is at least some chance that the documentation will be vaguely accurate for more than one day.

Certainly some people can make the effort to update documentation whenever they change the code. Both of the people who do that are worth their weight in gold and I respect them. But I doubt I could ever emulate them.