Bionic development started in late 2005 / early 2006, iirc. We explored a number of alternatives before settling on "BSD-libc-based with homegrown syscall and thread glue" which has worked pretty well.
The scripted processing of headers is a result of hand-curation of headers not scaling well. The goal is (obviously, I thought!) to extract the necessary userspace interface to the kernel without picking up internal-only or incompatible-with-userspace headers or subsections. At the time we got started it appeared that other libc projects were mostly doing this by hand.
We'll want to take a look at exclusion methods used by "make headers_install" -- if they provide a way to further reduce the scope of exported headers that's definitely interesting.
We strongly discourage the use of #include <linux/...> in Android userspace code, unless it's code that's directly talking to a kernel interface defined in that space -- primarily to keep things clean and reasonably portable.