under the covers, .htaccess files are not just access control files, they can contain any config options that can be in an apache config file, they just apply to that directory and it's subdirectories, discovered per-hit.
so yes, they are horribly inefficient
in terms of caching the combined contraints, that seems hard in the face of directories being moved around.
there's also the issue of the interaction with links and finguring out the 'true' path to a file.