Putting directories outside the kernel also means that a whole pile of
things POSIX guarantees become, as near I can tell, impossible to provide.
I can't see any way to keep cross-directory rename() atomic, for instance.
Also it's a grotesque security hole: now you can't keep stuff secret by
hiding it in unreadable directories anymore.
Periodically there are proposals to introduce an open()-by-inode-number
syscall. They are always shot down. I don't know what sort of system
you're thinking of, but it isn't Unix.
(And if you're going to go that route, make the inums 1024 bits long and
bingo, you've got a capability-based system.)