"This fallback set could be defined with some enormously complex config file"
You decided to ignore the army of fairies that would be required to maintain such a config file. This kind of static aliasing design only ever works when your font offering is static and limited, which is not the case on modern systems.
I want a pony too.
PS: pango and fontconfig have been heavily inspired by the CSS font resolution mechanism, so your "toy" interface is what powers the web today. 1:1 fontname <=> font file selection just does not scale