The root cause is something solved by systemd, with the .include and the new directory system ( ie, using /etc/system/foo.unit.d/foo ).
Upstart config file are also harder to parse ( ie, this is a full blown DSL ), which doesn't help to create a tool to audit or modify them. While auditing is not something I ever seen, I know that some organisations requires this, so every roadblock that can be removed is better. And as someone involved into automated package checking, having a parsable format is better.
One last issue is the whole copyright assignation to Canonical, that AFAIK is always annoying since company exec tend to avoid letting people work on such projects ( and IMHO for good reasons ).
RHEL and Fedora did used Upstart for a time, Opensuse and Debian also looked at it at the same time. So if they collectively decided to not switch or to use something else, there is likely a technical reason.