the fact that the x86 was used on the most common platform meant that there was more money for speeding up the x86 chips, which made them more popular, which provided more money for speeding them up......
This is why small companies like Transmeta folded, they were compatible, but they didn't have the R&D budgets and manufacturing capability to compete with Intel. AMD is barely hanging on, and if Intel hadn't made the Itanioum blunder (leaving the gap open for the AMD-64 chips), I doubt if AMD would have survived.
network effects matter, when everyone is running binary software, being binary compatible matters. Since the IMB PC became the standard, any chips that weren't PC compatible became marginal and the popularity -> money -> R&C -> speed -> popularity cycle started.
With mobile devices NOT being x86 compatible, we are seeing a resurgence in competition at the architecture level again for consumer devices (enabled by Linux's cross platform support), and Microsoft and Intel have been trying for years to ignore and block this, but now they are having to really recognize the competition.