Key pieces of X were written by individuals, not teams. I did the original Xlib (under extreme time pressure for X11); Bob did the device independent part of the X server up until X11, and so on.
The code base in that era was very much smaller.
A community large enough to provide useful code reviews did not exist in the early years of X... And security problems in that era were very small: the bad guys weren't out to get us in the 1980's and early 1990's, so there was little incentive to go find them or do such code reviews once there was a viable community.
And then things stagnated, particularly after the X Consortium closed. Long, long, long story there....
Unfortunately, many mistakes of my era were cut and pasted into later X extension libraries, replicating bugs. And other people created new bugs as X continued to evolve over the decades as well. Creativity is found everywhere ;-).