IIRC, there was also the fact that it installed in completely nonstandard directories, and the author was completely opposite to changing to a more normal directory scheme.
And the fact that qmail was a huge source of email backscatter (sending bounces to faked email addresses).
And the lack of AXFR/IXFR in djbdns (rsync, anyone?).
These are the complaints I heard the most about DJB's software. My own annoyance with it (besides the bounce spam) was the "million of tiny _invisible_ files (that is, all dotfiles) scattered all over the filesystem" model for configuration, which made administering an inherited mail server harder than it should be (of course, changing to postfix solved that for me); but I do not recall hearing many complaints about this particular point.
There are great ideas which came from DJB's software (Maildir, which solves the >From problem, is my favorite example), but other software quickly adopted them, negating any advantage for DJB's software. What was left was only the annoyances.