I think another reason for Asterisk being top of the vulnerabilities list is that it is a big target. There is *real money* for grabs by hacking phone accounts.
When I opened the SIP port in my firewall, I was inundated with password cracking attempts (mostly from IP addresses in Scotland, bizarrely).
I gave up on the well-known SIP port, and went with a non-standard, fail2ban covered port when I got bored listening to my honeypot SIP account receive forwarded calls.