Except the "application-specific passwords" are not application or device specific. They never get locked down to be for the application with which they're first used.
So if you create 3 app-specific passwords, you've created three passwords which have access to any of your Google services, bypassing the two-factor auth you may have setup. Sure, Google's auth system will only show you those passwords once, but if they're stolen from a device, game over.
Until Google start locking down those passwords so that they're only valid for accessing the service with which they're first used, they're a gaping hole; their only redeeming attribute is that the passwords have better entropy than most humans will typically choose.
Ideally, the app-specific passwords would have a checkbox, "needs to be a secure signature upon this password", so that a signing-key stored in a TPM could sign the password, nonce and a timestamp, then send all but the password. Publish the algorithm. Then the unchecked passwords are still as exposed, so if you're using a third-party app you have a problem, but anyone writing software with custom support could get something halfway decent.
But by this point you're half-way towards OAUTH and long-term issued credentials anyway. So don't do the custom schemes but accept the need to implement the crazy.