> IANAL, but I don't think that can happen. It seems to me that in the above scenario, the manufacturer would be in violation of the GPLv3.
You're assuming that the source request is addressed to the manufacturer. What if the source request is addressed to Ubuntu? They did choose to make a GPL code release, and they do have the key that's a necessary part of the installation instructions for that release. IANAL either, but I think they would be in violation if they chose not to answer.