Interesting. Somehow until now I had missed the "to give any third party, ..., copy of the corresponding source code"
Approx. equivalent words are provided in GPLv2 and GPLv3.
So if one distributes a received binary, without source, and without including the author's offer, then one must give the source code to *anyone* who asks for it. Somehow I had imagined that one only had to provide source to those to whom one had provided a binary.
I guess this makes the GPL more aggressive, so encourages people to provide source up front. But it also means that one should be careful to tick the right boxes when compiling source to make a binary for anyone.