You appear to be perpetuating a common misunderstanding.
Absent the permissions provided in the license you are not legally permitted to do much with the covered work. If you want to perform some act reserved to the copyright holder (like copy the covered work with or without combining it with other things) then you may legally do so only by the good graces of the license.
This is the relevant hook. Quoting v2, "You are not required to accept this License, since you have not signed it. However, nothing else grants you permission to modify or distribute the Program or its derivative works. These actions are prohibited by law if you do not accept this License. Therefore, by modifying or _distributing the Program_ (or any work based on the Program), you indicate your acceptance of this License to do so, and _all its terms and conditions_ for copying, distributing or modifying the Program or works based on it." (em mine)
The FSF can't stop you from wearing party hats*, but they can write a no party hat condition into their license, and your only options are to decline to practice the copyright-exclusive acts (distributing the software), eschew party hats, or violate copyright (at your own legal peril).
Moreover, if it _didn't_ work this way if the license continued to permit copying when you violate its terms then the license could have no useful effect except creating a blanket permission. The GPL makes _many_ such requirements of normally permitted activities conditions for enjoying the license, e.g. offering the source code.
In the few cases where someone might link to GPLed code without ever subjecting themselves to the requirements of the GPL (e.g. by not distributing the covered work or engaging in other copyright-protected activities, including creating derivatives in the strict legal sense) then they can potentially get away with it. An example of this might be a proprietary software vendor linking a GPLed system library which they are careful to never distribute themselves.
*[A silly example; I would expect a court to invalidate such a term as having no meaningful connection to the licensed activity. The same could not be said about rules about how the software was employed]