It's interesting to hear of cease-and-desist orders, certainly.
Again, farnz makes the crucial point upthread: it's all about the distribution. When you have someone distributing things to be used together as one system and where the nature of the GPL-licensed software signals the intent that anything taking advantage of that software must adhere to the conditions of the GPL, the "derived work" defence is merely something that can be used to limit the claims of the GPL if the relevant copyright law imposes restrictions on what licences can require of others.
So, if you take something which deliberately says that "you had better be able to release the source for all this" if you distribute your Linux system "with added Nvidia magic" and then combine it with the Nvidia drivers for which you don't have sources, then you really have to have a better argument than "they're not really touching (come on, your honour!)". After all, there are plenty of licence agreements that impose more peculiar and arbitrary restrictions on how works are used.