It's not to bad.
For example on my distro broadcom cards typically will have a firmware extraction kit that
will download the XP drivers and rip the firmware from them. Mostly automatically.
Just because they can't distribute it doesn't mean that they can't make it easy. It would be
easier to deal with binary firmware then, say, having a GUI button to download and compile
proprietary ATI or Nvidia drivers.
But ya it would be nice to have a standardized way to distribute stuff like that.