Unfortunately, the answer is not as simple as the question. Or, rather, it is simple, but it's a bit long. Please bear with me.
You know what? I know nothing about painting, and I'm sure I'd suck at it if I tried. I still want the freedom to paint the walls of my house with whatever colors and paint I choose, and I'm sure I'll be able to find a competent painter to paint it for me whenever I fancy. Wouldn't it suck to buy the house and then find out that, if you want it painted, you have to hire the very person who sold you the house to do that job? That's what you get with non-Free Software, be it an application, an operating system or firmware for some piece of hardware.
Now consider this: do you agree that freedom of the press is important? Most people do. But how many of them write on newspapers, or otherwise work in the press, actively exercising that freedom? You don't have to be a journalist and exercise the freedom yourself in order to benefit from the freedom of the press. You can *choose* to exercise this freedom, if it's there, but even if you don't, the fact that many others do makes things better for you. But if it isn't, everyone loses. Consider how silly our current debate would be, if taking place between a proponent of freedom of the press and a non-journalist who doesn't perceive the indirect value that this freedom affords her.
On to software. Even if you never even realize that an improvement to a piece of software could make your computer (think cell phone, portable media player, media recorder, watch, notebook, desktop, whatever) work better for you, odds are that others will, and if they have the freedom to do it and share it with others, odds are that the benefits will percolate to you as well. And, if you *do* realize an improvement that would help you, you have the freedom to learn how to program, or to lure a programmer to do the job for you (hiring, asking kindly, whatever).
So, you see, the I'm not a programmer argument is fatally flawed, it misses the forest and the ecosystem focusing on a single tree. What's more, it seems to be some kind of a mantra that the industry of control spreads unto people, making them believe that it knows better than ourselves what's best for us, promoting such myths as I need to hide this from you to protect you (security by obscurity) and you're not smart enough to do this job, close your eyes and trust me. Don't be a fool, don't believe these myths!
Why would you buy a piece of hardware from a company that treats you like dirt?
How does Regular Linux limit my freedoms, whereas Linux-libre does not?
Besides the point that you identified yourself, it promotes and endorses the horrible idea that it's normal and acceptable to give up control over certain parts of your computer, it gets people used to that. As the amazing and terrifying growth of the blobs required by Linux shows, this is a dangerous slippery slope. If we don't revert this trend, Linux and other low-level pieces of software will be increasingly kept as closely guarded secrets, and we'll be back to the point in which RMS and everyone else were 25 years ago: nearly all the software around them had turned proprietary. And this is happening again.
And, worse, it's not only happening in some small bits and pieces of Linux. Freedom is evaporating into the cloud. More and more phones (the most common computing platform nowadays) are using GNU/Linux or Android or BSD, but non-Free versions thereof. Non-Free through withholding of source code (iPhone's BSD-based system) or Tivoization of Linux and Symbian.
Now, of course these businesses want to keep people ignorant and incapable of perceiving the value of freedom: such people are much easier to control and lure with ooh, shiny traps and tricks, and they *will* buy pieces of hardware from companies that treat them like dirt, because they don't see there is an alternative, they don't see they can take part in shaping the alternative, they aren't willing to make a tiny little bit of sacrifice to shape it, but they are willing to fight *for* those who keep them hostages (did I mention Stockholm's Syndrome?), and they will sacrifice freedom for short-term convenience (echoes of Ben Franklin come to mind), convenience that will be lost as soon as those whom they gave up their freedom find it profitable to also take it away. Then you're left with what? Freedom to give up your freedom, to choose your master? That's no freedom! When you get to that point, you're already under control, your freedom is already lost.
See how Linux's stance of accepting, promoting and endorsing the slippery slope affects your freedom? Keep in mind is that promoting the software is not the same as promoting the freedom: when the promotion of the software involves software that takes the freedom away, the message of freedom is fundamentally lost.
See how Linux-libre helps you, your gradma and everyone else, by exposing the problem, helping people realize they're losing their freedom, and inviting people to vote with their wallets for hardware manufacturers to respect their customers? Even if you don't choose to run it on your current computer, it might help you choose a computer that respects you next time you go shopping, so that you can vote right with your wallet. And then, since you can, we can hope you will, and so will many others, and then we'll get the change we need, rather than change they can spare ;-)
(Was it just me that read change we need as something a bunch of street beggars might say, and that could be answered to their satisfaction with handful of coins? :-)