"They show you precisely what folks actually get stuck on, as opposed to what they say they get stuck on, or what you think they will get stuck on."
In theory that is what they should do. With enough funding, the right design and careful execution, they can do that. It's hard but not impossible.
But like studies in other fields, they can also give you the results you want, cheaply, quickly, and easily. Good design is difficult, and good experiments can get expensive. And who wants to waste a lot of time and money on a more elaborate experiment than they need to do before they get on with the fun part?
With unlimited resources and a lot of creativity and imagination you might make mockups of a few hundred different systems, built around different design assumptions, following users long term... yeah no one does that. Yet everyone seems to be able to draw from the much more limited work that's been done absolute certainty for their own preferred theories.
So usability studies are no panacea. They need to be looked at skeptically and yes, both designed and evaluated using common sense if you can find any. It seems to be quite rare though.
Copyright © 2018, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds