LWN.net Logo

Sorry, but this is not so straightforward

Sorry, but this is not so straightforward

Posted Nov 7, 2011 17:55 UTC (Mon) by khim (subscriber, #9252)
In reply to: This is attitude Apple is using - and they DO have users, strangely enough... by pboddie
Parent article: Rawhide gets GNOME Shell for all display types

When people are so motivated to write down their frustrations (see elsewhere in this discussion for a really good example), this is material that people in the "customer satisfaction" business would pay good money for: you have someone legitimately trying to offer feedback, even solutions, that no tick-box survey with bland questions and a 1-to-4 scale can ever attempt to match.

Surveys with checkboxes are just a way to show that you do something. I've participated in the usability studies and the rule #1 is: never trust the user. Never. If ask him or her to do something (for example find the document and print it), you videotape his or her efforts and then you analyze these videos. If you'll ask why s/he spent so much time looking for the printer icon you'll hear that said icon is of unfamiliar shape or size or something equally nonsensical, but if you'll take a look on the video you'll see that in reality s/he tried to print using "Print Preview" panel - and this panel just does not have such icon at all! Obvious problem, but not even close to what you'll hear from user.

I'm not saying that GNOME developers did many studies (probably not: there are obvious problems in GNOME Shell), but the fact that they ignore loud complains is fine - these are not a good sources of UX ideas anyway.

P.S. I remember one stricking example: one [relatively large] company did a usability study related to Office switch. They used Office 2003 and the question was: should they go with Office 2007 (with it's all-new ribbon interface) or with OfficeOffice.org? First they did 5-minutes test (Office 2007 and OpenOffice.org were just shown and the task was just to print "hello, world" on printer) - and OpenOffice.org was clear winner: 80% liked it more then MS Office 2007. Then they did few more tests - and results were disastrous for OpenOffice.org. It was hated now and failure rate was much higher. Why? Because underlying technology means more then buttons. One example: when you need to insert page number and use appropriate item in menu of Office 2003 or on ribbon of Office 2007 the reaction is the same: you are given choice - do you need to put it on the top or on the bottom of page. When you do the same thing with OpenOffice.org you can page number in the middle of page. To do what any sane user will want (put number in header or footer) you must go and manually create header or footer, switch to it and then insert page number. Great idea from mathematical abstraction POV, but the end result - over 50% failure rate.


(Log in to post comments)

Sorry, but this is not so straightforward

Posted Nov 7, 2011 18:22 UTC (Mon) by pboddie (subscriber, #50784) [Link]

Surveys with checkboxes are just a way to show that you do something. I've participated in the usability studies and the rule #1 is: never trust the user.

Saying things like this just gives people the wrong idea: "This guy on LWN said that you can't trust the user, so shut up, users!". In fact, you can and should trust the user when they tell you they can't do something, but you can't rely on them to accurately report what went wrong.

Again, when people give you qualitative feedback of a reasonable quality, it is frequently of far higher value than, "How easy did you find it to launch Firefox? (1 = very hard, 4 = very easy)" And even people's "wrong" descriptions of their experiences are enlightening to anyone willing to listen. Telling users that they "don't get the conceptual metaphor" or whatever conveniently avoids the issue of why they didn't "get" it.

Copyright © 2013, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds