User: Password:
|
|
Subscribe / Log in / New account

Some Ubuntu Unity usability testing results

From:  Matthew Paul Thomas <mpt-Z7WLFzj8eWMS+FvcfC7Uqw-AT-public.gmane.org>
To:  ubuntu-desktop-nLRlyDuq1AZFpShjVBNYrg-AT-public.gmane.org, ubuntu-devel-nLRlyDuq1AZFpShjVBNYrg-AT-public.gmane.org
Subject:  Re: Default Desktop Experience for 11.04 - User testing results
Date:  Fri, 15 Apr 2011 03:00:31 +0100
Message-ID:  <4DA7A6BF.1030603@canonical.com>
Archive-link:  Article

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Rick Spencer wrote on 08/04/11 02:38:
>...
> Back at UDS for 11.04 in Orlando, Mark set the goal of using Unity by
> default on the Ubutu desktop. Given the current course of development,
> it appears that we are going to achieve this goal, and Unity will stay
> the default for 11.04.
> 
> I'm following up on this list at the suggestion of the Tech Board to
> give folks a chance to respond or escelate any concerns.
>...

Last week, Charline Poirier ran a user test of Unity, with 11 individual
participants. This week, I have helped Charline analyze the results.

In this summary, I have numbered the participants:
- - P1, 19, a student and Mac user
- - P2, 33, an administrator and Mac user
- - P3, 25, a student and Windows user
- - P4, 32, a teacher and Windows user
- - P5, 27, a compliance officer who uses both Windows and Mac
- - P7, 44, a life coach who uses both Windows and Ubuntu
- - P8, 30, an IT network manager and Windows user
- - P9, 22, a student and Windows user
- - P10, 21, a student and Windows user
- - P11, 47, a teacher and Windows user
- - P12, 34, an operations manager and Windows user.

The test machine was a Lenovo ThinkPad T410i running Ubuntu Natty with
unity 3.8.2-0ubuntu1 and compiz 1:0.9.4git20110322-0ubuntu5.

Charline asked each participant to try several tasks. Not every
participant had time to try every task.

*   Every participant who was asked understood most of the launcher
    items. P7 and P11 thought that "LibreOffice Calc" was a calculator,
    and P7 and P9 thought Ubuntu Software Center was the Recycle Bin.
    Nobody understood Ubuntu One. (The Classic session has much smaller
    icons for everything, but has a visible-by-default label plus an
    extra tooltip for each app.)

*   Almost everyone understood most of the indicators, but 4/11 people
    (P7, P9, P11, P12) thought the Me menu icon might be a close button.

*   11/11 people easily launched Firefox to check Web mail.

*   8/10 people could find a window's menus, but 7/8 of them learned to
    access them by hovering over maximized close/minimize/unmaximize
    buttons then moving horizontally -- which was extremely slow, and
    failed whenever the window wasn't maximized.

*   10/11 worked out how to open a new Firefox window, though 5/10 first
    tried clicking Firefox in the launcher again, which didn't work.

*   Only 4/11 worked out how to change the background picture. This is
    not as bad as it looks: for some of the others, Charline had asked
    them *not* to right-click on the desktop, because she was testing
    access to settings in general. Nevertheless, no-one found System
    Settings, in the session menu or anywhere else.

*   Only 5/11 could easily rearrange items in the launcher. For the
    other six, the main problems were that the launcher scrolled when
    they were trying to drag an item, or that it didn't accept a drop.
    (P10 was particularly unlucky in doing a Dash search for "menu" and
    finding the Main Menu editor, which is useless in Unity but still
    present by default.)

*   6/10 could easily find and launch a game that wasn't in the
    launcher. (P2 deserves special mention for finding and launching the
    game's .desktop file amongst piles of detritus in Nautilus's "File
    System" search results.)

*   Only 1/9 (P4) easily added that game to the launcher. For the other
    eight, the main problems were that the launcher disappeared when a
    window was maximized or at the left of the screen, that Dash items
    didn't have context menus, and that the launcher often didn't accept
    a drop.

*   2/2 successfully removed an item from the launcher.

*   Only 2/6 noticed an XChat Gnome notification, despite (1) a
    notification bubble appearing, (2) the Ubuntu button going blue,
    (3) the messaging menu envelope going blue, and (4) an emblem
    appearing on XChat Gnome's launcher.

*   9/9 easily launched LibreOffice Writer to write a letter.

*   5/5 easily found today's date.

*   9/9 easily saved their LibreOffice Writer document. (P1 recovered
    amazingly well after trying to save "Letter to Mr Smith 08/04/11",
    and getting the vile response "Error stating file '/home/ubuntu
    /Documents/Letter to Mr Smith 08/04': No such file or directory").

*   9/11 people could easily close a window. The other two (P2, P7)
    were not the only ones to be attacked by a bug that hid the title
    bar for a window underneath the menu bar; they were just the only
    participants for whom that bug really cramped their style.

*   9/9 easily found and opened an existing document.

*   8/9 easily copied text from one document into another. The other one
    (P2) managed it eventually, but the missing title bar for one of the
    document windows was again a major stumbling block.

*   Only 1/7 (P9, a Windows 7 user) easily arranged windows side by side
    by discovering the window snap feature. (That's probably not really
    a problem; it's a power user feature.)

*   Only 5/10 could easily delete a document, and two of those five (P3,
    P5) weren't sure that it had actually worked. For those who failed,
    the major problems were that files in the Dash didn't have context
    menus and weren't selectable, and that they couldn't see the Rubbish
    Bin (this was a UK test) folded away at the bottom of the launcher.

*   6/10 could easily see how many applications were running. (This was
    something people learned by experiment, and again probably isn't
    important.)

*   Only 3/6 could easily reveal the launcher when a window was touching
    the left of the screen (for example, a maximized window). For those
    who failed, the problems were that the effect of hovering over the
    Ubuntu button was unpredictable (P7), that the launcher popped up
    unwanted when aiming for a close button (P9), and that it popped up
    unwanted when trying to drag a file to a different target near the
    left of the screen (P11).

*   0/2 people could play MP3 songs stored on a USB key. This appeared
    to be entirely because the "Search for suitable plugin?" is too
    geeky, and was not Unity's fault at all.

During these tasks, the participants also revealed many other
interesting quirks and bugs. For example:

*   Nobody seemed to understand what the Ubuntu button was for, or the
    distinction between the Dash main screen and the Applications
    screen.

*   P1 and P12 both clicked in the Dash search field several times, but
    both concluded that the field could not be typed in (probably
    because the caret didn't blink and the hint text didn't disappear).

*   4/11 participants (P2, P3, P7, P10) tried double-clicking on
    "Applications" or "Files & Folders" in the launcher, but that just
    made the screen flicker.

*   5/11 participants (P2, P3, P5, P9, P10, P11) crashed Unity during
    their hour of testing. And towards the end of her test, P11 opened
    a zombie quicklist that stayed on top of everything and didn't
    respond to clicks.

I'll post more about the design issues (including those not specific to
Unity) in a separate message to the ayatana@ list.

- -- 
mpt
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk2npr8ACgkQ6PUxNfU6ecqBWQCgulHgSGo4anzShhPtLFubUtDE
fRkAn24p/kDBbbWZ4t42e9RJ4DUYQF78
=GdiZ
-----END PGP SIGNATURE-----



(Log in to post comments)

GNOME shell usability testing results?

Posted Apr 15, 2011 14:00 UTC (Fri) by ipilcher (guest, #73401) [Link]

Does anyone know if the GNOME folks have done anything like this with GNOME Shell?

Some Ubuntu Unity usability testing results

Posted Apr 15, 2011 14:52 UTC (Fri) by rfunk (subscriber, #4054) [Link]

Interesting, and not especially surprising. I too wonder what the results would be with GNOME Shell, as well as KDE.

This is unfair, but P7 seems to confirm some unflattering stereotypes about life coaches.

Some Ubuntu Unity usability testing results

Posted Apr 15, 2011 17:18 UTC (Fri) by Sufrostico (subscriber, #68053) [Link]

I was thinking that was only a weird idea on my head but, this usability
test were just like the ones on the commercial software sector.

Mac users, Windows users and none of them was a GNU/Linux user (not even
non-developers Ubuntu users).

That even between us, we ignore our selfs.

But it wasn't just me! Scott Kitterman (on the same thread) got the same
kind of doubt that attacked me:

On Friday, April 15, 2011 14:45 Scott Kitterman wrote:
>One dichotomy I see between this study group and this discussion is a
>lack of any dedicated Ubuntu users. I know there's a lot of focus on
>jumping the chasm, but I would have thought it would have been useful to
>see how an experienced user of Ubuntu's version of Gnome managed to
>transition to Unity.

Some Ubuntu Unity usability testing results

Posted Apr 15, 2011 20:15 UTC (Fri) by vonbrand (guest, #4458) [Link]

Yes, that would have been an interesting experiment too. But it looks like it wasn't the one set up here.

Some Ubuntu Unity usability testing results

Posted Apr 16, 2011 20:28 UTC (Sat) by istok (guest, #74401) [Link]

Ubuntu's users such as they are are small in numbers and the project is heading nowhere after 6 or 7 years. In all this time Ubuntu only took users from other distros, not from Windows and that other thing.

Clearly their aim now is to bring those new users in, and they are adjusting the product (or "distro") to that aim. What existing Ubuntu users think or feel seems to be of no interest to Canonical, and this little "test" shows it clearly.

Ubuntu target audience

Posted Apr 17, 2011 11:36 UTC (Sun) by sladen (subscriber, #27402) [Link]

[citation needed]

Ubuntu users and developers are people too. I think many of them realise there's little to be gained by having users swap between distributions, when there are bigger fish out there to fry. The aim is to fundamentally grow the market out there for Free Software.

One would hope that this much is (and has been) obvious right from the start by reading the title of Bug #1, or noting the inclusion of Win32 versions of OpenOffice/Firefox/Gimp on the CD image where there was space, to the stated aims of jumping the chasm.

To progress that expansion of the market, the areas Ubuntu have tended to focus on are the rough edges at any particular point in time; that other people haven't necessarily been working on (why duplicate work that other people are good at, when it's possible to go off and work on something else that needs fixing). In 2004 that might have been laptop support or shipping sudo by default; in 2011 it might be multi-touch, readable on-screen libre fonts or facilitating ARM support by getting people to work together upstream.

Some Ubuntu Unity usability testing results

Posted Apr 22, 2011 4:17 UTC (Fri) by nicooo (guest, #69134) [Link]

> In all this time Ubuntu only took users from other distros, not from Windows and that other thing. Or relatives of Linux users, specially the computer illiterate type.

Some Ubuntu Unity usability testing results

Posted Apr 15, 2011 22:52 UTC (Fri) by tajyrink (subscriber, #2750) [Link]

"Only 5/11 could easily rearrange items in the launcher."

But five did... I've been using Unity for a month wondering how to rearrange items, and I thought it's unimplemented feature :) Only now that I got to know it should be possible, I found out that you need to _long press_ mouse button before dragging it. I had tediously tried to drag items around, only to find out the launcher scrolling.

Some Ubuntu Unity usability testing results

Posted Apr 16, 2011 0:53 UTC (Sat) by pabs (subscriber, #43278) [Link]

Seems like they might be trying to do touchscreen stuff there, but doing it wrong. IMO the gestures you indicated should happen only using touchscreen input. When using mouse input the scroll wheel should scroll and dragging should move stuff.

Some Ubuntu Unity usability testing results

Posted Apr 16, 2011 7:47 UTC (Sat) by amaranth (subscriber, #57456) [Link]

It's not long press, just drag the icon out to the side away from the launcher. If you drag up/down it scrolls, if you drag to the side it rearranges.

Some Ubuntu Unity usability testing results

Posted Apr 21, 2011 2:55 UTC (Thu) by dberkholz (guest, #23346) [Link]

Wow. I just don't imagine dragging something sideways when I want it to move down as a natural action. Maybe my brain is broken.

Some Ubuntu Unity usability testing results

Posted Apr 21, 2011 19:20 UTC (Thu) by sciurus (subscriber, #58832) [Link]

It's not just your brain. I did not figure it out until I read http://askubuntu.com/questions/28086/unity-keyboard-mouse... . There are lots of useful tips there. I've become a big fan of using the numpad to tile windows.

Why nobody could find system settings

Posted Apr 17, 2011 14:52 UTC (Sun) by dank (guest, #1865) [Link]

System Settings is hard to find for several reasons:
1) it's not installed by default
2) it's not listed in the System menu group (it's in Themes & Tweaks)
3) the Dash makes discovering menu groups really hard, so even if
#1 and #2 were fixed, most users would be unable to find it without
lots of scrolling and squinting.

A related problem is that the Dash only has a hardcoded list of
menu categories, so any app that uses a menu cateorgry not on that
list (say, *any* Wine app) will be hard to find.

Wow, Software Center's icon *does* look like Trash!

Posted Apr 17, 2011 15:36 UTC (Sun) by dank (guest, #1865) [Link]

I used the Dash to install Flash. It was taking a while, so I
went back to Chromium while I waited. Then I tried to switch
back using the Launcher... and couldn't see the task anywhere.
After a while I noticed that mousing over the garbage can
showed mouseover text "Ubuntu Software Center"!
I did a double take, and realized finally that the nice icon
for Ubuntu Software Center is a grocery bag with lots of stuff
heaping out the top. Do they not use grocery bags for kitchen
garbage in South Africa, or wherever the UI designers live?

One more rough edge to smooth off!

Wow, Software Center's icon *does* look like Trash!

Posted Apr 18, 2011 13:15 UTC (Mon) by nye (guest, #51576) [Link]

It's never occurred to me before just how much the Ubuntu logo, especially at lower sizes, looks like the 'recycle' logo.

The (quickly sketched) icon proposal for Software Centre at https://wiki.ubuntu.com/SoftwareCenter#Branding looks even *more* like a recycling bag.

Don't know what they have against the cardboard box metaphor that's been widely used since forever.

Wow, Software Center's icon *does* look like Trash!

Posted Apr 20, 2011 1:40 UTC (Wed) by vonbrand (guest, #4458) [Link]

Why not use the cornucopia (I think that is what the horn overflowing with corn and fruit is called that the romans used to signal abundance)?

[But please wait until I patent this incredibly, unheard of, first-time-in-the-universe idea]

Wow, Software Center's icon *does* look like Trash!

Posted Apr 20, 2011 15:58 UTC (Wed) by __alex (subscriber, #38036) [Link]

No one seems to have come up with an intuitive icon for app stores yet. Even Apple just use some horrendous blue orb that resembles the new iTunes logo. And if you look at things like TV on Demand interfaces those also have meaningless logos.

Of course coming up with some icon to represent "some stuff you might want to install" is a bit of a challenge. I think this is probably more to do with the whole repository/app store model being a bit of a technical optimisation we have to suffer rather than a usability enhancement in the first place.

I think the solution to the Software Center icon problem is to just not give Software Center an icon in Unity and move all the functionality into Dash. Throw an extra option into each category group that directly offers you the option to install new items from the repository into this category and also extend the Search feature to give results from Software Center.

Wow, Software Center's icon *does* look like Trash!

Posted Apr 21, 2011 18:38 UTC (Thu) by kh (subscriber, #19413) [Link]

Shopping cart? Store on a cloud?

Some Ubuntu Unity usability testing results

Posted Apr 19, 2011 8:15 UTC (Tue) by rsidd (subscriber, #2582) [Link]

I tried Unity with Natty Beta 1, and switched back to "classic". I'm not sure what, if anything, has changed, but I gave Unity another spin with Beta2, and quite like it despite its drastic differences. For example,
  • I like the dock (or whatever they call it) and the fact that, if I click on the icon of an application with many open windows, it does an expose-style presentation of only those windows.
  • It is also very obvious (in contrast to, say, OS X) which applications in the dock are actually running versus which are merely launchers that are not running; and for each running application, how many open windows there are.
  • I like the active top and side edges that make the window go to full-screen or half-screen size -- even though I previously hated this behaviour. I think it used to instantly resize the window without warning, whereas now it makes a coloured full-size outline to warn me, but resizes only when I release the mouse button. That makes a huge difference.
  • I find the applications menu and files menu easy to navigate, though the icons are rather intuitive.
  • I find the use of screen space rather efficient, even on a large screen (previously it seemed like they had designed it for notebooks while ignoring other users.
Overall, it suddenly looks good!

Unity Benchmark Usability – April 2011

Posted Apr 23, 2011 0:27 UTC (Sat) by sladen (subscriber, #27402) [Link]

Possibly more interesting than the early observations above are the finding themselves, pushed subsequently: The final report itself is still to follow.


Copyright © 2011, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds