August 14, 2013
This article was contributed by Bruce Byfield
GNOME's last formal usability testing was
conducted by Sun Microsystems in 2001—before the release of
GNOME 2. Since then, any usability testing that has occurred has been
informal, usually carried out by individual developers with a variety of
methods. However, the situation shows signs of improvement for GNOME 3, thanks to the work of Aakanksha Gaur, a graduate student currently completing her thesis at the National Institute of Design in Bangalore, India.
Gaur's interest in testing GNOME began when her computer crashed, and she
replaced her proprietary operating system with Ubuntu and GNOME. Looking
around for help, she realized that she was one of the few students at the
Institute who was using free software, and that GNOME could be the focus
she needed for her thesis on usability. Later, when she successfully
applied for a mentorship with the Outreach Program for
Women (OPW), it also became a source of funding for four months. In turn, Gaur has become one of the OPW's success stories.
In the three months since the conclusion of the mentorship, Gaur has continued to work with her mentor, GNOME design lead Allan Day, filing bugs and suggestions for improvements. At the start of Gaur's research, Day commented in an email that it "will be one of the first opportunities we have had to do an extended research study."
GNOME executive director Karen Sandler has helped to arrange token payments for Gaur's usability testers — a standard practice in academic research involving test subjects.
Designing the tests
With this support, Gaur has focused her thesis on usability testing of GNOME — mainly, utilities and configuration tools — and on users' perceptions of GNOME.
Unfortunately the blog notes on her early work became unavailable when her
provider closed down, and Gaur has yet to repost them. For now, the
clearest record of them is from an article
I did when she was beginning her work. These blog notes included Gaur's early research into the desktop metaphor and her earliest informal testing, together with some rough suggestions for redesign of the To Do and Character Map utilities.
This early work also shows Gaur learning how to conduct her research. After
trying to study usability by constantly asking questions as people worked,
she wrote, "I was under the confident assumption that I shall take long interviews of users and magically they will reveal the design mistakes which we shall fix and hence, rule the world."
In practice, though, she immediately found the technique lacking. The
feedback was "very vague and very unfocused" and she realized that, "I
ended up putting words in the mouth of the interviewee." The end result was
a complete lack of "data that challenged my existing beliefs about the
system in any way," and was therefore of minimal use.
Gaur's work beyond this point is documented on her current blog. Instead of micro-managing interviewees' experience, she opted for a test script in which interviewees are given a dozen basic tasks, such as changing the desktop wallpaper, searching for documents, and managing virtual workspaces. Meanwhile, Gaur observed how efficiently interviewees did each task, what mistakes they made, whether they could recall tasks later, and their emotional states after finishing a task.
The blog includes transcripts
of pilot test sessions, as well as recordings of the sessions. Like her
earlier blog entries, the available ones show Gaur making mistakes and
improving her methodology, a degree of transparency that she suggests is appropriate for a free software project. After the pilot sessions, Gaur went on to interview eight testers with her revised methodology.
The first observations
Gaur is still finalizing her results. However, she does have a few general
observations about both her methods and the state of GNOME
usability. First, based on her research, Gaur concludes that "GNOME suffers
from the issue of discoverability." That is, the tools users want are
available, but may not be easy to discover. "The problem to crack is how to
make them visible," she said, adding that "utilities like the Tweak tool
and the rich set of extensions" might be the most immediate way to deliver
improvements.
Second, people's expectations of GNOME are based heavily on the operating
systems with which they are familiar, and the web applications that they
use. While she has not finished assembling her research, she suspects that
web and
mobile applications have become more important than the operating system
for people who spend more time on the web.
Never forgetting the self-criticism, Gaur also observed that her work would be improved if she made greater efforts towards "making a user comfortable in the first few seconds of interaction."
Her preliminary conclusion? "GNOME is doing the best it can," especially
since free software development has traditionally been driven by developers
rather than users.
However, integrating usability testing and continuous efforts to monitor
and measure the impact of new design will help immensely. Having said that,
I really want to be the volunteer to take this up seriously.
Integrating Usability into GNOME Practice
These tentative conclusions are hardly startling. However, the point is that they have not been systematically recorded for GNOME 3. Instead, like all free software projects, GNOME has relied on bug reports and personal impressions, both of which are considerably better than nothing, but do not necessarily provide accurate views of the average user's experience.
Bug reports, for example, are likely to describe the experience of those
with enough knowledge to know how to file them, and not
newcomers. Similarly, for all the controversy over GNOME 3, all available
records indicate that the designers believed that they were providing simple and practical solutions to major problems. By putting matters on a more impartial basis, usability testing like Gaur's may act as a reality check to design proposals.
Certainly, GNOME is taking Gaur's work, as preliminary as it is, seriously. "I'm hopeful that this work will serve as a template for user testing exercises in the future," Day said. "There are certainly challenges involved in doing usability testing without a dedicated lab and equipment, so having a publicly accessible account of a successful open research exercise will be valuable."
Sandler agreed, adding, "The GNOME Foundation whole-heartedly supports this work. GNOME 3.8 has had a really good response, but employing systematic tests will help us improve further."
For now, Gaur is focusing on completing her thesis. Once it is accepted,
her first concern will be to make her work as widely available as possible,
especially outside of GNOME. Then, she plans to go into more detail:
Looking at specific apps and making processes for usability testing in
GNOME. GNOME has got a lifelong contributor in me, and I have a list of
nifty UX [User Experience] research tributaries that have emerged from my
current study that I would like to continue. Outside of GNOME, I will be
looking to collaborate with more FOSS projects and make my career as an
Open Source UX Researcher.
Gaur's work is just beginning. Yet the degree to which it has been accepted in
eight months speaks highly of its quality and transparency, to say nothing
of its
originality. Perhaps in another twelve years, usability testing won't have to be
re-introduced, but will have long ago become a routine concern for GNOME
and
other free software
projects.
Comments (7 posted)
Brief items
There's some magic here that I'm not going to get into right now. It's even better than I'm telling you.
—
Matthias
Clasen, explaining ... at least to a degree ... how to develop applications with GTK+ 3.10.
Kind of disappointed that Subversion (the project) hasn't moved its source control to git. Get with the program guys.
—
Geoffrey Thomas
Comments (none posted)
Version 2.18 of the GNU C library is out. It contains a lot of bug fixes,
a number of architecture-specific optimizations, a new benchmark framework,
and partial support for
hardware lock
elision, among many other changes.
Full Story (comments: 8)
Version 1.2 of tig, the ncurses-based front-end for git, is now available. Among the numerous new features in this release are "the ability to jump directly from diff to the corresponding line in the changed file, a stash view, and improvements to the log view.
Full Story (comments: none)
Version 6.14 of the TestDisk disk recovery application and PhotoRec, its sibling geared toward recovering flash storage card, have been released. This update incorporates a lengthy list of improvements, including presenting more disk information in the user interface, support for the Nintendo Wii's backup format, and recognition of 15 additional file formats.
Full Story (comments: none)
The KDE Community has
announced
the release of major updates to the Plasma Workspaces, Applications and
Development Platform. "
The Plasma Workspaces 4.11 will receive long term support as the team focuses on the technical transition to Frameworks 5. This then presents the last combined release of the Workspaces, Applications and Platform under the same version number."
Comments (none posted)
Newsletters and articles
Comments (none posted)
The Linux Journal has
an overview
of development in the graphics area. "
By providing a complete
direct rendering solution for OpenGL, the DRI3 and Present extension
provide applications with the key benefits promised by Wayland, without the
need of replacing X. Initial implementation of these extensions has been
done, but they've not yet landed in the Xserver repository. If it lands
soon, we may see this functionality in Xserver 1.15, scheduled for release
in September or October; otherwise it'll need to wait for 1.16 this
spring."
Comments (40 posted)
Øyvind "pippin" Kolås of GIMP and GEGL fame has posted an extremely compact, "spatially stable" dithering algorithm that he believes will prove useful in applications like creating animated GIFs and e-ink videos. Spatial stability means that the color of a particular pixel will not change if the original image pixel is static beneath it; this property can cut the file size of animated GIFs considerably and saves on e-ink refreshes. Kolås notes that the small code size "makes it suited for micro-controller or direct hardware implementations," and believes the algorithm used is in the public domain.
Comments (5 posted)
Page editor: Nathan Willis
Next page: Announcements>>
