Development

GNOME gains a usability specialist

GNOME's last formal usability testing was conducted by Sun Microsystems in 2001—before the release of GNOME 2. Since then, any usability testing that has occurred has been informal, usually carried out by individual developers with a variety of methods. However, the situation shows signs of improvement for GNOME 3, thanks to the work of Aakanksha Gaur, a graduate student currently completing her thesis at the National Institute of Design in Bangalore, India.

Gaur's interest in testing GNOME began when her computer crashed, and she replaced her proprietary operating system with Ubuntu and GNOME. Looking around for help, she realized that she was one of the few students at the Institute who was using free software, and that GNOME could be the focus she needed for her thesis on usability. Later, when she successfully applied for a mentorship with the Outreach Program for Women (OPW), it also became a source of funding for four months. In turn, Gaur has become one of the OPW's success stories.

In the three months since the conclusion of the mentorship, Gaur has continued to work with her mentor, GNOME design lead Allan Day, filing bugs and suggestions for improvements. At the start of Gaur's research, Day commented in an email that it "will be one of the first opportunities we have had to do an extended research study." GNOME executive director Karen Sandler has helped to arrange token payments for Gaur's usability testers — a standard practice in academic research involving test subjects.

Designing the tests

With this support, Gaur has focused her thesis on usability testing of GNOME — mainly, utilities and configuration tools — and on users' perceptions of GNOME. Unfortunately the blog notes on her early work became unavailable when her provider closed down, and Gaur has yet to repost them. For now, the clearest record of them is from an article I did when she was beginning her work. These blog notes included Gaur's early research into the desktop metaphor and her earliest informal testing, together with some rough suggestions for redesign of the To Do and Character Map utilities.

This early work also shows Gaur learning how to conduct her research. After trying to study usability by constantly asking questions as people worked, she wrote, "I was under the confident assumption that I shall take long interviews of users and magically they will reveal the design mistakes which we shall fix and hence, rule the world." In practice, though, she immediately found the technique lacking. The feedback was "very vague and very unfocused" and she realized that, "I ended up putting words in the mouth of the interviewee." The end result was a complete lack of "data that challenged my existing beliefs about the system in any way," and was therefore of minimal use.

Gaur's work beyond this point is documented on her current blog. Instead of micro-managing interviewees' experience, she opted for a test script in which interviewees are given a dozen basic tasks, such as changing the desktop wallpaper, searching for documents, and managing virtual workspaces. Meanwhile, Gaur observed how efficiently interviewees did each task, what mistakes they made, whether they could recall tasks later, and their emotional states after finishing a task.

The blog includes transcripts of pilot test sessions, as well as recordings of the sessions. Like her earlier blog entries, the available ones show Gaur making mistakes and improving her methodology, a degree of transparency that she suggests is appropriate for a free software project. After the pilot sessions, Gaur went on to interview eight testers with her revised methodology.

The first observations

Gaur is still finalizing her results. However, she does have a few general observations about both her methods and the state of GNOME usability. First, based on her research, Gaur concludes that "GNOME suffers from the issue of discoverability." That is, the tools users want are available, but may not be easy to discover. "The problem to crack is how to make them visible," she said, adding that "utilities like the Tweak tool and the rich set of extensions" might be the most immediate way to deliver improvements.

Second, people's expectations of GNOME are based heavily on the operating systems with which they are familiar, and the web applications that they use. While she has not finished assembling her research, she suspects that web and mobile applications have become more important than the operating system for people who spend more time on the web.

Never forgetting the self-criticism, Gaur also observed that her work would be improved if she made greater efforts towards "making a user comfortable in the first few seconds of interaction."

Her preliminary conclusion? "GNOME is doing the best it can," especially since free software development has traditionally been driven by developers rather than users.

Integrating Usability into GNOME Practice

These tentative conclusions are hardly startling. However, the point is that they have not been systematically recorded for GNOME 3. Instead, like all free software projects, GNOME has relied on bug reports and personal impressions, both of which are considerably better than nothing, but do not necessarily provide accurate views of the average user's experience. Bug reports, for example, are likely to describe the experience of those with enough knowledge to know how to file them, and not newcomers. Similarly, for all the controversy over GNOME 3, all available records indicate that the designers believed that they were providing simple and practical solutions to major problems. By putting matters on a more impartial basis, usability testing like Gaur's may act as a reality check to design proposals.

Certainly, GNOME is taking Gaur's work, as preliminary as it is, seriously. "I'm hopeful that this work will serve as a template for user testing exercises in the future," Day said. "There are certainly challenges involved in doing usability testing without a dedicated lab and equipment, so having a publicly accessible account of a successful open research exercise will be valuable." Sandler agreed, adding, "The GNOME Foundation whole-heartedly supports this work. GNOME 3.8 has had a really good response, but employing systematic tests will help us improve further."

For now, Gaur is focusing on completing her thesis. Once it is accepted, her first concern will be to make her work as widely available as possible, especially outside of GNOME. Then, she plans to go into more detail:

Gaur's work is just beginning. Yet the degree to which it has been accepted in eight months speaks highly of its quality and transparency, to say nothing of its originality. Perhaps in another twelve years, usability testing won't have to be re-introduced, but will have long ago become a routine concern for GNOME and other free software projects.

Comments (7 posted)

Quotes of the week

— Matthias Clasen, explaining ... at least to a degree ... how to develop applications with GTK+ 3.10.

— Geoffrey Thomas

Comments (none posted)

GNU C Library 2.18 released

Version 2.18 of the GNU C library is out. It contains a lot of bug fixes, a number of architecture-specific optimizations, a new benchmark framework, and partial support for hardware lock elision, among many other changes.

Full Story (comments: 8)

tig 1.2 released

Version 1.2 of tig, the ncurses-based front-end for git, is now available. Among the numerous new features in this release are "the ability to jump directly from diff to the corresponding line in the changed file, a stash view, and improvements to the log view.

Full Story (comments: none)

TestDisk and PhotoRec 6.14 available

Version 6.14 of the TestDisk disk recovery application and PhotoRec, its sibling geared toward recovering flash storage card, have been released. This update incorporates a lengthy list of improvements, including presenting more disk information in the user interface, support for the Nintendo Wii's backup format, and recognition of 15 additional file formats.

Full Story (comments: none)

4.11 Release of Plasma Workspaces, Applications and Development Platform

The KDE Community has announced the release of major updates to the Plasma Workspaces, Applications and Development Platform. "The Plasma Workspaces 4.11 will receive long term support as the team focuses on the technical transition to Frameworks 5. This then presents the last combined release of the Workspaces, Applications and Platform under the same version number."

Comments (none posted)

Development newsletters from the past week

• What's cooking in git.git (August 13)
• Haskell Weekly News (August 7)
• OpenStack Community Weekly Newsletter (August 9)
• Perl Weekly (August 12)
• PostgreSQL Weekly News (August 11)
• Ruby Weekly (August 8)
• Ruby Weekly (August 13)
• Tor Weekly News (August 14)

Comments (none posted)

Linux Graphics News (Linux Journal)

The Linux Journal has an overview of development in the graphics area. "By providing a complete direct rendering solution for OpenGL, the DRI3 and Present extension provide applications with the key benefits promised by Wayland, without the need of replacing X. Initial implementation of these extensions has been done, but they've not yet landed in the Xserver repository. If it lands soon, we may see this functionality in Xserver 1.15, scheduled for release in September or October; otherwise it'll need to wait for 1.16 this spring."

Comments (40 posted)

Kolås: A dither algorithm

Øyvind "pippin" Kolås of GIMP and GEGL fame has posted an extremely compact, "spatially stable" dithering algorithm that he believes will prove useful in applications like creating animated GIFs and e-ink videos. Spatial stability means that the color of a particular pixel will not change if the original image pixel is static beneath it; this property can cut the file size of animated GIFs considerably and saves on e-ink refreshes. Kolås notes that the small code size "makes it suited for micro-controller or direct hardware implementations", and believes the algorithm used is in the public domain.

Comments (5 posted)