Do you sometimes feel like your usability reports aren’t taken seriously? Perhaps part of the problem is that the users of your reports have trouble understanding or acting on your recommendations. Writing useful and usable recommendations is a skill that all usability professionals should master. Evidence that usability professionals sometimes miss the mark when describing usability solutions comes from an examination of the usability comments from the seventh Comparative Usability Evaluation (CUE-7).

In CUE-7, experienced usability specialists independently wrote usability recommendations for six typical usability problems taken from two websites. Problems with the quality of recommendations include:

• Recommendations that are not actionable
• Recommendations that developers are likely to misinterpret
• Recommendations that do not improve the overall usability of the application

This article outlines guidelines for developing useful and usable recommendations—that is, recommendations that are likely to be accepted, that can be implemented, and that significantly improve the system.

A Recommendations Contest

Rolf Molich, Kasper Hornbæk, Steve Krug, and Josephine Scott arranged an informal contest. Usability professionals were invited to submit useful and usable recommendations for six usability problems from e-commerce websites.

We gave them this scenario: “Imagine that you have been hired by IKEA. The company has asked professional usability consultancies to carry out usability evaluations of selected parts of their website. The consultancies have reported six problems [which were included in the scenario]. The consultancies have not provided recommendations on how to fix the problems. Your task is to provide usability recommendations based on the—possibly imperfect—usability problem descriptions.”

Our comparative analysis is based on recommendations for identical problems submitted by eight experienced usability professionals, including all of the authors.

Test Your Skills

Here’s a chance to test your skills in writing useful and usable recommendations. This example presents a usability problem that is analyzed further in this article. Before you read on, consider what recommendation YOU would have provided. Then compare your recommendation to those in the article.

PROBLEM: Usability tests in CUE-5 had shown that many users were confused by the initial screen of the IKEA PAX Wardrobe Planner (Figure 1). When users start from scratch, they don’t know what they should do first to build a wardrobe.

Figure 1. Users were confused by this screen.

Figure 2. Sample of a submitted recommendation.

IKEA Wardrobe Planner Recommendations

Here are three recommendations submitted by CUE-teams, followed by our master solution.

1. “Increase the visibility of the instructions. Increase the visibility of the areas that will be clicked on.” COMMENT: This may indeed be a good idea, and the recommendation is commendably short. But perhaps it is too short. More details would have been useful; for example, how is the visibility increased? An illustration would have been helpful.
2. “Redesign this as a wizard (sequence of pages corresponding to steps) in which each page has controls for specifying only one wardrobe-design issue. At the bottom of each page are the standard wizard “Back,” “Next,” and “Cancel” buttons.” COMMENT: Wizards were part of several recommendations. There were few discussions of the tradeoffs: more usable interface versus more development and potentially a slower (more steps) interface. One recommendation argued that speed didn’t matter because there would be no repeat users but IKEA disagreed. Many provided insufficient details. This solution is not the least possible change.
3. “Allow the user to adjust closet dimensions with a familiar free transform frame and have the frame snap to legal dimensions. Present user with all editing options on initial screen, as illustrated.” COMMENT: This may create new problems because, for example, it limits the work area considerably and makes the first screen quite complex.

The recommendation is well illustrated (see Figure 3), but important details about the redesign are not described. It’s not entirely clear how it’s a solution to the “I don’t know what to do first,” problem. It’s not the least possible change that would solve the problem.

Figure 3. Well illustrated, but important details are not described.

Authors Master Solution

The authors had extensive discussions on the best possible recommendation based on all recommendations we received. After several iterations, we arrived at a “master solution” for each of the six problems. The full problem list and the master solutions are available from Rolf Molich at molich@DialogDesign.dk.

• Add the text “Click to create a wardrobe frame” and “How does this work? Click here to see tip” to the grid (see Figure 4).

Figure 4.

• Start the work area with a frame (users would have to create one anyway). The frame also ensures that something will happen when users click the controls that regulate the width, depth, and appearance of a frame.
• Write an unobtrusive tip that users can summon if they are in doubt (see Figure 5).

Figure 5.

During our discussions we found the following guidelines particularly helpful:

• Provide detail
• Illustrate
• Recommend the least possible change

The rest of this article presents the lessons we learned about useful and usable recommendations from our comparative study.

We end with some advice on the process of creating great recommendations.

Useful and Useable Recommendations

1. Make recommendations constructive and direct

Don’t simply criticize the current design. Some of the submitted recommendations were not true recommendations. For example, “It may not be obvious to users how to fill out the form. No instructions provided in Help.” Such indirect recommendations sound like complaints. “Include instructions to complete the form in Help” reduces the critical tone of the former recommendation.

2. Provide detail

Detail prevents misunderstandings and “creative misinterpretation.” The best recommendations included rewritten on-screen text and doctored screenshots. Avoid vague recommendations (“Make the error message clearer.”) For example, the recommendation “The way specials are promoted needs to be redesigned,” provides insufficient detail. A better recommendation is: “Specials are often overlooked. Redesign the Specials page, for example as shown in Figure x.” In short: Provide more details than you think are necessary.

3. Illustrate

A picture is worth a thousand words—even for recommendations. Illustrations—wireframes, marked up screenshots, or doctored screenshots—demonstrate what you mean. Also, examples from other systems that do what you are recommending are helpful.

4. Justify

This is particularly important if the recommendations may conflict with business goals, technical constraints, and the application’s current basic workflow.

5. Recommend the least possible change

Small is beautiful. Small changes increase the chance that your recommendation will be adopted, unless the system being evaluated is an early prototype. With released systems, complicated redesigns should be avoided when simpler solutions work just as well. Usability problems should not be solved by adding new features (“creeping featurism”).

6. Address only the original usability problem

Stick to the point and don’t overstep your charter. For example, some submitted recommendations suggested replacing the word “frame” in the IKEA PAX planner with “cabinet,” but we had no indication from the tests suggesting that users were confused by the word.

7. Speak the readers’ language

The readers of recommendations are developers, designers, and managers, not usability specialists. Avoid usability jargon such as “user model” or “508 Accessible” in recommendations.

8. Provide a quick-and-dirty, low-cost alternative

If the ideal solution is potentially expensive or time-consuming, provide an alternative that will reduce the impact of the problem to an acceptable level. Make it clear that the alternative solution is less than perfect.

9. Solve the problem

A great recommendation solves the problem for all users, not just for a small subset. It solves the cause of the problem, not just symptoms.

10. Keep recommendations short

Readers of usability evaluation reports are busy people, so we suggest keeping recommendations short. Long recommendations are less likely to be read and acted upon. This advice, of course, may conflict with other advice—for example, “Provide detail” and “Justify.” The point is to provide as much detail as required but no more.

Creating Great Recommendations

Creating great recommendations is not just a question of mastering guidelines for the content. It is also important to master the process by which you develop recommendations.

1. Nurture your professional skills

Writing great recommendations requires skills that differ from those required to run great usability tests. Run lots of usability tests to see what works and what doesn’t work for users. Discuss your usability recommendations with the “users” of your reports—developers, marketing, sales—to learn what works and what doesn’t work from a business and technical point of view. Study test results and recommendations from other usability professionals.

2. Understand how it works

Some participants in our contest unintentionally wrote recommendations that showed that they did not fully understand how the system that was tested worked or was intended to work. To write great recommendations, one must be fully familiar with how it works.

3. Take your time

Mediocre recommendations such as “Solve it in the help system” can be written in seconds. Writing great recommendations takes time. Usability testing often reveals problems quickly, but great recommendations are sometimes hard to produce.

4. Be humble

Before conducting this study, the authors had the naive perception that if you showed a useful and usable recommendation to other experienced usability professionals, they would immediately accept it even if they had provided a less optimal recommendation. However, our study shows that it takes a lot of discussion to convince usability professionals that their recommendation is not optimal. Often, when presenting your recommendations you will discover that someone has a better idea. Be humble; be prepared to set your ego aside and accept it.

5. Tweak but verify

Recommend the least possible change, and then recommend a quick usability test to see if you’ve solved the problem. If you have, fine. If not, try another tweak or move on to a major change.

6. Follow up

Show an interest in what happens to your great recommendations. Ask polite and timely follow-up questions if your recommendations are not adopted. Maybe you can learn something.

Enough Complaining

Agreeing on what constitutes useful and usable recommendations is more difficult than we thought. We often complain that developers don’t listen to us. Enough complaining! Let’s make our advice more usable and useful to them!

Topics: Report Writing, Transportation
Published in: 11/2008 in Transportation

Comments are closed.