What is the typical reaction to your research reports?

“Wow, thanks for sending that report. Those findings are really important and now we know what we need to do.”

Or

“Uh, gee, we’re still working through that report, thanks. However, we do have some questions about the data. Can you answer those for us now?”

If you sometimes hear the second response (and who hasn’t?), or you want to review your writing style to make sure it’s having the impact you desire, there are two key things to keep in mind:

• Flexibility: Use a report format that can expand or contract easily depending on how much detail your audience wants.
• Glanceability: Present information in a way that readers can understand it “at a glance.”

Aren’t there existing standards for user research reports?

Yes, there are existing standards.

The usability community has an international standard, ISO/IEC 25062:2006 that provides a “common industry format” (CIF) for reporting usability test findings. The standard consists of a seventy-four-item checklist for what should be included in a report, and can be found at http://zing.ncsl.nist.gov/iusr/documents/cifv1.0.htm. While important and useful for thorough documentation and archiving, the CIF comes across as a dissertation, and most of us don’t want to read one. So let’s move on past the CIF.

Classic usability textbooks (for example, Rubin and Chisnell’s Handbook of Usability Testing, or Barnum’s Usability Testing Essentials), offer a mixed bag of report styles. They start by outlining how to write a formal report and then suggest different formats that might work better. As Rubin and Chisnell write, “Deliver your report in a slide deck, in a company blog, or even an email if that way works best in your situation.”

How are other researchers doing their reports?

A quick survey with 28 respondents, distributed via the chi-consultants discussion list, found the following:

• The majority are split between writing up findings in Microsoft Word or PowerPoint.
• The executive summary and the usability issues list are two critical pieces included in just about any format used.
• The typical audience for reports is the whole product development team, with a subset focused on management.

Arguments in favor of the document-style report are that it leaves a comprehensive record of the research and satisfies an audience that likes a lot of detail.

Arguments for the slide deck-style include that it’s easier to write and consume, and that it provides better formatting for visuals. Your decision on what style to choose will ultimately come down to your audience:

• For management types that want something polished but quick to digest, use a PowerPoint presentation with very glanceable information.
• For busy audiences that want an easy-to-read format, use an executive summary.
• For skeptical audiences that need detail, provide a discussion section formatted in a question-and-answer style.

PowerPoint Presentation

PowerPoint has become one of our most requested, and, we believe, most effective styles of reporting. Why?

• PowerPoint can be effective in three slides or 300.
• A three-slide PowerPoint can easily be converted into an email summary.
• Good PowerPoint slides are glanceable.

Create your PowerPoint report in two parts: the first two-to-three slides are the executive summary and the rest are the usability issues list. The list is especially effective when you use screenshots to point out the issues and explain recommendations.

Does PowerPoint automatically make information glanceable?

No, it does not.

Some business executives believe PowerPoint presentations should consist of no more than three slides. Others believe all content should fit into one slide. The danger in that case is that you can end up with a very busy slide (see Figure 1).

The slide in Figure 1 requires careful reading of the summary, tracking numbers back from each cell, and guessing why some are highlighted. This works directly against one of the most appealing features of PowerPoint—its flipability. Readers should be able to flip through slides quickly to get to what is relevant to them. Therefore, you should keep the information clear and concise.

What does it mean for a slide to be “glanceable?”

A glanceable slide is one that uses callouts or pointers to show what’s important, rather than describing it in figure captions or paragraphs (see Figure 2).

The Executive Summary

An executive summary starts with the basic gist, and then expands as necessary. What’s the basic gist? Think:

1. Highlights
2. Lowlights
3. How to turn the lowlights into highlights

The executive summary is the most important part of a report because it’s the only part that most people will read. It needs to stand in for the entire report. In discount usability testing, it will be the entire report in an email.

Don’t spell out methodology here. Your reader will generally assume you know how to design a good study, and won’t want to get bogged down by details. Focus on what you learned, and distill those findings down into the briefest possible phrases and points.

How can I draw my readers in right at the start?

Open with a quote.

What better way to communicate results of research with real people than to quote one of them right away? The best examples are the juicy quotes that exude love and joy for the product design or express frustration over poor usability. Picking one or two quotes from participants to start off a report brings your results to life and gives your readers an immediate sense of the validity and importance of the research. For example:

“I’m shocked at the things I was able to do…(my old program) took me several weeks to figure out some basic stuff…with this I could feel my way around right away…”

How can I get my readers invested in reading the whole thing?

Pull out some positives up front.

Everyone likes praise. Most usability studies will find some things that are working just fine, but there is a tendency to ignore those in favor of things that need to be fixed. We’ve found that opening with “highlights” or “things working well” helps to ease the tension and lower defenses.

Even in a straightforward usability issues list, we include a priority category of “good” or “working well” to make the point that certain UI elements are well-designed and should not be changed. This is especially important if the positive findings are answering research questions.

Document-Style Report

A good way to organize a document-style report is to use a question-and-answer format. It’s an easy-to-read, conversational style that helps readers locate the information most relevant to them. For the designer who created the blue dot display, what better way to get her the data she needs than to have a heading, “Did people notice and use the blue dot display?”, followed by the quick answer “Yes, they did.” A sample format would look like this:

• Question header
• Short answer
• Description and examples of data

How can I make my report glanceable?

Add a Table of Contents at the start, created from section headings. Readers will see at-a-glance what the report contains and be able to jump to whatever section they are interested in reading.

If I have to include numerical data, what’s the best way to present them?

Graph the data.

Tables may be a great way to organize data, but they’re convoluted and take time to decipher. Graphs are better; because readers can immediately see comparisons. Include the actual numbers in the graph and use error bars to show confidence intervals or standard error of the mean, and you will satisfy even the most statistically savvy audience (see Figure 3).

Topics: Report Writing
Published in: March, 2012 in Beyond Books