Figure 1. The Peer Tutoring method in action.

In traditional usability testing, the participant is trying to accomplish two things at the same time: learning to use an unfamiliar product and thinking aloud. While this method might work well for adults, it doesn’t fully represent children’s natural behavior of learning through play. It also doesn’t take into account the fact that multi-tasking skills do not develop until later years, and a concurrent verbal protocol can seem very artificial to a child. Not to mention that a child might feel uncomfortable talking to a stranger.

So how can we better engage and encourage children in a usability test situation?

We found Peer Tutoring to be well suited for usability testing with children. The method originated from educational sciences and is commonly used in schools. In Peer Tutoring, a student helps a peer learn something new, and at the same time, that student learns as well (see Figure 1). An important benefit of the method is that the children establish a rapport because the tutor naturally gives advice to the peer using his or her own language. Also, there is equality in authority and knowledge between the peers, which can make their communication easier and livelier.

Over the past year and a half, we’ve tested a number of digital LEGO products using Peer Tutoring. We’ve found that the biggest difference between this method and the traditional usability test is in session moderation: an adult moderator interacts with two children instead of one, gently facilitating the children in an unobtrusive way. This leaves more space for the children’s own spontaneous dialog and discovery. One of the highlights we experienced with Peer Tutoring occurred fifty minutes into a session, when an eight-year old girl (the tutor) asked the moderator when the actual interview was going to start, because her mom told her she would have to answer some questions.

Planning a Peer Tutoring Usability Study

When setting the objectives for a Peer Tutoring study, we typically have the same goals as we would for a traditional usability study. We still want to know how useful the product is to the target group, which features are “cooler” than others, how fast the children are able to learn to interact with the product, and how satisfied or happy they seem after using the product.

The unique aspect of study planning is in our participant screening criteria. When recruiting dyads for each session, we look for children who are the same age and who already know each other. To prevent too much similarity in responses, however, we try to make sure that the children are not best friends.

Moderating Test Sessions

After the two friends and their parents have arrived at our office, we explain what will happen during the session, and ask the parents to provide written permission for the children to participate in the study. After assigning roles to the two children, we tell the tutee, “First, your friend will try out something new on the LEGO website, and then we’ll come out and get you so that he can show you what he did on the computer.” The tutor and the moderator then go into the test room.

The tutor can try everything out and provide any comments he wants to, but he doesn’t have to think aloud. The moderator tries to guide the child as little as possible to prevent bias from occurring, but if the child is not spontaneously using the features we want to test, the moderator encourages him to use them. At some point the moderator asks the tutor, “Do you think that you can explain this tool to your friend now? Or do you have any questions?” As soon as the child thinks he is ready, we invite the second child into the test room.

Figure 2. The LEGO CITY Comic Builder web application we tested using the Peer Tutoring method.

Now the tutor starts teaching his friend by showing what he just did on the computer. In a study of a Comic Builder tool that allows users to build their own comics (see Figure 2), a conversation between the two participants might sound like this:

TUTOR: That’s the LEGO website. And there [pointing on the screen] you can make your own comic. I just did one.

TUTEE: Where do I start?

TUTOR:  First, pick one of these [pointing to the library with the background images]. See, you can zoom in and out if you want.

TUTEE: Where? [trying to zoom by double-clicking on the image] This doesn’t even work!

TUTOR: It does. It’s there, this one with the brick. But I didn’t know how to do that too at the beginning.

The moderator observes the children and acts as a helper in case they get stuck. If the tutor forgets his role, the test moderator prompts him to help his friend, “Can you tell Sarah something about the blue ribbon there? Remember? You used it before.”

After the session, the moderator asks the children a few general questions related to satisfaction and enjoyment. Also, the participants can ask questions about the product we’ve just tested or about any other LEGO products.

Analyzing the Results 

All sessions are recorded for internal use. We transcribe the videos afterwards to accurately track how many participants were able to use a particular feature. The videos are also used to trace children’s behavioral workflow when interacting with our product, and where the product did not meet their expectations (as in the comment above, “This doesn’t even work!”).

Based on the study results, actionable design recommendations are made to the internal client commissioning the study, usually a producer or project manager. Since the producer or project manager is involved in planning the study and usually also watches most of the sessions, the results and recommendations aren’t a surprise.

Want to Use Peer Tutoring with Your Product?

• Peer Tutoring works best when testing software applications or games that have a rather linear workflow with a clear beginning and end. The example used in the article was a LEGO Comic Builder tool. At the beginning of the session, the children are presented with an empty sheet on the screen. By the end of the test, they have built their own comic. This means the children had to follow certain steps to get to the predetermined goal.
• If the tested product is a website, the moderator has to make the pre-defined task scenarios or the features she wants to test obvious to the tutor, if the tutor is not using them spontaneously.
• When selecting tutor and tutee, choose the more talkative child as the tutor. We usually put this in our screener questionnaire we send to the recruiting agency. However, we might also determine the tutor shortly before the test session, when we’re talking to the parents and children in person.
• Make sure that the children remain focused on your product instead of playing around together. The moderator might have to gently redirect, “It looks like you’re done. What do you think about this game (or website)?”
• Try to make the test session interesting to the children. Peer Tutoring might not be ideal for simplistic task scenarios or products, as the tutor might not want to teach a friend about something too obvious.
• Avoid using abstract wireframes or sketches. The higher the fidelity of the testing materials, the more concrete the responses you will receive.

Conclusions

Peer Tutoring is a useful usability evaluation method when children are the end users of a product. By taking into account the cognitive skills of the participants and the natural way in which children learn, Peer Tutoring elicits more self-motivated user comments than other techniques, and makes it particularly easy to evaluate engagement. Because the tutor is testing the product by teaching it to his friend, we are able to assess learnability in a very natural and elegant way. If the tutor is not able to teach something to his friend, we can easily see where our software must be improved.

Topics: Usability Testing, Children
Published in: August 2012 in Featured Articles