Formative research methods such as contextual user visits are becoming more commonplace in the field of user experience, allowing researchers to provide increasingly compelling insights to product teams and business organizations in an effort to drive strategy. This is a testament to the creativity and skill of those in our field.
However, it seems all too common that such insights are met with indecision regarding appropriate action to take. While researchers are able to highlight key learnings and offer recommendations as to potential solutions, such information is often thrown into a pot along with countless team member opinions. The unfortunate outcome is often that the final product falls short in addressing the insights obtained from the research.
This is not so much the researcher’s fault, but rather a lack of understanding among the product team of how best to translate formative research insights into product strategy, as well as the absence of a plan for doing so. We propose an approach that is premised on the notion that the best way to create solutions for your audience is to work closely with a small set of target users to understand their needs, brainstorm solutions with them, and then present solutions to them for feedback and iteration.
The crux of the approach is to form a partnership with a small set of target users and work with these users across the different research and design activities. All activities are conducted in a highly interactive and personal way over a short period of time and at a relatively low cost. A key assumption of this approach is that by working with users to improve their personal situations, they will be more thoughtful, engaged, and conscientious during the process.
The power of such an approach can be striking; the first example we describe produced an innovative and compelling vision for a longstanding product challenge, and was completed in two months for less than $10,000 with only one dedicated researcher and one dedicated designer.
Rethinking the eBay Seller Dashboard
The existing eBay Seller Dashboard serves as a destination where sellers can learn more about their current standing on eBay, how they are perceived by the eBay community and eBay itself, and whether they need to take action to avoid problems in the future. Despite many product iterations, the usage of the dashboard and attitudinal metrics among sellers were below target goals.
As a starting point, we hypothesized that sellers would be more likely to visit the dashboard if it provided a more holistic view of their selling activities rather than just their standing. The effort was then focused on determining what information would belong in such a destination, and how best to present it to sellers.
The Approach
Our approach consisted of five phases, with each phase informing the next. Such a method offered a systematic way to explicitly define participant needs and pain points, and then work with the same participants to come up with solutions while leveraging their context and past experience.
Phase 1: Recruiting Users
We recruited a small set of target users (twelve sellers) who were interested in improving the product and willing to commit to the multi-phase effort. All sellers participated in all six hours of activities without any no-shows or dropouts. In fact, many of the participants approached us afterward to thank us for giving them such a meaningful stake in the future direction of the product.
Phase 2: Problem Investigation
We visited each participant in their home or office for a two-hour session consisting of an interview and observation of their natural behavior. Our goal was to understand what questions our users sought to answer in the course of their daily activities, how they answered them (in terms of the places they would go to and the information they would seek), and how the environmental and situational constraints influenced their approach.
Phase 3: Brainstorming
A two-hour dynamic participatory design session was conducted individually with each user. The session was dedicated to brainstorming ideas, solutions, and insights raised during the prior visit. A compelling advantage of this approach is that the users were working to solve the problems they themselves identified, not problems identified by other users. This led to more engaged and conscientious participation.
At the beginning of the session, we presented each user with a summary of insights from our visit in the form of questions they tried to answer and how they answered them. This was intended to ground the conversation and orient the participant to the subsequent activities. It also had the welcome side effect of putting the participant at ease.
The remainder of the session consisted of three main activities (see Figure 1):
Figure 1. The three steps of the dynamic participatory design session.
- Participants were asked to imagine there was a single place where they could go to answer all of their questions. They then received a large blank sheet of paper on which they had to sketch out initial ideas. The sketch could be in the form of boxes and arrows, words, or whatever other representation participants deemed appropriate or felt comfortable using.
- Each participant was presented with existing information elements from the eBay site that they could potentially use to answer their questions (see Figure 2). The elements consisted of laminated cutouts that could be affixed with Velcro to a felt whiteboard. This allowed users to create a more well-defined page based on their initial sketch.
- The final component of the session consisted of a “joint sketch” during which the participant worked side-by-side with the designer (see Figure 3). Based on the participant’s initial sketch and elements placed on the felt whiteboard, the designer began to draw a solution representing something that might more realistically become a product on the website.
Figure 2. (left) Laminated cutouts of information elements on a whiteboard.
Figure 3. (right) “Joint Sketch” activity with the designer and a participant.
The method used in the session is called “dynamic participatory design” because the participant guided the designer as she created that final sketch to further improve the positive aspects and help correct those that would not meet the participant’s needs. This dynamic interaction also successfully addressed the common criticism associated with “asking users without design skills to design products,” because the participant was merely guiding the skilled designer based on their real-world experience and domain knowledge.
Phase 4: Evaluation
We conducted two 120-minute long focus group sessions, during which the same participants evaluated the concept solutions created based on the prior activities. Unbiased feedback on the concepts was gathered by asking participants to first silently note their thoughts on paper and then present them individually before the discussion began.
Phase 5: Iteration
We continued to iterate on the proposed concept solutions based on feedback collected during the focus groups.
Tailoring the Partnership Approach to Different Types of Projects
The partnership approach was adapted to inform a PayPal initiative focused on helping people organize and collect money for groups, events, and fundraising causes.
As with the eBay Seller Dashboard project, a small group of users was recruited to participate in a multi-phase research project. However, because the research was intended for a social process involving multiple people, rather than an information hub for an individual, two changes to the approach were made. First, because the research had to accommodate multi-user dynamics, additional consumers who interacted with the original participants were also asked to participate in order to provide perspective on their role in the process.
Second, because we were initially focused on identifying the ideal high-level process of organizing (rather than the details of the design), we modified the structure of our dynamic participatory design in the brainstorming phase. After the initial sketch exercise, participants were given a stack of cards containing action phrases representing potential steps in the overall process (such as “create sub-account”). We wanted to know how they envisioned the process to work, and how technology could alleviate certain logistical and social burdens.
For example, those involved in fundraising were positive to the “Show progress toward a goal” concept, but one participant who collected dues for her volleyball team wrote “For myself only” on the card to indicate that only she needed to see that information from PayPal. When it came time for the “joint sketch,” the designer used the rest of the whiteboard to sketch out how certain parts of the process might work, with the participant providing input in real time (see Figure 4).
Figure 4. Whiteboard containing elements from our design process, including photos from the field visit and the “joint sketch”
The Impact of This Approach
During the eBay Seller Dashboard focus groups, participants were overwhelmingly positive toward the proposed concepts. When we first presented the results to the stakeholders, there was a lot of skepticism. How could participants be so supportive of the proposed ideas? Surely there must have been some bias involved, perhaps due to the rapport established with the researcher and designer throughout the course of the effort.
Fortunately, there was a simple reply to this concern. Participants didn’t like everything about the proposed concepts and were quick to point out the flaws in the approach. However, as a whole, they felt the proposed concept represented a compelling improvement over the status quo. Also, and perhaps more importantly, why would anyone expect the participants to not like the proposed concept? After all, it was based on an understanding of their real-world needs and took into account their ideas and suggestions of how to address them.
The close partnership with target users afforded by this approach has had a lasting impact on the affected product teams. The discussion has changed from “What should we do?” to “How can we make this vision a reality?” Team members involved in these examples still refer to specific participants’ stories, ideas, and reactions gathered during the research.
When to Use This Approach
Based on our experience using this approach for two different projects, we offer the a few guidelines.
The approach should be used when:
- Defining a product offering that is new to the company or market as a whole
- Rethinking an existing product that users feel is clearly “broken” and would like to see fixed
- Defining a new product or product change that will be implemented to address business needs (without an obvious corresponding user need), with the intent of informing the best way to do so for users
The approach should not be used when:
- Users are accustomed to the current product and feel that it effectively meets their needs. In such cases, users may feel that significant product changes would require re-learning of established behaviors without any clear benefit.
- Areas of improvement are difficult for users to identify. For example, optimizing presentation of search results would be more effectively driven by applying cognitive principles and observing user behavior rather than brainstorming of solutions with users.
Our hope is that more researchers will leverage this framework and partner with their users to help bridge the gap between insight and strategy.
Retrieved from https://oldmagazine.uxpa.org/engaging_users_in_product_design/
