The early 2010s will not be remembered as good years for internet anonymity. From news outlets likethe CBC, ESPN, and Huffington Post, to social media sites YouTube and Facebook, many leading organizations have at least dabbled in the prohibition of anonymous comments from its users (even sometimes backpedaling in the face of criticism after implementation).

In the user experience (UX) design field, the candid, unadulterated comments and feedback that researchers gather from usability testing or interviewing participants are the most valuable assets to inform a design. Surrendering a participant’s anonymity during a user research interaction risks compromising the authenticity of the feedback being sought, ultimately resulting in a less informed—and probably less effective—product.

User anonymity is particularly important in the enterprise research environment, that is, conducting user research and usability testing for tools, applications, and services administered by the participant’s employer. Participant feedback elicited under the duress of perceived employer review or scrutiny (imagine the mindset: “Am I going to lose my job because I expressed dissatisfaction with my supervisor’s website?”) yields biased input, putting the validity of your design at risk.

Even outside of the enterprise environment, you may be conducting research or testing designs for products that users trust with confidential or personally identifying information. For example, have you ever designed a website that provides health information to the public? If you’re facilitating a contextual interview with a member of that target audience, there’s a good chance that the participant may share a personal story about a medical condition that they or a loved one sought treatment for. This story could prove valuable for your design, but you need to protect the privacy of the storyteller—which is why facilitators should anonymize all participant information by default.

User experience professionals should be conducting research studies that are supportive of anonymity-protected user feedback throughout every step of the design process.

• Recruitment, Outreach, and Screening – Understanding and being transparent about how candidates were sourced.
• Usability Facilitation – Gaining trust of the user during contact phases (from the first contact or invitation through to the dialogue—whether it be during activities like contextual interviews or cognitive walkthroughs).
• Summary and Findings – Protecting the user’s anonymity after findings are collected and summarized (and negotiating with insatiable stakeholders and team members who are inclined to identify the participant by name).

Set Expectations with Team and Stakeholders

Protecting anonymity during user research may be instinctive for UX professionals, but the same isn’t necessarily true for cross-functional colleagues and client stakeholders. Imagine their surprise after you’ve spent hours conducting dozens of user research interviews with substantive findings, only to hear during a client presentation that the senior executive stakeholder was expecting names and contact information for each of the participants— especially the ones that had not-so-nice things to say about the organization’s Help Desk experience.

Do’s

• Set expectations early about how research participant information is protected from the onset of the project, as opposed to waiting until after the studies have been conducted. Position this expectation as a benefit to the client stakeholder: “We’ll make sure that your staff’s input is heard and protected, ensuring that the ultimate design isn’t formed from external influences.”
• Be clear to the participant about how you will protect his or her anonymity.
• Socialize those expectations before beginning any research effort. Reinforce the idea that users are the product owner’s most valuable asset, and that keeping them happy, engaged, and trusting will lead to a better outcome.
• Include limited persona information with key pieces of feedback—like broad department or division names—specific enough that it will resonate with stakeholders, but vague enough that no individual name can be deduced.

Don’ts

• Don’t let manager’s observe sessions or watch videos of their direct reports.
• Don’t identify an individual’s role if there is other corroborating information available that would uniquely identify the participant.
• Don’t ever let a team member or stakeholder strong arm you into revealing identifying information about participants who were already told that their feedback would be kept anonymized! Failing to live up to your word regarding anonymity protection breaks your trust with not only that participant, but also the participant’s peers. If you and your user research team are not careful about participants’ privacy, word will get around that you team is not trustworthy.

Recruitment, Outreach, and Screening

After contacting a research candidate to gauge interest in participating, you may have gotten the response, “How did you get my name?” Fair question.

User experience design professionals are no stranger to the challenges of recruitment. Our eyes tend to light up with excitement when we’re handed a lengthy list of usability testing candidates complete with contact information all ready to go. However, consider how those names got on those lists in the first place. Did the candidate opt-in voluntarily? Or was his or her inclusion passive—scraped unknowingly from an unrelated registration source?

When handed a list of recruitment candidates, have a clear understanding of where those names came from, and be prepared to answer pointed questions about how you got their contact information, who nominated them, and what other information is being stored.

Honor requests to be removed from recruitment lists, and see to it that your sources do the same. On the other hand, engaged participants may be interested in helping again, so be sure to document candidates who are willing to return in case you need quick feedback with minimal screening for an urgent design need in the future.

During User Research Sessions and Usability Tests

Participants that you engage with for user research activities are forming their sense of trust in you from the moment you make first contact. Every interaction you have with the participant—whether written, oral, or in person—is moving the needle one way or the other about how much feedback he or she is willing to offer and how candid they’ll be about it.

When first contacting a research candidate who’s not otherwise familiar with you or your organization, be transparent about who you are and what you do. Participants may be more comfortable if you’re able to establish contact through a mutual connection. Explicitly mention the name of a colleague, supervisor, or department that will resonate with the participant as a trustworthy source.

At the beginning of the actual user research session—whether it be a contextual interview, usability testing, or a cognitive walkthrough—always conclude your introduction with some disclaimers that clearly define exactly what kind of information is being collected and how it will be used. Give the participant confidence that their identity will be protected even if their views and opinions about the broader organizational process and past design efforts aren’t so positive.

Understand that with anonymity ensured, some participants may be more willing than you imagined to “vent” about all different aspects of their user experience troubles, ranging from web applications to service design—sometimes even outside the scope of your study! Use this additional context to your advantage; refactor your designs with these external pain points in mind, and use them to draw big picture conclusions about holistic design problems that can’t be solved in the silo of the particular interface you were originally targeting for the study. Your stakeholders should appreciate that you’re trying to solve the design problem from more than one angle.

Telephone Sessions and Virtual Meetings

Even though user research efforts are sometimes limited by budget, scheduling, and logistical constraints that prevent you from conducting in-person studies, that’s no excuse for sacrificing your participants’ comfort and anonymity over the phone or through a virtual meeting.

In many ways, without the physical contextual clues offered by an in-person study, user experience professionals need to go out of their way to convey to remote participants that their anonymity will be protected. Imagine it from the position of the participant who suddenly finds him or herself in the middle of an in-depth phone conversation with someone they’ve never met, being asked to convey their opinions and attitudes about a product or service that may pertain to their employer. How comfortable would you feel in that situation?

Make sure you begin remote sessions using the handset of the telephone, rather than speakerphone; ask for permission from the participant to place the call on speaker, even if no one else is with you. If there is someone with you, make sure to disclose that person’s identity before continuing. You can address this as simply as “I’m joined by my UX colleague Sam. Is it alright if I place the call on speakerphone so she can listen, too?”

On the topic of undesired eavesdropping, particularly if you share work space with colleagues or stakeholders, make it clear that that during user research sessions unannounced “drop ins” are not permitted. Imagine the betrayal perceived by a participant hearing a superior chime in during the middle of a qualitative research question, “Hey who’s that on the phone you’re talking to?” You might also try to find private office space.

Summary and Findings

With your user research studies wrapped up and usability tests complete, you should be on your way to a well-informed, user-centered design solution. But what becomes of all that great feedback and input you’ve received from your users along the way?

Next up in your design process, you’ll surely find yourself sharing the fruits of your labor with project team members and stakeholders. They’ll be wondering what their users were interested in, what they thought about the experience of using their product or website, and sometimes, what their names are.

There are myriad of reasons why project team members or stakeholders may ask for the names of usability participants. Maybe the marketing team sees a cross-sell opportunity after hearing one of your customer journeys. Perhaps a project manager is concerned that a negative piece of feedback won’t go over well with a content owner and wants to see if the participant might want to think about the experience through a different lens. Or, conceivably a stakeholder overseeing an enterprise product might be unhappy with how a user group told you they’re “misusing” a company tool or website, and wants to personally put a stop to it.

As a responsible user experience practitioner, you’re not only prepared with the raw data to support your design, but you’ve also done your due diligence to properly scrub it of identifying contact information. There are a number of effective techniques for anonymizing participant names in your studies.

• Username Key. Always stored separately from your findings, create a table with each participant’s real name, each with a corresponding alphabetical letter. For example, Mary as participant A, Jose as participant B, and so forth. On the findings, each piece of feedback is only identified by the alphabetical letter. This is the most traditional technique for anonymizing user research participant data, but the method does add a bit of complexity for the practitioner, having to keep and maintain a separate protected file.
• Encoded Name.: To reduce the risk of at-a-glance identification by the vast majority of audiences—and without having to maintain a separate key file—consider web-based encoding tools. For example, the name Melissa Abraham is masked as TWVsaXNzYSBBYnJhaGFt when encoded using Base64. Without getting into the computer science of base encoding (just understand that it’s a technique used to convert clear text to symbols, often for the purpose of communication among machines), the names of your participants are relatively safe to eyes peering over your shoulder. Don’t mistake encoding for encryption however; the anonymity of your participants won’t be protected if shared electronically with a computer science savvy audience. This technique merely guards against casual attempts to glean identity in a presentation setting or working environment.
• Pseudonyms. If the audience for your user research findings is having trouble relating to alphabetical letter or encoded name representations, give your participant names some life—someone else’s that is. While typically reserved for persona profiles, using alternative names with stock photos can give your usability test results some character. Stakeholders might be more receptive to negative feedback about an anticipated feature when associated with a friendly-looking face, even if sourced from a third party digital asset library. Be careful about using funny names (“Silly Sally”) or names of famous people (“D. Trump”) as that might reduce the credibility of your findings.

Of course, the above recommendations largely apply to written findings—raw data and summaries. If you’re collecting feedback on designs using other formats, like audio or video, your ability to completely anonymize the content may be more limited. For example, while video recordings of usability testing and cognitive walkthroughs need only contain footage of the design being used (a web browser or mobile device), the participant’s voice will remain intact. Short of having access to voice distortion equipment, most UX professionals probably aren’t going to get around that. Instead, think carefully about when to begin recording; don’t begin until after introductions and names are exchanged. On the media files themselves, use file naming conventions absent of any identifying participant information as described above. Finally, take solace in the fact that you probably don’t need to distribute the raw data outside of your UX team anyway; you’re more likely to summarize findings in written form for stakeholders.

Rather than keeping these techniques in the back of your mind, try actively working them into the plans for your next usability test. Brief stakeholders on the value that anonymized user research will deliver to the design of the product. Before screening begins, understand where your candidates are sourced from. Update your research script to disclose anonymity protections as part of your greeting. Finally, make the necessary changes to your summary template so that your participant information is protected. The change in your findings will be subtle and gradual, but over time, you’ll glean those unfiltered insights that may have been held back before.

Topics: Trust, Privacy, Ethics, Enterprise UX
Published in: November, 2016 in Trust and Security

Comments are closed.