When conducting user research in the enterprise, recruiting can be one of the most pressing and constant challenges. Internal participants can be hard to recruit, often requiring buy-in and prompting from senior management. More care than usual must be taken with participant confidentiality to ensure participants don’t worry about potential repercussions from their feedback—after all, they are sometimes asked to participate by their managers. Finally, there’s scheduling to consider. For certain populations, such as sales people or executives, you may find yourself continually rescheduling sessions. This can lead to researcher frustration when what starts as a single day of testing 3-5 participants turns into 3-5 sessions scattered across a full week.

For external audiences, the customer base for enterprise software represents an even bigger challenge. Whereas colleagues or managers can convince internal participants, enterprise software buyers tend to be high level executives in operations, IT, or other business functions. Grabbing a few people from an internal department or visiting the local Starbucks to conduct rapid usability tests is unlikely to help you sell your research to overly critical colleagues.

For all these reasons, annual user group conferences can prove fruitful for groups performing user research in the enterprise space. Conferences provide an unparalleled opportunity to reach a concentration of qualified prospective research participants in a single location over one or more days, making it a potential goldmine for collecting rich user data. Planned well, with the right size team, you can get feedback on designs in progress, understand user needs to help prioritize new features, and reinforce the presence of a design and UX practice within your organization.

Regardless of how well you plan, you’ll also leave the conference exhausted, with an overflowing inbox from the time spent away from the office. This exposes you to the risk of going straight into playing catch-up when you return from the conference, leaving you unable to effectively synthesize and successfully use what you learned. So how do you know when the conference will be worth it?

When Conferences Are Worth It

There are several situations where a conference can be considered worth exploring for user research:

• When the audience is highly specialized and hard to recruit via traditional recruiting methods. For example, in cases where the product reaches a global audience, the annual conference may be one of the few ways your team can effectively recruit international participants without having to manage time zone conversion, potential language barriers, and other tricky aspects of running user research in-house. Additionally, for some products, the annual user conference presents one of very few opportunities to connect with high-value customers for interviews or short feedback sessions.
• When you are just starting a UX practice and still trying to learn about the user base. When you are new to an organization and beginning to bring user experience into the fold of activities, attending the annual conference can be a key way of expanding your knowledge of the institution and its products. For example, one of the first research activities I performed when I arrived at Pegasystems was to attend the annual Sales Kickoff conference where I was able to interview more than 20 members of the corporate sales organization and learn more about how the company sells its products. That research initiated various enhancements to both internal websites and our flagship corporate website. The information gleaned from talking to sales representatives also helped the web team better understand the specific types of issues customers care about.
• When conference or breakout sessions directly relate to research goals or provide opportunities for observation that will help inform personas or other UX work. As our company’s annual Sales Kickoff includes breakout sessions that provide essential training for the sales organization, I was able to gain valuable insight into the complexity of the sales process just by attending some of these sessions.
• When you’re sure that the audience is representative of the people you want to reach or the behavior you want to observe. For my master’s thesis, I spent more than a year traveling to conferences and meet ups around the world to study how contributors to Drupal, a popular open source content management system, worked together to improve the software. As these in-person events are crucial to moving the project forward and providing opportunities to complete focused work via contribution sprints, participating in these sprints and interviewing active contributors was a key component to understanding their workflow and challenges.
• When you want to build a recruitment panel for later use. Finally, one option for conferences, particularly as a new UX manager, is to use the conference to approach people about being part of a user research panel. Printing postcards with a link to sign up for your user research panel is a simple and and relatively inexpensive way to boost your UX practice within an organization, especially when it’s combined with interviews and other research at the conference.

When the Risks Might Outweigh the Rewards

While the above situations can make a strong case for going to the annual conference, there are other situations where you might want to consider whether the costs involved, in terms of both money and staff, actually outweigh the value you’d get from attending the conference.

• You primarily want to show off new work or demonstrate that the organization has a UX or design presence. Demonstrating your organization’s UX expertise through presence at an annual conference can be a valuable goal and win points politically within the organization. However, the expense required in creating and managing a booth, along with taking several members of your team off projects to prepare for and staff the booth, might outweigh the value you get from the experience.
• You have extremely limited UX staff. If you’re just ramping up and have a team of fewer than two to three people, taking them out of commission to attend a conference can significantly hurt your team’s productivity, particularly if you aren’t getting solid data out of the effort.

Assessing Your Research Needs

Like all research, you want to start by figuring out what you are there to learn. Maybe there is a process or workflow you’re trying to understand. Maybe you want to get some quick usability feedback on a new product you’re developing. Either way, you need to figure out what you don’t know now, and how that fits into what you can learn at the conference.

Some teams maintain a backlog of user research questions that you can use to identify research goals. You can also look at the format of the conference to see how various types of research activities can fit into your time there. For example:

• Which sessions, if any, will support your research goals?
• When are the scheduled breaks?
• What will the layout of the conference be like? Will there be room for a booth/table, or quiet places to sit down?

Based on your research goals, start to plan how you’re going to get the information you need. If you’re doing formative research, focus on the behavior you want to study, along with any content, information, or tools people might use to solve a given problem. Unless you’re actively doing usability testing during the conference, try to avoid focusing too heavily on the specific product you’re researching.

Research methods that work well at conferences

You can successfully employ several research methods at conferences; however, certain considerations are essential:

• Participants will be time starved. In most cases, you’ll be able to catch people for 10-20 minutes at most; choose methods that are easy to accomplish in a relatively short amount of time and avoid methods that take longer to complete. While it’s tempting to do full-scale usability testing, it’s difficult to do it effectively in a conference format, both from a recruiting standpoint and in terms of running a session without distraction.
• It is unlikely users will do things outside of the conference. Conferences generally include activities that occur throughout the day and into the evening; asking someone to spend 20 minutes filling out something or taking a usability test while they’re at the conference is unlikely to succeed. Stick to short surveys that you promote as extracurricular activities via postcards or social media during the conference.

Given these considerations, three methods seem to work best for conference-centered research:

1. Interviews and observation. If you have a solid protocol, it’s possible to conduct an interview within 10-15 minutes, about the maximum attention span of most conference attendees outside of a session. Another benefit to using interviews is that in many cases, it is indistinguishable from networking. While I always let people know that I’m conducting research and that their feedback is confidential, some of the most interesting data I have received from participants has been sitting on the plane going to the conference, catching people during coffee breaks, or networking with people after sessions are over.
2. Short Surveys. A brief, 5-minute survey, such as an invitation to a research panel or questions about specific workflows or preferences, can work well as a way of getting data from many people, as well as engaging people for future research activities.
3. Games and short participatory design activities. These can bring some extra fun to conference attendees while giving them a “taste” of design thinking. If you have a booth at the conference, consider different activities you can include that will allow people to provide very quick feedback about problems they’re facing or features they need. In the recent UXPA Boston Conference presentation, “Your Company’s Annual Conference: Boon or Bust for UX,” Chauncey Wilson talked about giving users fake money and asking them to “buy” new features for the product as a way of quickly gauging user preferences. Another option is to print postcards or custom sticky notes that allow participants to write their own job stories as a way of quickly understanding the types of activities they should engage in. Once the participants fill out the card, you can display them in the conference booth as a way to add visual interest and entice more people to engage.

To Booth or Not to Booth?

If your company’s conference is larger, you may be able to convince the organizers to give your team a booth or table on the tradeshow floor. Having a booth gives you some distinct advantages. First, it provides people with a consistent place to find you and participate in activities. Second, it can help to create awareness of the company’s focus on good design and UX. Finally, a booth gives you opportunities to experiment with different types of activities to gather data and keep participants engaged. At the annual Red Hat Summit, design architect Serena Chechile Doyle engaged attendees in participatory design activities using repositionable dashboard components, and conducted hundreds of customer surveys. At EMC World, UX designer Joanna Hubbard gained insight on feature ideas by asking customers to finish the sentence “I want an EMC mobile app that will….”

However, it’s important to note that the effort to plan, staff, and maintain a booth at a conference—particularly at a large one—is considerable. You’ll need to:

• Design and manufacture signage for the booth
• Design and create postcards, swag, and other assets
• Create multiple research plans
• Bring people to staff and run the research
• Make sure people are available to set up and break down the booth

All of this interferes with the team’s available time for day-to-day work, both in the months before the conference and the time after the conference. And since your team’s booth may take the place of a sponsor’s booth, you may have to spend significant time convincing management to have a booth in the first place, possibly further distracting you from other priorities.

Keeping Yourself—and the Team—Sane

It’s hard to understate how overwhelming research at conferences can be. The nature of conferences makes it such that, whether you have a booth or not, you’re running around constantly. If you don’t have a plan, it’s easy to get caught up in the rush of things, miss important details, and get overwhelmed. Below are a few tricks to help you stay focused.

Have someone else do the recruiting

The fast-paced nature of conferences means that unless you have extra people around you it’s going to be nearly impossible to keep up with the number of participants by yourself. If you have a team, make sure at least two of you are at the conference to schedule sessions or find willing participants. If you have a booth at the conference, have one person work as a “barker” to sign people up for research slots, while another team member runs interviews/usability sessions. If possible, have a third person available to take notes and observe so the interviewer can focus on moderating. To keep everyone sane, have team members switch roles periodically between sessions.

If you aren’t bringing a member of the team, find a sponsor who knows everyone and is comfortable directing people toward you. Treat this person well, and have their phone number on hand to coordinate interview participants during breaks. If you’re unable to find a research sponsor, think about snowball recruiting where one participant recommends another and so forth.

Leave the laptop in your room

Laptops are frustrating when you’re running around a conference doing research. After a full day walking around the conference floor your back will be sore by the time you get to lunch. Additionally, laptops are hard to manage when you’re trying to have a conversation in a noisy environment and present a barrier between you and the participant when you’re trying to build a rapport.

During my last conference, I used a combination of an audio recording app on my iPhone, Evernote, and my Rocketbook Wave notebook to record interviews and take notes. Using Evernote on my iPhone was useful during situations where it was either too loud to record, or I was having casual conversations that turned into impromptu interviews. The notebook allowed me to take notes during sessions, and quickly sketch process flows as they started to become clear (Figure 1).

Figure 1. Notes from sales process research, Pegasystems Sales Kickoff 2015.

ALT text for accessibility: Page of notes with process graph of sales process and roles.

Record, Record, Record

The first time I did research at a conference, I forgot to bring any kind of recording equipment. Bouncing from interview to sessions to interview, I found myself forgetting what I had just heard almost immediately after I heard it, and had to rely largely on hastily scribbled notes to make sense of my findings. Thankfully, the next time I was prepared with a recording app on my iPhone (Highlight by Cohdoo that was tragically discontinued), which gave me something to return to in case I couldn’t remember a conversation afterward. While observing, it helped to occasionally take pictures of artifacts or collections of information, particularly when performing participatory design activities (Figure 2).

Figure 2. Drupal contribution sprint, NYC Camp 2014.
ALT text for accessibility: Photo of tasks required for key Drupal 8 initiatives organized via sticky notes during a contribution sprint in New York City.

Aggressively defend your alone time

Be sure to make time to find a quiet location to reflect and write notes; you can also set aside time after interviews to write down key points. Some of the most interesting information can be gleaned during evening events, so you’ll need your energy. Don’t be afraid to take a quick nap if you can.

Conclusion

If you plan carefully, user group conferences can yield invaluable information about the workflow and needs of your product’s users. With a good team, you can get feedback on designs in progress, understand user needs to help prioritize new features, and reinforce the presence of a design and UX practice within your organization. When you’re an internal consultant, it also lets users know that you care about their needs, increasing their loyalty to your products. Have fun, take care of yourself, and learn something new!

Topics: User Research, Enterprise UX
Published in: July, 2016 in Research Methods

Comments are closed.