Figure: Dual switches.
Imagine this situation: you have been asked to evaluate a kiosk for access to government services to see if people with disabilities can use it. One of the personas for the project is Virginia Martz, an academic and employment accommodations specialist with the Resource Center for Persons with Disabilities (RCPD) at Michigan State University. She has manual dexterity issues related to osteoarthritis; she cannot hold her arms up for extended periods of time and experiences pain when she does repetitive tasks, such as keyboard typing and button pressing.
The Michigan State University Usability/Accessibility Research and Consulting (UARC) laboratory found itself in just this situation recently. When we tried the product ourselves, we discovered that users would have to press a button over 1,200 times to complete the task using the two-button interface designed for people with manual dexterity issues. That’s assuming they made no errors. Each mistake could cost more than 100 extra button presses.
Two-button switches, like the one shown at the top of the page, are necessary for people with extremely limited use of their hands to interact with technology. Virginia, who cannot use the standard inputs on the kiosk due to osteoarthritis, would experience difficulties with the alternative input due to the number of button-presses required to complete the task.
Asking a person who tires easily or experiences pain when performing manual tasks to click a button a few thousand times would cause, at the very least, significant discomfort and likely lasting physical pain. When we were trying out the product to collect expert timing and button press data, our hands hurt! Projects like this raise important questions about research ethics.
We asked colleagues with disabilities to give us their feedback on using the product. Based on their responses, it was obvious that running a usability evaluation was not going to be reasonable. This experience got us thinking about our professional responsibilities regarding ethics in user experience research.
What are the types of situations in which it is unethical to conduct a user experience evaluation?
Having worked as researchers, practitioners, and consultants in academic, commercial, and military environments for many years, we have been responsible for determining whether or not it is appropriate to use human research participants in various types of projects. The decisions are usually more straightforward in both military contracting and university research settings because the projects generally fall under Federal Institutional Review Board (IRB) regulations, and therefore, have institutional oversight from a scientific, ethical, and regulatory standpoint.
If you are not familiar with the IRB process, IRBs are special oversight committees empowered by the United States Food and Drug Administration and Department of Health and Human Services. They are responsible for approving, disapproving, modifying, and reviewing biomedical and behavioral research involving humans.
When research involving human subjects is done by higher education and government organizations, it must be approved and overseen by an IRB. Most colleges and universities have their own internal IRB, which has final authority over research with human subjects, regardless of the funding source or project scope. All investigators have to complete regular IRB investigator training in the protection of human research subjects, and all project applications made to an IRB must detail the risks to participants and explain how risk will be minimized, confidentiality and anonymity will be ensured, and how vulnerable populations will be protected. These questions must be satisfactorily answered and the application formally approved prior to any contact with subjects. Thus, the IRB process forces researchers to think about protection issues for every human research project.
What about usability professionals in commercial settings?
In the commercial environment, the decision about whether to proceed with a usability evaluation is usually up to the individual usability specialist. In fact, when the need for informed consent paperwork is brought up, it is common to hear responses from, “Oh, we didn’t think about that. Is it really necessary?” to “We don’t want to alarm participants by bringing up potential, and unlikely, risks of participation.” So, the question then is, “What are our responsibilities as professionals?”
Federal, state, and local laws may apply to testing for certain commercial projects. The “Federal Policy for the Protection of Human Subjects,” or “Common Rule,” requires organizations that receive U.S. government funds to have approval from their organization’s designated Institutional Review Board before conducting any research involving human participants. Mainly, the Common Rule includes requirements for obtaining informed consent, institutional record keeping, and additional provisions for certain vulnerable research participants.
The Office of Human Research Protection provides a handy list of the 1,000+ laws, regulations, and guidelines in the form of the “International Compilation of Human Research Protections.” Depending on the project and jurisdiction, there may be other regulations that must be complied with, even if federal funds are not involved.
Additionally, the UPA Code of Conduct (and similar codes from related professional societies, including STC, HFES, APA, ACM, IEEE, and others) lays out our responsibilities in clear, unambiguous, and simple terms. UPA, and now UXPA members, are required to:
- Act in the best interest of everyone
- Be honest with everyone
- Do no harm, and if possible, provide benefits
- Act with integrity
- Avoid conflicts of interest
- Respect privacy, confidentiality, and anonymity
- Provide all resultant data
What responsibility do we have, as professionals, to be sure that we are protecting participants, especially when their physical abilities might be a performance issue?
Specifically, is it ethical to use people with disabilities to test the usability of a system that you know to be hard to use, and may, therefore, cause them unnecessary pain or discomfort?
Carefully thinking through potential ethics issues in a usability evaluation involving participants who have disabilities is critical since they are more likely to experience significant discomfort or physical harm as a result of participation, and are generally considered to be a vulnerable population. Just as physicians are bound to “first, do no harm,” the burden is on user experience professionals to ensure that conducting an evaluation will not cause unreasonable and unnecessary discomfort or, worse yet, physical harm, to the participant.
In the project with the two-button device, it became clear after we tried using the product ourselves that agreeing to conduct this evaluation could possibly violate human participant protection requirements. We recommended against performing a usability evaluation with persons with disabilities at that point. Instead, we proposed conducting a detailed user interface review and accessibility compliance inspection. We would then conduct the usability evaluation with the participants who have dexterity limitations after the company enhanced the product to address the usability and accessibility issues found in the expert reviews, thereby reducing the risks to participants.
A broader, related question is what responsibility we have as professionals to be sure that the products we test are ready for full-scale testing?
When clients ask us to perform an accessibility compliance inspection, we always ask them a few basic questions and conduct a one-minute inspection before accepting a contract. For us, this is a professional integrity issue because we believe that it is not appropriate to take money from a client when a detailed evaluation cannot be conducted because of major accessibility failures—failures which can be fully documented or explained in a few minutes.
The same issue comes up in usability evaluations and heuristic user interface reviews. We strongly recommend that all products be briefly pre-tested for accessibility and user interface design, as well as best-practice guidelines, before undergoing testing with participants. This approach not only mitigates the time and cost of running studies on products that are likely to fail, but will also do much to prevent unnecessarily subjecting individuals with disabilities to frustrating, painful, or difficult situations.
What is your response as a user experience professional to these ethics questions?
Every user experience professional is responsible for understanding and applying the principles of human research participant protection in design and testing. Review the UPA Code of Conduct, institutional guidelines and requirements, and federal, state, and local laws; and become familiar with human research participants considerations. Finally, remember the decisions we make impact real people, and we are responsible for their safety and wellbeing.
Retrieved from https://oldmagazine.uxpa.org/should-we-conduct-this-usability-study/
