A review of
Quantifying the User Experience
By Jeff Sauro and James R. Lewis
Morgan Kauffman, 2012
A review ofIn Quantifying the User Experience, Jeff Sauro and Jim Lewis accomplished two very important goals. First, they provided a convenient reference text for practitioners who need to measure the user experience and need guidance to determine the appropriate way in which to do the measuring. Second, they uncoupled the complex statistical theories underlying those methods of measurement from the application of the measures themselves. The former makes all of us in the UX field more effective by helping us select the right measurement tools to get the job done. The latter gives us the ability to apply those tools with confidence, even if we cannot recite from memory the relationship between a critical value of t and degrees of freedom, or a confidence interval and a standard error.
Practical Statistics for User Research is the perfect subtitle, as the practical application of statistical tools is truly Sauro and Lewis’ primary focus throughout the book. If you endeavor to thoroughly understand the formulae used to calculate statistical measures, you can find that in Quantifying the User Experience. However, if all you seek is an understanding of a measure sufficient to apply it appropriately, this as well can be found with ease. Sauro and Lewis have demonstrated a propensity for making sometimes complex or convoluted topics accessible to those who lack a strong background in statistics.
This is not the ivory tower statistics textbook you had to read in graduate school. Sauro and Lewis acknowledge the theoretical underpinnings of the statistics discussed, but for every commonly held academic convention described, a practical approach is also provided. For example, in statistics courses we are typically taught that in order for an observed difference to be considered significant, p must be less than .05. Sauro and Lewis remind us that this criterion is largely arbitrary and, when conducting statistical analyses in industry, a less conservative value of α is very often appropriate.
Many other classic debates in measurement and statistics are discussed in this book, including:
- Do you need to test at least thirty users to estimate or compare means?
- Is it okay to average data from multipoint scales?
- Which is it: the “Magic Number 5” or “Eight Is Not Enough?”
In each case, Sauro and Lewis give the background for each side of the debate and provide just enough detail for the reader to make an informed decision about which approach best matches the problem they are trying to solve. For every academic absolute, a practical consideration is provided; my favorite is that while it took Einstein years to give us E = mc2, you don’t have to be a genius to understand the economics of sample size (n = $). “Fishing with Jakob Nielsen” was also as entertaining as it was informative.
In addition to tackling common debates in statistics related to sample size, significance testing, and choosing the most appropriate measure, Quantifying the User Experience provides readers with a valuable resource for selecting the most appropriate standardized usability questionnaires. Evaluations of both post-study and post-task questionnaires are included, and rather than simply describing the standardized questionnaires that are available, Sauro and Lewis take it a step further by providing guidance on assessing the reliability, validity, and sensitivity of those questionnaires, as well as recommendations for their questionnaires of choice.
In my opinion, the true value of this book does not lie in the clear and concise manner in which the authors describe statistical measures—though that is certainly valuable in and of itself. The most unique contributions of this book are the logic and practicality used to describe the appropriate application of those measures. Those of us tasked with quantifying the user experience know that convincing stakeholders that we are using the correct measures is just as difficult, if not more so, as selecting the measures themselves. The authors provide readers with the rationale needed to make informed decisions about the application of statistical measures, but they also provide sufficient detail to understand both sides of every argument so that when questioned, researchers have the information needed to defend the validity of their results.
Sauro and Lewis strike a perfect balance between the complexity of statistical theory and the simplicity of applying statistics practically. Whether you wish to delve deeper into the enduring controversies in statistics, or simply wish to understand the difference between a t-test and Chi-square, you will find your answer in this book. Quantifying the User Experience is an invaluable resource for those who are conducting user research in industry.
Retrieved from https://oldmagazine.uxpa.org/the-p-stops-here/
Comments are closed.
