As many UX professionals know, unexpected issues seen in the context of actual deployment can trump an initial rosy impression about the usability of a system. These kinds of surprises can be especially dramatic when working in the area of ICT4D (Information and Communication Technology for Socio-Economic Development), where technology applications are designed to alleviate poverty among economically disadvantaged populations.
This article describes a case study of the deployment of a prototype low-cost digital slate for assisting the tracking of child malnutrition in rural India. The prototype runs on Windows CE and looks like a small clipboard with a calculator at the top (see Figure 1). It accepts handwritten input on ordinary paper notebooks placed on the A5 size-digitizing pad of the slate, and provides immediate electronic feedback on the 3.5 inch interactive touch screen display. The slate captures writing from a digital ballpoint pen as raw strokes, and the back of the pen serves as a stylus for touch screen input. The digital slate system allows for a fairly seamless transition from older, paper-based systems to digitally recorded information. In addition, the slate preserves the paper copies of all forms; this is particularly valuable in developing world contexts where paper records are still very important. We designed the system for people who have little or no experience working with digital systems beyond simple cellular phones or calculators.
Figure 1. Village woman in India working on the prototype digital slate.
Prior work, coupled with our initial controlled user trials, suggested that users strongly preferred the current design to other alternatives, and were eager to use it in the field. But in a follow-up three-month field trial with ten low-income, low-literate health workers in rural central India, a number of unexpected challenges were encountered that overwhelmed our initial optimism. This experience demonstrates that more attention needs to be paid to issues well beyond the simple usability of technological systems, to include broader socio-technical concerns that arise in real-world settings.
Previous Work
In previous explorations, we had tested the slate with low-income users in a rural microfinance setting. India’s extensive microfinance network brings formal savings and credit services to 86 million poor households. Yet, the inability to maintain high-quality records remains a persistent weakness in the smooth functioning of microfinance groups. We worked with a non-profit organization that facilitated microfinance loans among self-help groups. They faced a range of problems with their paper forms, including errors in arithmetic calculations and legibility issues during transcription, to a computerized management information system (MIS). Often, certain mandatory fields in the paper forms were left empty. Interruptions in the physical transport of the paper forms from the meeting to the transcription location and back caused further delays in resolving these errors. We studied these problems and built a financial record management application on our prototype. In a two-day supervised trial with 200 microfinance group members in rural eastern India, we saw that the use of the slate solution resulted in shorter data recording time, fewer incorrect entries, and more complete records, compared to the existing paper-based system. Furthermore, the slate device won strong approval from the microfinance groups and partner organization we were working with.
Adaptation to Health Context
With the success of this experience, we began looking for other potential applications for our prototype. We were approached by a non-governmental organization (NGO) that manages a malnutrition tracking and treatment program for about 65,000 children under the age of five across five districts in rural central India. The Real Medicine Foundation (RMF) was facing persistent problems with long delays in aggregating per-child data in their existing paper-based system. Local health workers record child-specific data in paper diaries during household visits, and this information is subsequently transcribed into ledger books that are eventually aggregated into a computerized MIS. It takes about a month for the information collected by health workers to make its way to decision makers, resulting in serious delays in remedial actions, such as providing emergency food rations and medical care.
We worked with RMF to build a .NET Compact Framework health data record management application on our slate prototype. The application’s design matched the format and workflow of the existing paper forms and diaries. All text labels and text field entries were in Hindi, the language spoken by the health workers. Based on the pen’s location on the digitizing pad, the application detected the cell of the paper notebook that was being filled. Each handwritten numeric digit printed on paper was simultaneously digitized, run through a digit recognizer, and placed in the corresponding field on the screen. We saved all the data on the slate’s micro SD card, which was transferred to the health worker’s phone and sent to the backend database via general packet radio service (GPRS) at the end of each day. The server was updated daily, and a summary of the data was made available to decision makers. Finally, as noted above, the slate seamlessly provided a paper trail and backup, which was seen as a critical feature to RMF.
A one-day training and evaluation session of the application was conducted with ten health workers, which was very encouraging. All of the health workers were women who were from the small town of Khandwa and surrounding villages in the district, with household incomes between USD $80-150 per month, and relatively little education. All had at least some exposure to technology (four of the women owned a mobile phone, and the rest had received calls on a family member’s mobile phone at least once).
The slate application was demonstrated to groups of five and had each worker practice individually for an hour. We were excited to see that workers quickly picked up the familiar pen and paper interaction and were very enthusiastic about it. Our partner NGO was equally pleased by our application and wanted to deploy the system. Following the success of the slate prototype in the microfinance context, this was the second feather in our cap and we were eager to see how our prototype could help fight child malnutrition in India!
Buoyed by the positive results, we deployed our prototypes with the same ten health workers for a field trial. For three months, the health workers were to visit individual rural households and use our system to record child-specific data. At least, that was the idea. Instead, the initial optimism we felt from the training was quickly overwhelmed by a number of challenges that rendered our prototype unusable.
Figure 2. Rural woman doing data collection for the Real Medicine Foundation.
Challenges Encountered
1. Application usability
Despite laboratory testing of the software, there was a limit to the real-world usage scenarios we could simulate in our lab. This was all the more challenging because the literacy level and information and communication technologies (ICT) experience of our end users was very different from the software testers. One worker from a remote village once accidently clicked on the Windows Media Player icon on the desktop instead of our application icon, and was not able to use the application for three weeks because no one in her village knew how to get back to the desktop screen. On a number of occasions, workers accidently closed the application without saving data they had entered. Workers often had problems using the stylus to double-click and ended up executing various functions that they were not trained to use. They landed on unfamiliar screens and did not know how to make their way back to the application. Given our limited time and resources in the field, we were not able to observe and design for such scenarios. Based on observations from the training session we did introduce some changes while in the field, but there were many issues we were unable to foresee.
2. Physical infrastructure
There were problems associated with the poor physical infrastructure of rural India. In some of the remote villages where the health workers lived, there were regular power outages of twenty hours throughout the day, with wildly varying voltage levels. The slate device required a power connection for around six hours to fully charge the battery, so devices were often not completely charged. On some occasions, the batteries died while using the device at patients’ houses, which seemed to embarrass the health workers.
3. Device management
According to the health workers, the odd size of the slate made it awkward to transport. Many workers traveled for work in very crowded public buses and found it difficult to manage the device in the rush; the only ICT they carried with them were mobile phones which easily fit into their handbags. Others traveled to work on the back of their husbands’ bicycles through muddy, uneven village roads. Bicycle crashes were very common, and workers were worried that they would break the device when they fell. Finally, our trial coincided with the monsoon season; in spite of multi-layered packing in plastic sheets, the health workers were concerned that water would seep through and spoil their devices during transport.
4. Human
Finally, there were a host of human-centered issues that worked against a successful trial. While our NGO partner helped us with many logistics in the field, they had very little technical competence. When health workers experienced technical glitches on the device, they would bring it to their sole point-of-contact, the program coordinator of the organization. However, the coordinator typically lacked the expertise and training to fix the glitches. While she could reach out to us over the phone for remote troubleshooting, she was reluctant to do this. We suspect that her lack of enthusiasm may have been partially due to the fact that she was not particularly vested in the project. We had not involved her in the initial training program, and so she had little or no ownership in the success of the slate intervention program.
A couple of the health workers tried seeking help from local mobile phone shop owners and photo studio owners with computers. However, due to lack of knowledge about the digital slate device, they were turned away from there as well.
Conclusion
The experience with our prototype in rural India exposed a number of broad challenges in deploying a promising technological system into a real-world setting. We believed our initial, limited-user trials had shown that our slate solution was an unqualified success. However, like many travelers before us, we found that the road to success is not so simple. During actual deployment of the system in the wild, we experienced a number of unprecedented challenges that marred our initial optimism. We conclude that there needs to be deep attention beyond simple usability to broader socio-technical issues that may arise in field deployment of technological systems.
Retrieved from https://oldmagazine.uxpa.org/the-fate-of-a-digital-slate/
