From time to time, organizations need to review all their forms, a task often performed by inexperienced staff with limited resources. This article is based on the author’s recent experience in assisting with such a review for Centrelink, an agency of the Australian Department of Human Services that handles social security, veterans’, and similar types of payments.
Numerous studies of both government and non-government forms have shown typical rates of error by form fillers at 80 to 100 percent. Almost all of this organization’s completed forms had one or more errors in the data collected. Some errors may have been trivial, but they were still errors and were costing millions of dollars per year to correct.
Many of the forms were difficult for people to understand. The problem became so bad that Joe Hockey, Minister for Human Services, demanded that something be done. In 2006, his letter to the head of the agency said:
“Centrelink should strengthen its focus on improving communication with customers…I will continue to take a high level of interest in improvements you make to letters and forms to make them easier to understand and use for customers… In 2004-05, the percentage of customers rating the ease of completing Centrelink’s forms as ‘good’ or ‘very good’ was 58%. I expect you to significantly increase customers’ ease of providing necessary information to Centrelink…
“Centrelink’s culture must be responsive to the needs of citizens and stakeholders, not a culture that is unduly defensive and process orientated. The organization should be willing to question itself and its performance on an ongoing basis
As part of this you should fully explore whether any criticisms and customer concerns are valid and, if so, take appropriate corrective action….
“The challenge for executive management is to recognize potential weaknesses and ensure that arrangements for monitoring, assessment, reporting, and review are sensitive to the actual operating environment. In particular, the arrangements should provide for adequate and early feedback to enable corrective action by management, and there should be clear triggers for oversight and involvement at executive level. This requires the identification of problems and acknowledgement of those problems by senior management.”
Realizing the normal government approach to improving forms takes a long time to come to fruition, the minister commissioned a parliamentary committee headed by Senator Colbeck to oversee the project and to ensure that progress was not hindered by red tape. Having a parliamentary committee conduct oversight on the project at this level ensures that decisions are made at the highest management level.
Professional forms analysts spoke to the committee at its first meeting in July 2006. Many issues familiar to forms analysts were discussed, and I was impressed with committee members’ understanding and their desire to improve the lot of the public in filling out forms.
However, the initial reaction on the part of departmental staff was essentially one of fear and disbelief. They were to be given approximately twelve months to redesign all of the department’s major public-use paper forms and make them available for the public to use. With past lead times extending over vastly longer periods, the general consensus was that it was impossible for the project to be completed on time. There are some forms that couldn’t be completed due to technical difficulties, and the size of the task means that the project is still on-going. The initial work was so successful, however, that the new Labor government is continuing the project using the same approach, and not allowing politics to interfere with public usability.
Common Questions
The work is based on a great deal of scientific research into the way people use forms (see sidebar). The first tasks in 2006 began with a series of workshops on the most problematic forms, during which a lot of repetition became apparent. As a result, a series of common question sets were developed to be applied across the board. For example, most forms include questions asking for details such as name (and whether respondents had used other names), address, date of birth, sex, and contact details. Once the questions were worked out, they could be applied to any form asking for the same information.
The question sets have been designed for easy pasting into any form. Form colors change automatically to match those of each form, and all that needs to be done is to change the question numbers.
As an example, the set of name questions must allow for various cultural issues in today’s Australian society. Figure 1 is an example of how the questions appear in the common question set.
After further review, some sets of questions were amended, but they have remained substantially the same. With the common question sets in place, redesign of many of the forms is faster than originally estimated.
Figure 1. Example of a common question set.
Layout Approach
The designs are based on the half-page column approach introduced in the UK in the early 1980s and subsequently used in many Australian forms. This layout results in a significant reduction in user error and, at the same time, saves a lot of previously wasted space. Figure 2 shows part of a page using the new half-page column design.
But just copying this layout style without understanding the reasons for the various components will invariably lead to trouble. When graphic designers latch onto appearance and copy styles without understanding the reasons for the design elements, the result is forms that often don’t work as required.
Figure 2. Example of a half page column layout.
Signature block
A huge problem for many forms is respondents who don’t sign in the spaces provided. The tested solution was to include a declaration and signature block together as a numbered question, as shown in Figure 3.
Figure 3. Example of statement and signature block as a numbered question.
Check list
Most forms had a check list at the end, and people ignored it. Again, the solution was to include it as a numbered question, as shown in Figure 4.
Figure 4. Example of check list as a numbered question.
Additional pages
Questions requiring attachments had additional graphics to highlight them, as shown in Figure 5. The use of the paperclip icon was found to be effective in enabling form fillers to go back and locate the question.
Figure 5. Example of use of paperclip icon to link to check list.
Reading path
The old designs sometimes didn’t allow for the way people read a form. Figure 6 shows a typical example. In it, the reader typically goes from the bold question text (ending in “private trust”) to the No/Yes ballot boxes, and misses the instructional text below.
Figure 7 shows a better approach. The addition of the bold header about reading the note achieves close to 100 percent success.
Figure 6. Example of old design that ignores typical form filler reading path.
Figure 7. An improved layout for instructions.
Project Success
Aside from using proven design and language techniques, the greatest factor in the success of the project has been the use of error analysis and usability testing. Both methods help to reduce the political problems associated with changing someone else’s designs—a common forms management problem.
Error analysis
As an important prerequisite for data gathering, error analysis helps to identify potential problems to deal with during redesign and usability testing.
We conducted a number of error analysis studies in which we examined many completed forms to identify where people typically went wrong in filling them out. Then we were able to quantify the real cost of the forms. Most organizations consider only printing and processing costs, but when close to 100 percent of forms have errors, the cost can run into hundreds of thousands of dollars per form and, in at least one case, over a million dollars in one year—that’s the cost just to repair the errors.
These figures are typical of most organizations. We don’t yet have data on the revised government forms, but another example will illustrate the potential savings. We redesigned the application forms for one of our major life insurance companies. Prior to redesign, 100 percent of their forms came in with one or more errors. After usability testing and redesign in the same style as Centrelink, the company reported that the error rate had dropped to five percent, and most errors were minor. Besides reduced errors, form fillers in the testing took only twenty to thirty minutes to complete a 24-page application accurately and without assistance.
Usability testing
The agency previously used focus groups to test forms, but our experience over the past twenty years shows that focus groups are useless for testing usability. They provide some information about people’s opinions, but next to nothing about how a person actually uses a form. Time constraints and form complexity can lead to times when usability testing isn’t feasible, but these are rare and we prefer to test all forms prior to release.
During the Centrelink project, all testing was carried out in a motel room with the observer(s) sitting across the table from the form filler. With two, or occasionally three, observers, there was no indication of hindering the form filler. The agency has recently built a new testing facility with a specialized observation room and video capability. Recent testing still allows the main observer to sit with the respondent and for both to be recorded. The form itself is also recorded, which is very important when testing form filling.
All in all, the redesigned forms are expected to save the Australian government millions of dollars per year by reducing the need to contact customers, correct errors, and deal with bad data.
A summary of Lessons from Forms Research accompanied this article.
Lessons from Forms Research
By Robert Barnett
This summary accompanied the article Redesigning Centrelink Forms: A Case Study of Government Forms
The story of scientific forms research began in the mid-1970s and continued on through the 1980s and 1990s with UK researchers such as Robert Miller, Philip Barnard, Patricia Wright, and Robert Waller; Janice (Ginny) Redish and her team from the Document Design Center in Washington D.C.; David Frohlich; Robyn Penman, David Sless and others from the Communication Research Institute of Australia (CRIA).
We can never rest on our laurels and stop learning about forms. People who claim to know all there is to know about form design are kidding themselves. Methods for examining forms don’t have to stay in universities and research institutes. Any forms analyst can learn them and begin to apply the principles in day-to-day work. The following are lessons we’ve learned from research over the past thirty-plus years.
Patricia Wright and Philip Barnard came to the following conclusions:
- Questions should deal with one thing at a time.
- Forms should use familiar words.
- Designers should consider alternatives to prose, for example: a string of conditions separated by conjunctions might be more easily understood if written as a list.
- Provide adequate answer space.
- Type size should be at least 8-point and preferably 10-point.
- There should be good contrast between printing and background.
In the later 1970s, studies by Wright, Barnard, and Wilcox dealt with the constraints placed on legibility by the use of character separators, often referred to as delimiters. The computer world had introduced the idea of little boxes for each character, and later changed to small tick mark combs on the bottom of boxes to separate the characters. The research showed how the use of such marks slowed reading of the forms during data entry. (This research went hand-in-hand with my own observations that such marks even cause significant errors in reading.)
Patricia Wright’s early research taught us about our poor understanding of basic forms issues. A lengthy article in Visible Language reviewed research investigations into form design and usage up to 1980. One of the most significant conclusions was that, “those who seek simple recipes for designing adequate forms have failed to understand the complexities of the problem.”
In 1982, Grant, Exley, Lonsdale, and Goddard produced a major British government report, “Forms Under Control.” While it didn’t include much in the way of new design knowledge, the study showed the extent to which forms need to be controlled and the problems people faced with government forms. The report came up with sixty-two recommendations on the management of government forms.
In 1984, Robert Waller (now professor of information design in the Department of Typography & Graphic Communication at the University of Reading) reported on the design of a British government form,
“Claim for Supplementary Benefit.” The information from the project provided an excellent starting point for later researchers. Some conclusions would be modified by subsequent studies, but most of the general conclusions still apply to public-use forms:
- Short documents are not necessarily easier than long ones.
- All the textual variables in a form interact.
- Test results require careful interpretation.
- Where a form is drawn with two-column arrangements, the columns need to be visibly separate with space or a strong rule separating them. If these layouts are used and it’s necessary to include full-width items, these are best placed at the top so as not to interfere with the reading of the other columns.
- With open-ended questions, the size of an answer space can indicate to the form filler how long the answer should be.
- When a form contains hierarchical information, this is best indicated by the graphics rather than the text.
- If color is needed, use mid-tones that are both legible and conspicuous.
- Small page formats offer less design flexibility.
In 1985, Ginny Redish of the Document Design Center at the American Institutes for Research and Jack Selzer of Pennsylvania State University, published an important article on “The Place of Readability Formulas in Technical Communication.” The report highlighted how inappropriate the use of formulas such as the Flesch Reading Ease Scale and Gunning’s Fog Index are for technical communication.
In 1986, David Frohlich (now director of Digital World Research Centre at the University of Surrey and professor of interaction design) undertook one of the most important pioneering studies in forms research. His conclusions, recorded in the paper “On the Organisation of Form-filling Behaviour,” form the basis of the observational study approach we still use today. He summarizes his findings on the way people use forms as seven question principles:
- Linear progression: work through the questions in the order they appear on the form.
- Least reading effort: only read what seems to be necessary to maintain form-filling progress.
- Question routing: jump directly to a new question if the form tells you to.
- Question omission: miss out questions which don’t seem to apply to you.
- Question preview: if in doubt about the meaning of a current question, read the subsequent question.
- Question review: if in doubt about your interpretation of the previous question, review that question and the answer provided.
- Topic scan: if in doubt about the relevance of the current question topic, scan the local topic context.
We consistently find during usability testing that if any of these principles are violated, people tend to make errors in their form filling.
In 1990, the Communication Research Institute of Australia (CRIA) conducted the world’s first study on the way people use life insurance application forms. One hundred percent of the traditional forms produced one or more errors. After redesign and usability testing, error rates were reduced to fifteen percent, and most errors were trivial. The savings in processing time provided the funding to maintain the whole Forms Management Department. Robyn Penman and David Sless from CRIA reported on studies of insurance documents showing that just designing them by following rules of plain English was not good enough.
In 1999, Michael Tyler from Robert Barnett and Associates reported on a series of usability studies on forms produced by different government departments for aged people. The studies showed that many of the lessons learned from research with younger people didn’t apply. For example, simple form-filling processes such as sequential reading of questions, were replaced by random scanning of pages, and answer examples were often misinterpreted as being the only possible alternatives. The studies also highlighted the special needs of aged people.
Retrieved from https://oldmagazine.uxpa.org/redesigning_centrelink_forms/
Comments are closed.
