Article from the Boston Globe’s On Call Magazine, March/April 2005

By Helen Osborne, MEd, OTR/L
President of Health Literacy Consulting

You just wrote a new patient-education document. You checked with content experts (doctors, nurses, scientists, and others with subject-matter expertise) and confirmed that all the important information is included. You used words you assume are clear and simple, double-checking with a readability assessment tool to make sure the document is written at an appropriate reading level. And you added drawings, photos, charts, and other visuals to enhance the document’s appearance. But will your intended readers understand? You won’t really know the answer to that until you ask for their feedback. Indeed, intended audiences are the only true experts who can tell you how understandable, useful, and readable your written materials are.

Elizabeth Hoy, MHA, president of Polaris Consulting in Vienna, Virginia, knows a lot about reader feedback. Experienced in health policy and qualitative research, Hoy develops and tests a wide range of consumer materials. She knows firsthand the importance of finding out what readers have to say about it. “When we tested a consumer fact sheet with readers, we found out all sorts of things that people didn’t understand,” Hoy says. “And we learned how to improve the document in ways that didn’t occur to us.”

Archie Willard also appreciates the importance of reader feedback. As a new reader (an adult who recently learned to read), Willard wants to be asked his opinion about written materials. “If I can read and understand this, then it is worthwhile for everyone,” he says. “But if I cannot read it and others can, the material starts eliminating people from getting information.”

Despite its many benefits, Hoy says that document developers (those creating written materials) often don’t ask for reader feedback. She says one reason is that developers may assume they already know what their readers need and want. Another is a concern that getting reader feedback costs a lot of money or takes a lot of time. And a third reason is that developers may be unaware of how valuable reader feedback truly can be.

In a recent discussion, Hoy and Willard both offered some good suggestions for getting reader feedback that can help you become a better communicator.

Explore the Options for Getting Feedback

You can get reader feedback in any number of ways. Here are three.

• Focus groups. Also known as “group interviews,” focus groups are an excellent way to get general feedback from readers about a document’s design, content, and key points, according to Hoy. Focus groups usually have 6 to 10 members of similar age, culture, and familiarity with the subject.
• You might want to test your material with more than one focus group. This way, you can compare responses from different groups. For example, you could test a booklet about hospital services with a group of people who were hospitalized on an emergency basis and those who had planned admissions.
• Lab studies. You can learn how user-friendly your document or Web site is by testing it with readers in a controlled setting, or lab. This is an excellent way to compare two versions and find out which is better. Lab studies involve asking people to complete a specific task, such as finding their way around a Web site. As the tester, you observe which aspects of this task are easy and which are hard for people to do. You can then use your observations to make your material more user-friendly.
• Structured interviews. One-on-one interviews, Hoy says, are the best way to get detailed feedback about a document, particularly if the material is about sensitive subjects. You can ask an intended reader to “think aloud” as he or she looks at your document. This way, you can see how the reader uses your document. Listen for general comments and follow-up with specific questions such as, “What do you think about the pictures?” or “Why did you skip over (x) section?”

Plan Your Questions

Regardless of how you get feedback, make sure to plan your questions ahead of time. Focus on the areas you want to learn more about, such as key messages or use of pictures. Ask questions in a neutral way so as not to lead readers to particular answers. For example, a neutral question might be, “Tell me in your own words what this document is about,” rather than “Do you think this document does a good job explaining health benefits?”

Budget for Feedback

Admittedly, reader feedback takes time and costs money. While you do need to allocate money for these expenses, you do not need to let these costs skyrocket. Think creatively about ways to get feedback as part of your regular business. For example, instead of making appointments for structured interviews, you could ask patients for their opinions while they sit in your waiting room.

Jeanne McGee, author of Writing and Designing Print Materials for Beneficiaries: A Guide for State Medicaid Agencies, advises that, whatever your budget, you should allow for at least two rounds of reader feedback. The first is to identify initial problems. The second is to make sure you didn’t add new ones when you revised the document.

Let People Know Their Opinions Matter

Willard, who often assists in the evaluation of printed material, says that he is pleased to give feedback as long as his opinion makes a difference. He appreciates it when the person seeking feedback establishes an atmosphere of mutual respect with a smile and friendly tone of voice. Hoy recommends that, after getting reader feedback, you send thank-you notes letting readers know how their feedback will help others.

How Reader Feedback Helps.

Elizabeth Hoy, MHA, president of Polaris Consulting in Vienna, Virginia, shares some examples of how reader feedback can make a difference:

·        An insurance document used the term “evidence of coverage” as a synonym for insurance benefits. When tested, readers said they found this term threatening, as they thought “evidence” was a legal term meaning they had to prove something. Hearing this, the developers used different words.

·        A patient-education handout used the term “high blood pressure.” But testing showed that some readers understood this term as “high blood” as in too much blood. Consequently, “high blood pressure” was explained in a clearer way.

·        One company insisted on comparing eight different health plans within one tri-fold brochure (a 24” x 11” piece of paper folded into three sections). When a reader saw the many charts and tables squeezed onto a single piece of paper, he commented, “I don’t know where my eyes should go.” After hearing this, the company’s medical director not only changed the brochure’s format, she also declared, “I am never again going to put out any document without testing it first with readers.”

How to Find Out More:

• Elizabeth Hoy, MHA, is president of Polaris Consulting in Vienna, Virginia. She can be reached by phone at (703) 242-3888 or by email at elizabethhoy@cox.net. You can find her report about reader feedback at www.chcf.org/topics/medi-cal/index.cfm?itemID=104947&subTopic=CL272&subsection=medi-cal
• Krueger RA, Casey MA, 2000. Focus Groups: A Practical Guide for Applied Research, 3^rd edition. Thousand Oaks, CA: Sage Publications, Inc.
• McGee J, 1999. Writing and Designing Print Materials for Beneficiaries: A Guide for State Medicaid Agencies. HCFA Publication Number 10145. Baltimore, MD: Centers for Medicare and Medicaid Services, U.S. Department of Health and Human Services. A portion of this document is available on line at www.hsph.harvard.edu/healthliteracy/how_to/resource_create.html