Article from the Boston Globe’s On Call Magazine, December 2000

By Helen Osborne, M.Ed., OTR/L
President of Health Literacy Consulting

Originally designed to help classroom teachers choose textbooks for their students, most readability formulas give a score in terms of a grade level. But it isn’t enough just to know how far your readers went in school. Many adults actually read at a level 3 to 5 grades lower than their last completed year of school. That’s just one of the facts you need to know to be able to effectively assess the readability of your printed material.

Many people tell me that they write easy to read materials. When I ask how they know, they say that they check it with the readability formula that’s included with their word processing program. I cringe when I hear this. While being better than nothing, these formulas are less than a perfect tool to assess reading ease. And whether you use a computerized application or test the readability by hand, there are things you need to know and do in order to get a meaningful result. You need to carefully select the sample text. You need to correctly apply the assessment tool. And you need to understand how to accurately interpret the results.

There are over one hundred factors that affect how easy, or hard, a given document is to read and understand. These factors include sentence length, word choice, layout, tone, organization, use of illustrations, and appeal to the reader. Readability formulas often look at only two or three of these factors – most commonly, the number of words in a sentence and the number of syllables in a word.

Despite their limitations, though, readability assessments can be useful tools, says Audrey Riffenburgh, MA, a specialist in plain English and readability and president of Plain Language Works, LLC. The key, she says, is to take the necessary steps to accurately interpret the results. Here are some things to keep in mind when you are assessing the readability of written healthcare information.

Use the Proper Method to Get an Accurate Score

Selecting your sample. Different tests for readability use different methods for choosing samples. It is important to follow the prescribed method for selecting a sample in order to get an accurate score. There are, however, general guidelines you should follow in order to get the most accurate reading.

Riffenburgh recommends that when you choose text to assess, you should choose text that has at least 30 sentences, or 300 to 500 words. If you want to assess lengthy text, be sure you take samples from the beginning, middle, and end of the document. Select passages with connected, flowing text. Do not include the first and last sentences, however. They often are not characteristic of the rest of the text.

Assessing readability by hand.” There are several effective tools for assessing reading levels “manually.” Two of the most commonly used tools are the Fry and the SMOG assessments.

To use the Fry Formula, you count the number of syllables and sentences in three 100-word passages. You then average each set of figures and refer to a graph to find the grade level equivalent of the text. Although some people find this method tedious, Riffenburgh feels that it is a particularly useful tool for assessing lower reading-level materials.

For the SMOG, you count all the words with three or more syllables in three 10-sentence passages. Then you refer to a table to find the grade level equivalent. This tool includes a conversion chart to assess text that has less than 30 sentences.

The SMOG can be a useful tool to assess higher reading-level materials. But it is not as accurate as other methods for more basic written information. Regardless of which manual assessment tool you use, be sure to follow the directions exactly. This includes counting repetitive multi-syllabic words, such as “med-i-ca-tion” each time the word appears in the text.

Using computerized readability assessment tools. Most word processing programs include a feature that assesses readability. In addition, there are commercially available readability programs. Regardless of which computerized tool you use, Riffenburgh recommends that you clean up your text before you assess readability:

• Save a copy of the document with a different name to use for the analysis.
• Check that the software you are using is accurately counting the number of sentences and words.
• Delete words that are not full sentences from the sample being tested. This includes such items as headings and bulleted lists.
• Delete extraneous periods that do not mark the end of a sentence. Periods found in numbers like 98.6 and abbreviations such as U.S.A. will make the results of assessment inaccurate.

Don’t Over Interpret the Results

Grade level equivalent scores are, at best, only accurate by plus or minus 1.5 grade levels, says Riffenburgh. This means that when you revise a material so that the score drops from 7.3 to 6.8, it is not necessarily easier to read. Results may also vary depending on which formula you use. For example, the Fry formula often returns a score that is one to two grade levels lower than the SMOG. The computerized Flesch-Kincaid tool often returns a score two to three grades lower than other computerized formulas. “Results give us a ballpark figure,” says Riffenburgh. “They are not precisely accurate.”

While using one or two syllable words and short sentences will result in a lower readability score, the score does not necessarily mean that the document is easy to read. Sometimes the message gets lost when text is choppy and key points are overly simplified. To make written texts truly readable, apply all the principles of clear and simple writing. That includes:

• Focusing on information the reader “needs to know”
• Organizing information in ways that make sense to the reader
• Assembling materials so that they are inviting to read

Match the Reading Level to the Reader

Readability formulas only measure the relative difficulty of the text. By themselves, the formulas are not sufficient to determine whether there is a match between the document and the reader. The only way to truly find out if a document is easy to read is to ask. Work with intended readers as you develop new materials. Then test the materials to confirm that patients actually do understand the information as it is presented.

How To Learn More About Assessing Readability

Audrey Riffenburgh, MA, is president of Plain Language Works, LLC. She can be reached by e-mail at ar@plainlanguageworks.com, or by phone at (505) 345-1107.

Helen Osborne, OTR/L, MEd, is president of Health Literacy Consulting in Natick. She can be reached by e-mail at helen@healthliteracy.com, or by phone at (508) 653-1199.

Printed Sources About Readability

Chall, JS and Dale, E (1995). Readability Revisited: The New Dale-Chall Readability Formula. Brookline Books, Cambridge MA.

Doak, C, Doak, L, and Root, J (1996). Teaching Patients with Low Literacy Skills. J. B. Lippincott Company, Philadelphia. This book includes detailed directions for using the Fry readability formula.

Stewart, K (1996). “Written Patient -Education Materials: Are they on the level?” Nursing96, January, 32-33. This article includes directions for using the SMOG formula.

Sources for Computerized Testing of Readability

Readability Calculations. Micro Power & Light. 12820 Hillcrest Road, Dallas, Texas 75230. Includes the Dale-Chall, Reading Ease, Flesch Grade Level, FOG, and the Powers-Sumner-Kearl Formulas.

Rodriguez, R and Stieglitz, E. Readability Master 2000: A Software Program for Readability. Brookline Books. To order, call toll-free: 1-800-666-BOOK.