Helen: Welcome to Health Literacy Out Loud. I’m Helen Osborne, president of Health Literacy Consulting, founder of Health Literacy Month and your host of Health Literacy Out Loud.

In these podcasts, you get to listen in on my conversations with some truly remarkable people. You will hear what health literacy is, why it matters and ways we all can help improve health understanding.

Today I’m talking with Cindy Brach, who is the lead for health literacy and cultural competence at the Agency for Healthcare Research and Quality, or AHRQ, which is part of the US Department of Health and Human Services.

Cindy helped create several very important health literacy tools and resources, including AHRQ’s Health Literacy Universal Precautions Toolkit and the Institute of Medicine’s discussion paper “10 Attributes of a Health Literate Healthcare Organization.”

Now Cindy is one of three authors of PEMAT, or Patient Education Materials Assessment Tool. Welcome, Cindy.

Cindy: It’s a pleasure to be speaking with you again, Helen.

Helen: I am so glad you keep finding new ways to advance our field of health literacy. Do tell. What is PEMAT?

Cindy: PEMAT, as the name implies, is a tool for people to be able to assess how understandable and actionable patient education materials are.

There are a lot of different places where one can find patient education materials. There’s a range in the quality, target audiences and how understandable they are. A lot of people are left to figure out, “Which materials should I give to my patients?”

Helen: Cindy, you talk about how the PEMAT looks at understanding and action of materials. You and I both know there’s a lot of ways that health literacy folks tend to look at materials. Just put this into context. What are those other ways to be looking at materials? Then we can talk about how PEMAT fits in with those.

Cindy: There’s one class of assessment tools that looks at readability, which is essentially a formulation of what grade level a material is at. That doesn’t really take into account all the things that we know will lead to better understanding, and moreover, better action.

There’s also a host of tools out there that help people write better materials. One of my favorites is on the CMS website about creating effective and clear written materials. These are terrific guides for people who are developing patient education materials.

PEMAT is an assessment tool. It could be used in conjunction with a readability formula, but it’s really trying to help you choose from among all of the patient education options out there.

Helen: That’s great. I know from way back in my hospital days that we weren’t always creating materials. Sometimes we had a whole bunch of different stuff there. Which one should I give out? That’s what you’re saying PEMAT is most useful for.

Cindy: Yes. This actually came out of a project where we were thinking about how to leverage electronic health records and their potential power to deliver better patient education materials either through clinicians getting the materials out of their electronic health records or patients getting them directly from patient portals.

We realized that we couldn’t urge vendors who were creating these EHRs to link to better materials unless we defined understandable and actionable materials.

Helen: That’s great. This is really setting those standards for what a good material is.

Cindy: The PEMAT could be used by medical librarians, practice managers and anybody who’s making those choices. While it comes with a user’s guide that teaches you how to apply the PEMAT, it doesn’t require specialized training.

A lot of the other assessment tools that have been developed have either been tested by people who have been trained to use it or have been found to be unreliable in terms of one person using the tool to assess a material getting a very different rating from another person who uses the tool on that same material.

Helen: You’re really filling in that huge gap. Tell us what’s in the PEMAT.

Cindy: There are two domains. They’re separately scored. One is about understandability. There are a number of topics where we look at the content, word choice and style, how numbers are used, the organization of the information, the layout and design, and very importantly, the use of visual aids.

There’s a second domain which to my knowledge no other tool tries to measure, and that’s this concept of actionability. Does this person know what to do after they read that material? We have a set of items that measure that as well.

Another innovation in the PEMAT is that it’s aiming not only at assessing print materials but also audio-visual materials. I’m sure you would agree that audio-visual materials are becoming an increasingly popular way to communicate health information.

Helen: I sure know through these podcasts that people are getting their information in so many different ways. Walk us through this a little bit. Tell us more about not just those broader categories, but what’s in a question and how would anybody be able to use it?

Let’s say a non-clinical person is choosing this, but it’s about a clinical situation. How would he or she go through doing this?

Cindy: The first thing is to read through the user’s guide because that gives you not only the definitions of terms but also examples of what we would indicate that we agree with and disagree with.

Helen: All the questions are either agree or disagree. It’s not a scale 1 to 5 or anything.

Cindy: Exactly. To make it even easier and more reliable, we moved to just saying agree, disagree or not applicable. There’s guidance for all of those.

For example, the very first item is that the material makes its purpose completely evident. You can agree or disagree. The examples would give you a title that would be completely evident, such as “What Will Happen During Your Mammogram.” That tells you what that material is about.

You would disagree if the title was something like “Mammograms and You,” which doesn’t really tell you what the purpose of that material is. We recommend that you read through the entire material before you start scoring.

To make it even easier to use the PEMAT, we have these auto scoring Excel files that are downloadable along with the PEMAT and its guide from the AHRQ website. You can go in and say you disagree or agree for any item and it will automatically calculate the PEMAT score at the end.

Helen: Wow! Not only can someone do it, but you almost can train yourself at the spot for how to do it. I’ve seen your user’s guide. It is filled with great examples.

Then you go through it and score it. What do the scores mean? Is there a threshold of what’s acceptable or a good choice, or does it just give you a number and you make your own choice?

Cindy: That is a terrific question. Right now there is not a threshold. What we assume is that people will use it as a comparative tool so that materials that score better on the PEMAT will be more understandable and actionable than materials with a lower score.

What you think the threshold is depends on who your target audience is. We didn’t say if you get a 70% or above that’s a good material and if it’s below that it’s a bad material. It depends who your audience is.

It may be that we’ll do some work in the future to try to establish some thresholds, but right now it’s really in the user’s hands.

Helen: There’s something I like about that a lot. The PEMAT itself seems really objective. It meets it or it doesn’t meet it. Here’s the score.

Not having a threshold allows that sense of subjectivity and opinion. Only you know who you’re writing for. You said this is how you make a choice. If you only have three materials and you have to choose one of them because your budget or resources don’t allow any other option, you can at least compare those.

Cindy: That’s exactly the situation we’re in. Unfortunately, a lot of materials out there aren’t as understandable or actionable as we’d like them to be, but we need to choose the best one from the crop.

I’m really glad you mentioned objectivity and subjectivity. One of the big goals in developing the PEMAT was that it would be what we call valid and reliable when we talk about testing instruments.

That means that different raters would rate the same material, they’d come up with similar scores. We went through four rounds of testing to keep refining the PEMAT to make sure that there was reliability.

We went one step further that I don’t know if any of these other assessment tools has done. We did some consumer testing where we looked at whether consumers found the materials more understandable and actionable if it scored higher.

Helen: They do, so we’re all benefiting from that great field research.

Cindy: We certainly are, but I don’t want to oversell this. In the very small group of consumers that we tested this, what we did find is that higher actionability scores were associated with a higher sense of knowing what to do as a result of reading the material, and understandability.

Consumers actually did better on our tests of what they understood and rated the materials more highly in terms of how much they liked them.

Helen: I’m so thankful that we have this guide to move us that much further. How do you make a choice and weigh different parts of it? Is it understandable? Can we do something about it?

When I think of documents and goals, I also think of that feeling part. Not only can I understand what this direction is telling me and do I know what to do, but will I do it? Does your guide look at that?

Cindy: No, it doesn’t. We did struggle with that one. We had a technical expert panel. We think about that as motivation. Does the material motivate you to do something? That’s not what the PEMAT measures.

It measures whether you know what to do and whether the steps are broken down in a way that you can take action, but it doesn’t measure whether or not the material actually makes you more likely to do those things.

What we do know is that if you don’t understand what your options are for action, you’re certainly not going to do them.

Helen: I agree with you. I think that’s the third tier of the materials. It’s really interesting. It sounds like the PEMAT covers two out of the three big components, do you know what to do and do you know what this is, but not the motivation part. You’re moving us along. That’s so much more than we had before.

Are there other things we need to know about what is in or not in the PEMAT? Should it be our only assessment tool or should we do something else?

Cindy: It’s absolutely not the only assessment tool. For example, one of the things that the PEMAT does not do is assess the accuracy. If you have a piece of patient education material, it could be very understandable and actionable but give you misinformation.

You want to use the PEMAT with other checks. Let’s say you’re a medical librarian. You might not have the clinical proficiency to say, “Yes, this is the best instruction for how to use an asthma inhaler,” but it gives you tools to differentiate among materials and to weed out those that are clearly not understandable and actionable.

Helen: That’s really important. In today’s world, we all have to be savvy about where information comes from. Is it right for me? I can’t imagine any tool that could do all of that, but I really am thrilled to see how far PEMAT helps us move along that process of choosing materials.

You said in the beginning anybody can do this. They don’t need special training. About how long does it take to fill out the PEMAT?

Cindy: It depends on the length of the material you’re rating. One of the things you have to do is read through the whole material.

Helen: What about not counting that part?

Cindy: Let me give you a sense in terms of the length of the PEMAT. There are only 19 questions for the understandability measure, and there are between four and seven questions for the actionability measure depending on whether you’re talking about audio-visual materials or print materials.

There aren’t that many items. That was another thing we were striving for, which was what we call a parsimonious set of measures. We wanted to only include those that really help us differentiate between understandable and not understandable, and actionable and not actionable.

Helen: There are maybe 25 things you have to check. Can somebody do this in 10 or 15 minutes, other than reading the document or watching the video?

Cindy: If it’s a longer document, in addition to reading it, you probably do want to go back through it as you’re filling out the PEMAT. If it’s a short document, absolutely you could do this in 10 or 15 minutes.

Helen: That’s consistent no matter who we are. None of us have much time. What’s the URL for the PEMAT?

Cindy: It’s www.AHRQ.gov/pemat.

Helen: Cindy, you are terrific. You are showering us with all these gifts of helping to move health literacy forward. Thank you once again for being a guest on Health Literacy Out Loud.

Cindy: Thank you for being one of the great disseminators out there and getting these tools into the hands of people who can use them.

Helen: I learned so much from Cindy Brach, and I hope that you did too, but health literacy isn’t always easy. For help clearly communicating your health message, please visit my Health Literacy Consulting website at www.HealthLiteracy.com. While you’re there, sign up for the free monthly enewsletter What’s New in Health Literacy Consulting.

NewHealth Literacy Out Loud podcasts come out every few weeks. Subscribe for free to hear them all. You can find us on iTunes, Stitcher Radio and the Health Literacy Out Loud website, www.HealthLiteracyOutLoud.org.

Did you like this podcast? Did you learn something new? If so, tell your colleagues and friends. Together, let’s let the whole world know why health literacy matters. Until next time, I’m Helen Osborne.