This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.
Providing people with understandable and actionable health information can considerably promote healthy behaviors and outcomes. To this end, some valid and reliable scales assessing the patient-friendliness of health education materials, like the PEMAT-P (Patient Education Materials Assessment Tool for printable materials), have been well developed in English-speaking countries. However, the English version of the PEMAT-P has not been translated and adapted into simplified Chinese and validated in mainland China.
This study sought to translate the PEMAT-P tool into a simplified Chinese (Mandarin) version (C-PEMAT-P, a Chinese version of the Patient Education Materials Assessment Tool for printable materials) and verify its validity and reliability for assessing the comprehensibility and actionability of health education resources written in simplified Chinese. As a result, the validated C-PEMAT-P could be used to guide health researchers and educators to design more comprehensible and actionable materials for more tailored and targeted health education and interventions.
We translated the PEMAT-P into simplified Chinese in the following three steps: (1) forward-translating the PEMAT-P into simplified Chinese, (2) back-translating the simplified Chinese version into English, and (3) testing translation equivalence linguistically and culturally by examining the original English version of the PEMAT-P and the back-translated English version of the tool. Any discrepancies between the original English tool and the back-translated English tool were resolved through a panel discussion among the research team of all authors to produce a revised forward-translated Chinese version (C-PEMAT-P). We then evaluated the clarity of construction and wording as well as the content relevance of the C-PEMAT-P using a 4-point ordinal scale to determine its content validity. After that, 2 native Chinese speakers (health educators) used the C-PEMAT-P to rate 15 health education handouts concerning air pollution and health to validate their reliability. We calculated the Cohen coefficient and Cronbach α to determine the interrater agreement and internal consistency of the C-PEMAT-P, respectively.
We finalized the translated Chinese tool after discussing the differences between the 2 English versions (original and back-translated) of the PEMAT-P, producing the final Chinese version of the PEMAT-P (C-PEMAT-P). The content validity index of the C-PEMAT-P version was 0.969, the Cohen coefficient for the interrater scoring agreement was 0.928, and the Cronbach α for internal consistency was .897. These values indicated the high validity and reliability of the C-PEMAT-P.
The C-PEMAT-P has been proven valid and reliable. It is the first Chinese scale for assessing the comprehensibility and actionability of Chinese health education materials. It can be used as an assessment tool to evaluate health education materials currently available and a guide to help health researchers and educators design more comprehensible and actionable materials for more tailored and targeted health education and interventions.
Developing accurate, comprehensible, and actionable health information is essential for delivering quality, safe health care [
Patient-friendly health education materials must be, first and foremost, readable. Readability means the ease by which readers are capable of reading and comprehending text [
Recognizing the weakness of readability formulas, researchers developed some checklists and instruments that can evaluate the health literacy demand of education materials [
Developed to address this shortcoming of the existing instruments, the PEMAT (Patient Education Materials Assessment Tool), which has been proven valid and reliable, was designed to enable laypeople and health professionals to evaluate the comprehensibility and actionability of printable and audiovisual health education materials [
The PEMAT-P assesses both comprehensibility and actionability. This scale comprises 24 items: 17 related to comprehensibility and 7 related to actionability. Comprehensibility is evaluated in terms of content, word choice and style, the use of numbers, organization, layout, and design, and the use of visual aids [
The PEMAT was widely applied to studies in English- and non–English-speaking countries (see
Although well-developed assessment tools are adopted for evaluating the suitability of health education materials in English-speaking countries, such instruments are hardly applied in Chinese-speaking communities [
This study sought to translate and adapt the PEMAT-P into a Mandarin Chinese version (C-PEMAT-P, a Chinese version of the Patient Education Materials Assessment Tool for printable materials) and verify its validity and reliability for evaluating whether Chinese health education materials are understandable and actionable. In this first stage of development, we intended to test the validity and reliability of the C-PEMAT-P among health educators, whose professional background in public health education and practical experience in engaging with patients could facilitate the development and adaptation of the C-PEMAT-P and ensure the quality of the tool we developed. In the next stage of development, we will conduct further research to test both the PEMAT and the Chinese version with a reasonably sized sample of end users on some target materials, to establish the equivalence of scores.
Previous studies adopted forward-translation, back-translation, bilingual testing, and monolingual testing in the scale translation process, which are essential for studies involving cross-cultural comparisons [
The whole process of developing the simplified Chinese version of DISCERN.
The PEMAT-P scoring sheet and user’s guide were translated from English into Chinese by a native Chinese speaker. An experienced bilingual translator was then requested to review and identify problematic words, phrases, or sentences in the translated Chinese version independently. Afterward, we discussed the reviewer’s comments and suggestions with the translator and revised the Chinese version. In this process, we paid close attention to cultural appropriateness, which means the correspondence of the core concepts in the materials with the logic, language, and experiences of the target culture and the use of positive cultural images and examples [
To minimize errors in back-translation, we selected a translator carefully [
Inspired by Sperber et al’s [
Based on cultural appropriateness [
Content validation sought to assess the item relevance in each domain and the clarity of the translated items [
Four health educators (ZD, ZX, DW, and XC) participated in the psychometric properties testing of the C-PEMAT-P. They all have a background in public health education. ZX and ZD were highly qualified health educators who have been working as professors and doctors at Qilu Hospital of Shandong University, China because they received their doctorates at Shandong University. DW and XC are currently studying for their master’s degree in public health education at Shandong University. Their professional educational background and experience in engaging with patients at Qilu Hospital of Shandong University can qualify them for the content validation and reliability testing of the newly developed tool.
Four Chinese health educators (ZD, ZX, DW, and XC) were requested to assess the content validity of the finalized Chinese version (C-PEMAT-P) by evaluating the clarity and relevance of the items in the C-PEMAT-P. We calculated the clarity of construction and wording as well as the content relevance using a 4-point ordinal scale of 1=item is not clear, 2=item needs major revisions to be clear, 3=item needs minor revisions to be clear, and 4=item is clear [
The interrater reliability was tested to determine the scoring consistency between raters. We invited 2 health educators (ZD and ZX) to assess the comprehensibility and actionability of 15 print health education materials on air pollution and health using the C-PEMAT-P. Cohen coefficient was calculated to evaluate the interrater agreement.
Four health educators (ZD, ZX, DW, and XC) were requested to verify the internal consistency of the C-PEMAT-P using a selected print material on air pollution and health. We calculated Cronbach α to assess the internal consistency of the C-PEMAT-P.
Print scoring sheets and rating sheets were used to collect data manually. Quantitative analyses were conducted using SPSS (version 22.0; IBM Corp) to determine the content validity index, Cohen coefficient for interrater reliability, and Cronbach α for internal consistency.
This study was approved by the ethics review board of Qilu Hospital of Shandong University, China (KYLL-202208-026).
According to Sperber et al’s [
Comparison of the original and back-translated English versions of the Patient Education Materials Assessment Tool for printable materials in terms of comparability of language and similarity of interpretability. The scores in bold indicate problematic items.
|
Original English version | Back-translated English version | Comparability of language | Similarity of interpretability |
1 | The material makes its purpose completely evident. | The article has an apparent purpose. | 2 | 1 |
2 | The material does not include information or content that distracts from its purpose. | The article does not contain irrelevant content. | 2 | 1 |
3 | The material uses common, everyday language. | The article uses everyday language. | 1 | 1 |
4 | Specialized terms are used only to familiarize audience with the terms. When used, specialized terms are defined. | Specialized terms are used only to familiarize the reader with the words. If specialized terms are used, the author also gives explanations and definitions. | 1 | 1 |
5 | The material uses the active voice. | This article uses the active voice. | 1 | 1 |
6 | Numbers appearing in the material are clear and easy to understand. | The numbers in the article are clear and easy to understand. | 1 | 1 |
7 | The material does not expect the user to perform calculations. | Readers do not need to do the calculations themselves. | 2 | 1 |
8 | The material breaks or “chunks” information into short sections. | This article breaks down large paragraphs of information into smaller paragraphs of content. |
|
2 |
9 | The material’s sections have informative headers. | The title of the article makes it easy to see what the article is about. |
|
2 |
10 | The material presents information in a logical sequence. | The content of the article is logical. |
|
1 |
11 | The material provides a summary. | The article has a summary. | 1 | 1 |
12 | The material uses visual cues (eg, arrows, boxes, bullets, bold, larger font, and highlighting) to draw attention to key points. | The article uses visual graphics (eg, arrows, etc.) to emphasize the main points of the article. | 2 | 2 |
13 | The material uses visual aids whenever they could make content more easily understood (eg, illustration of healthy portion size). | The article often uses diagrams (eg, a healthy diet nutrition chart) to help people understand the content more easily. | 2 | 1 |
14 | The material’s visual aids reinforce rather than distract from the content. | The diagrams emphasize the main content rather than distract the reader. | 1 | 1 |
15 | The material’s visual aids have clear titles or captions. | The diagrams have clear headings or notes. | 2 | 2 |
16 | The material uses illustrations and photographs that are clear and uncluttered. | The pictures and photos are clear and easy to recognize. | 2 | 1 |
17 | The material uses simple tables with short and clear row and column headings. | The diagrams are simple, with short and clear row and column headings. | 1 | 1 |
18 | The material clearly identifies at least one action the user can take. | The article clearly presents at least one action that the reader can take. | 2 | 1 |
19 | The material addresses the user directly when describing actions. | When the article recommends action, it clearly points out who the target audience is. |
|
1 |
20 | The material breaks down any action into manageable, explicit steps. | When recommending actions, the article further decomposes the actions into clear, actionable, concrete steps. |
|
2 |
21 | The material provides a tangible tool (eg, menu planners, checklists) whenever it could help the user take action. | The article provides simple, easy-to-plan, record-keeping tools (eg, recipe planners, checklists) for readers to take action. | 2 | 1 |
22 | The material provides simple instructions or examples of how to perform calculations. | The article gives simple examples of how to do the calculation. | 1 | 1 |
23 | The material explains how to use the charts, graphs, tables, or diagrams to take actions. | The article explains how to use various diagrams to take action. | 1 | 1 |
24 | The material uses visual aids whenever they could make it easier to act on the instructions. | The article uses diagrams as much as possible to make it easier for readers to follow the suggested actions. | 2 | 1 |
Nineteen of the 24 items were rated as
8. This article breaks down the whole passage into smaller sections.
9. The section headings of this article make it easy to see what the article is about.
10. The content of the article is presented logically.
19. The article directly tells the reader the recommended actions.
20. The article breaks down the recommended actions into clear, actionable clear steps.
Considering cultural appropriateness [
Four Chinese health educators evaluated the content validity of the C-PEMAT-P. For the whole C-PEMAT-P, the content validity index in clarity was 0.969, and the content validity index in relevance was also 0.969. Of the 24 items on the C-PEMAT-P scoring sheet, 3 items (4, 6, and 24) received a 0.75 rating for item clarity, and the remaining items received a 1.0 rating for item clarity. Three items (4, 6, and 24) received a 0.75 rating for content relevance, and the others received a 1.0 rating for content relevance. One of the 4 health educators gave items 4, 6, and 24 a score of 2, 1, and 2, respectively, for clarity. The same health educator gave a score of 2 to each of these 3 items for relevance. We did not further revise these items for 2 reasons: (1) the other 3 health educators all gave these 3 items a score of 4 for clarity and relevance, and (2) we concluded that these 3 items were not problematic after checking them against the original English version of the PEMAT-P.
Two raters assessed 15 printed health education materials on air pollution and health independently using the C-PEMAT-P. The Cohen coefficient for the interrater agreement was .944 (P=.0<.05). For the 15 materials rated, 19 (79%) of the 24 items in the C-PEMAT-P were given the same score by the 2 raters. Five (21%) of the 24 items (6, 7, 10, 12, and 21) received different scores in 8 materials out of those 15 materials assessed.
The Cronbach α was determined at .897 for the internal consistency of the overall Chinese scale. For comprehensibility, Cronbach α was determined at .847. For actionability, Cronbach α was determined at .751.
This study followed the cross-cultural translation model (comprising forward-translation, back-translation, and translation equivalence testing) that was proposed by Sperber et al [
Essential cultural aspects to be considered in the cross-cultural translation of health scales.
Cultural aspects | Gloss |
Cultural equivalence | Functional equivalence between the source and target scales that is achieved through well considering the types of the original text, the importance of cultural color in the original text, the purpose of translation, and the reader type of the target text. |
Cultural appropriateness | Correspondence of the core concepts in the materials with the logic, language, and experiences of the target culture and the use of positive cultural images and examples. |
Similarity of interpretability | The degree to which the source-language scale and the target-language scale would engender the same response even if the words are not the same. |
Item relevance | Relevance of the items of the translated scale to the corresponding subscales (if any) and the intended domain of the translated scale. |
Through the content validity testing of the C-PEMAT-P, we obtained a high content validity index of 0.969, which verified the validity of this newly developed instrument for evaluating the comprehensibility and actionability of Chinese health education materials on air pollution and health.
Cohen coefficient of .994 obtained for the interrater reliability implied a fair scoring agreement between the 2 raters. This interrater reliability was relatively higher than that in previous studies [
Various methods were used to translate assessment tools in previous studies [
Back-translation is a helpful approach to detecting translation errors [
A few previous studies that used the PEMAT to assess the comprehensibility and actionability of health education materials described interrater reliability. Studies using 2 or more raters usually calculated Cohen coefficient [
This study has some limitations. First, although we went through rigorous procedures to translate, adapt, and validate the C-PEMAT-P, probably, we could not eliminate all potential linguistic and cultural discrepancies. Second, we used the C-PEMAT-P to assess the comprehensibility and actionability of health education materials from the health educators’ perspective, but we failed to verify these 2 aspects from the patients’ perspective. Therefore, future studies need to invite non–health care professionals as informants who will use the C-PEMAT-P to assess the comprehensibility and actionability of health education materials. The third limitation is directly associated with the second limitation. Future research needs to be conducted to measure the actual behavioral change and the resulting health outcomes by following up with patient informants for a given period after the intervention. Fourth, we used the newly developed Chinese tool to assess health education materials only on air pollution and health. Further studies need to be conducted to evaluate other health education materials to verify the usability of the new tool. Finally, concurrent validity was not assessed due to the lack of validated Chinese-language scales similar to the PEMAT-P. However, we adopted the back-translation strategy, the translation equivalence testing, the content validity testing, and the reliability testing to ensure the validity and reliability of the C-PEMAT-P.
To address the unavailability of patient education materials assessment tools in Chinese-speaking communities, we translated and cross-culturally adapted the PEMAT-P into the C-PEMAT-P and verified its reliability and validity for evaluating the comprehensibility and actionability of health education materials written in Chinese. The C-PEMAT-P is the first validated Chinese-language scale for assessing the comprehensibility and actionability of Chinese health education materials. It has the potential to allow health care educators and practitioners to provide patients and the public with Chinese health education materials that are easy to understand and act on. It can also enable them to improve or develop patient-friendly health education materials. As a result, patients and the public cannot only increase their perceived self-efficacy but also promote their actual behavioral change to achieve desired health outcomes. In the future, it is imperative to verify the generalizability of the findings of this study and further validate the C-PEMAT-P by conducting many similar studies.
The Chinese version of the PEMAT-P (the original English version of the PEMAT-P can be consulted at [
comparability of language
a Chinese version of the Patient Education Materials Assessment Tool for printable materials
the Patient Education Materials Assessment Tool for printable materials
similarity of interpretability
Data are available upon reasonable request via victorsyhz@hotmail.com.
None declared.