This paper is in the following e-collection/theme issue:

Research Instruments, Questionnaires, and Tools (586) eHealth Literacy / Digital Literacy (395) Development and Evaluation of Research Methods, Instruments and Tools (501)

Published on in Vol 25 (2023)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/39808, first published .
The Chinese Version of the Patient Education Materials Assessment Tool for Printable Materials: Translation, Adaptation, and Validation Study

The Chinese Version of the Patient Education Materials Assessment Tool for Printable Materials: Translation, Adaptation, and Validation Study

The Chinese Version of the Patient Education Materials Assessment Tool for Printable Materials: Translation, Adaptation, and Validation Study

Authors of this article:

Yi Shan1 Author Orcid Image ;   Meng Ji2 Author Orcid Image ;   Zhaogang Dong3 Author Orcid Image ;   Zhaoquan Xing4 Author Orcid Image ;   Ding Wang3 Author Orcid Image ;   Xiangting Cao3 Author Orcid Image
Article Authors Cited by (2) Tweetations (5) Metrics

    Original Paper

    1School of Foreign Studies, Nantong University, Nantong, China

    2School of Languages and Cultures, The University of Sydney, Sydney, Australia

    3Department of Clinical Laboratory, Qilu Hospital of Shandong University, Ji'nan, China

    4Department of Urology, Qilu Hospital of Shandong University, Ji'nan, China

    Corresponding Author:

    Yi Shan, PhD

    School of Foreign Studies, Nantong University

    No. 9, Seyuan Rd., Chongchuan District

    Nantong, 226019

    China

    Phone: 86 15558121896

    Email: victorsyhz@hotmail.com


    Background: Providing people with understandable and actionable health information can considerably promote healthy behaviors and outcomes. To this end, some valid and reliable scales assessing the patient-friendliness of health education materials, like the PEMAT-P (Patient Education Materials Assessment Tool for printable materials), have been well developed in English-speaking countries. However, the English version of the PEMAT-P has not been translated and adapted into simplified Chinese and validated in mainland China.

    Objective: This study sought to translate the PEMAT-P tool into a simplified Chinese (Mandarin) version (C-PEMAT-P, a Chinese version of the Patient Education Materials Assessment Tool for printable materials) and verify its validity and reliability for assessing the comprehensibility and actionability of health education resources written in simplified Chinese. As a result, the validated C-PEMAT-P could be used to guide health researchers and educators to design more comprehensible and actionable materials for more tailored and targeted health education and interventions.

    Methods: We translated the PEMAT-P into simplified Chinese in the following three steps: (1) forward-translating the PEMAT-P into simplified Chinese, (2) back-translating the simplified Chinese version into English, and (3) testing translation equivalence linguistically and culturally by examining the original English version of the PEMAT-P and the back-translated English version of the tool. Any discrepancies between the original English tool and the back-translated English tool were resolved through a panel discussion among the research team of all authors to produce a revised forward-translated Chinese version (C-PEMAT-P). We then evaluated the clarity of construction and wording as well as the content relevance of the C-PEMAT-P using a 4-point ordinal scale to determine its content validity. After that, 2 native Chinese speakers (health educators) used the C-PEMAT-P to rate 15 health education handouts concerning air pollution and health to validate their reliability. We calculated the Cohen coefficient and Cronbach α to determine the interrater agreement and internal consistency of the C-PEMAT-P, respectively.

    Results: We finalized the translated Chinese tool after discussing the differences between the 2 English versions (original and back-translated) of the PEMAT-P, producing the final Chinese version of the PEMAT-P (C-PEMAT-P). The content validity index of the C-PEMAT-P version was 0.969, the Cohen coefficient for the interrater scoring agreement was 0.928, and the Cronbach α for internal consistency was .897. These values indicated the high validity and reliability of the C-PEMAT-P.

    Conclusions: The C-PEMAT-P has been proven valid and reliable. It is the first Chinese scale for assessing the comprehensibility and actionability of Chinese health education materials. It can be used as an assessment tool to evaluate health education materials currently available and a guide to help health researchers and educators design more comprehensible and actionable materials for more tailored and targeted health education and interventions.

    J Med Internet Res 2023;25:e39808

    doi:10.2196/39808

    Keywords

    actionability; adaptation; Chinese version of the PEMAT-P; comprehensibility; health education materials translation; validation 


    Background

    Developing accurate, comprehensible, and actionable health information is essential for delivering quality, safe health care [1]. Such information can help patients know their conditions, communicate with health providers and make decisions on medical actions. However, health education materials remain too complex to comprehend for many people, although policy makers and health care providers pay growing attention to them [2]. These materials are usually poorly comprehended by patients, particularly those who have inadequate health literacy [3,4] to “obtain, process, and understand basic health information and services needed to make appropriate health decisions” [5]. Many adults lack the skills necessary to engage in their health care successfully, making health literacy a well-recognized challenge for public health [2]. Inadequate health literacy is closely related to limited disease control, medical adherence, and health outcomes [3,6]. Regardless of health literacy levels, health education materials can be effective only when the intended readers are capable of reading, understanding, and accepting the provided information [7]. Patient-friendly health education materials are, therefore, essential for patients to take medical actions and improve health outcomes.

    Patient-friendly health education materials must be, first and foremost, readable. Readability means the ease by which readers are capable of reading and comprehending text [8]. This concept has long been integrated into the development and assessment of educational materials [9]. Readability formulas widely adopted to assess the reading difficulty of education materials overlook other factors influencing people’s ability to understand the information provided [10]. Education materials developers need to go beyond the only standard of readability levels [11] when determining the appropriateness of patient education materials for different readers [12,13].

    Recognizing the weakness of readability formulas, researchers developed some checklists and instruments that can evaluate the health literacy demand of education materials [13-17]. However, none of these tools measure the actionability of health education materials [2], which is an essential characteristic of education materials required in the National Action Plan [1].

    Developed to address this shortcoming of the existing instruments, the PEMAT (Patient Education Materials Assessment Tool), which has been proven valid and reliable, was designed to enable laypeople and health professionals to evaluate the comprehensibility and actionability of printable and audiovisual health education materials [2,9]. “Patient education materials are understandable when consumers of diverse backgrounds and varying levels of health literacy can process and explain key messages” [2]. “Patient education materials are actionable when consumers of diverse backgrounds and varying levels of health literacy can identify what they can do based on the information presented” [2]. Accordingly, the PEMAT has 2 versions: the PEMAT-P for printable materials (brochures and PDFs) and the PEMAT-A/V for audiovisual materials (videos and multimedia materials, including smartphone apps). Both scales comprise a scoring sheet and a user’s guide.

    The PEMAT-P assesses both comprehensibility and actionability. This scale comprises 24 items: 17 related to comprehensibility and 7 related to actionability. Comprehensibility is evaluated in terms of content, word choice and style, the use of numbers, organization, layout, and design, and the use of visual aids [2]. Actionability evaluates whether the material is actionable, that is, whether the material can effectively guide patients to take due medical actions. Each of the 24 items is rated in light of the PEMAT-P evaluation criteria in which 1 point is set for “Agree,” 0 points is set for “Disagree,” and N/A is set for items that do not apply to the material. To score a material, a rater needs to (1) calculate the total scores for the material on the understandability items only or the actionability items only, (2) divide that total by the number of items on which the material has been rated, excluding the items scored “Not Applicable,” and (3) multiply the result by 100 [2]. In this way, 2 scores can be provided for the comprehensibility and actionability of each material. A high score means a high degree of comprehensibility or actionability of the material assessed. The cutoff value for understandability and actionability is set at 70% by the developers.

    The PEMAT was widely applied to studies in English- and non–English-speaking countries (see Multimedia Appendix 1), enabling health care practitioners to select or develop suitable education materials for readers with varying health literacy [18-25]. Some studies reported the interrater reliability of one or both domains [18-21], the interrater reliability of the items classified by topic [22], or the interrater reliability at the item level of the PEMAT-P [23]. Some studies used the PEMAT to identify problems in patient education materials, for example, by evaluating web-based education materials for patients who took nonvitamin K oral anticoagulants [24]. Some studies used the PEMAT to develop or improve patient education materials, for example, to develop “an integrated diabetes-periodontitis nutrition and health education module” [25].

    Although well-developed assessment tools are adopted for evaluating the suitability of health education materials in English-speaking countries, such instruments are hardly applied in Chinese-speaking communities [7]. Without a ready-made assessment tool, translating existing scales into different languages is a rapid and practical approach [7]. The original English version of the PEMAT-P has been translated into Malay [26], Korean [27], and Japanese [28]. However, a simplified Chinese (Mandarin) version has not been developed through translation to assess the comprehensibility and actionability of Chinese health education materials.

    Objective

    This study sought to translate and adapt the PEMAT-P into a Mandarin Chinese version (C-PEMAT-P, a Chinese version of the Patient Education Materials Assessment Tool for printable materials) and verify its validity and reliability for evaluating whether Chinese health education materials are understandable and actionable. In this first stage of development, we intended to test the validity and reliability of the C-PEMAT-P among health educators, whose professional background in public health education and practical experience in engaging with patients could facilitate the development and adaptation of the C-PEMAT-P and ensure the quality of the tool we developed. In the next stage of development, we will conduct further research to test both the PEMAT and the Chinese version with a reasonably sized sample of end users on some target materials, to establish the equivalence of scores.


    Translation of the PEMAT-P

    Previous studies adopted forward-translation, back-translation, bilingual testing, and monolingual testing in the scale translation process, which are essential for studies involving cross-cultural comparisons [29]. Informed by these techniques, we translated the PEMAT-P into Mandarin Chinese to ensure semantic, pragmatic, and cultural equivalence following the rigorous procedures used in the model proposed by Sperber et al [30], as shown in Figure 1.

    ‎
    Figure 1. The whole process of developing the simplified Chinese version of DISCERN.
    Forward-Translation

    The PEMAT-P scoring sheet and user’s guide were translated from English into Chinese by a native Chinese speaker. An experienced bilingual translator was then requested to review and identify problematic words, phrases, or sentences in the translated Chinese version independently. Afterward, we discussed the reviewer’s comments and suggestions with the translator and revised the Chinese version. In this process, we paid close attention to cultural appropriateness, which means the correspondence of the core concepts in the materials with the logic, language, and experiences of the target culture and the use of positive cultural images and examples [12].

    Back-Translation

    To minimize errors in back-translation, we selected a translator carefully [31]. An experienced bilingual translator translated the revised Chinese version into English. This back-translator was blinded to the original English tool and different from the forward translator.

    Translation Equivalence Testing

    Inspired by Sperber et al’s [30] translation equivalence testing approach to validating translated instruments, we validated the revised initially translated Chinese tool by comparing the 2 English versions (original- and back-translated) in the comparability of language (CL) and the similarity of interpretability (SI). “Comparability of language refers to the formal similarity of words, phrases, and sentences. The similarity of interpretability refers to the degree to which the two versions would engender the same response even if the words are not the same” [30]. A native English speaker, currently teaching at a university in China, compared the 2 English versions in CL and SI to identify problematic items. He was asked to rank CL and SI, using an ordinal scale of 1=extremely comparable, 2=comparable, 3=not comparable, and 4=not at all comparable for CL, and an ordinal scale of 1=extremely similar, 2=similar, 3=not similar, and 4=not at all similar for SI [7], respectively. A panel comprising 2 bilingual health educators familiar with the PEMAT-P and the authors discussed and corrected problematic items in the Chinese version rated as 3 or 4. All problematic items that had been corrected were then retranslated until they were comparable and interpreted in the same way in both languages.

    Final Chinese Version

    Based on cultural appropriateness [12], we then consulted an experienced bilingual translator to finalize the Chinese version.

    Psychometric Properties Testing

    Content validation sought to assess the item relevance in each domain and the clarity of the translated items [32]. During the validation, a panel of experts gave constructive feedback on the quality of the newly developed scale and objective criteria for assessing each item involved [33].

    Four health educators (ZD, ZX, DW, and XC) participated in the psychometric properties testing of the C-PEMAT-P. They all have a background in public health education. ZX and ZD were highly qualified health educators who have been working as professors and doctors at Qilu Hospital of Shandong University, China because they received their doctorates at Shandong University. DW and XC are currently studying for their master’s degree in public health education at Shandong University. Their professional educational background and experience in engaging with patients at Qilu Hospital of Shandong University can qualify them for the content validation and reliability testing of the newly developed tool.

    Content Validity

    Four Chinese health educators (ZD, ZX, DW, and XC) were requested to assess the content validity of the finalized Chinese version (C-PEMAT-P) by evaluating the clarity and relevance of the items in the C-PEMAT-P. We calculated the clarity of construction and wording as well as the content relevance using a 4-point ordinal scale of 1=item is not clear, 2=item needs major revisions to be clear, 3=item needs minor revisions to be clear, and 4=item is clear [7] for clarity, and a 4-point ordinal scale of 1=item is not relevant, 2=item needs major revisions to be relevant, 3=item needs minor revisions to be relevant, and 4=item is relevant for relevance. We summed the items rated as 3 or 4 by the 4 health educators and divided the sum by the total number of items rated to measure the content validity index [33].

    Reliability Testing

    Interrater Reliability

    The interrater reliability was tested to determine the scoring consistency between raters. We invited 2 health educators (ZD and ZX) to assess the comprehensibility and actionability of 15 print health education materials on air pollution and health using the C-PEMAT-P. Cohen coefficient was calculated to evaluate the interrater agreement.

    Internal Consistency

    Four health educators (ZD, ZX, DW, and XC) were requested to verify the internal consistency of the C-PEMAT-P using a selected print material on air pollution and health. We calculated Cronbach α to assess the internal consistency of the C-PEMAT-P.

    Data Collection and Analysis

    Print scoring sheets and rating sheets were used to collect data manually. Quantitative analyses were conducted using SPSS (version 22.0; IBM Corp) to determine the content validity index, Cohen coefficient for interrater reliability, and Cronbach α for internal consistency.

    Ethical Considerations

    This study was approved by the ethics review board of Qilu Hospital of Shandong University, China (KYLL-202208-026).


    Comparison of the Two English Versions

    According to Sperber et al’s [30] translation validation method, the native English speaker compared the 2 English versions (original and back-translated) in respect of CL and SI. Table 1 shows the results of the comparison.

    Table 1. Comparison of the original and back-translated English versions of the Patient Education Materials Assessment Tool for printable materials in terms of comparability of language and similarity of interpretability. The scores in bold indicate problematic items.

    		Original English versionBack-translated English versionComparability of languageSimilarity of interpretability
    1The material makes its purpose completely evident.The article has an apparent purpose.21
    2The material does not include information or content that distracts from its purpose.The article does not contain irrelevant content.21
    3The material uses common, everyday language.The article uses everyday language.11
    4Specialized terms are used only to familiarize audience with the terms. When used, specialized terms are defined.Specialized terms are used only to familiarize the reader with the words. If specialized terms are used, the author also gives explanations and definitions.11
    5The material uses the active voice.This article uses the active voice.11
    6Numbers appearing in the material are clear and easy to understand.The numbers in the article are clear and easy to understand.11
    7The material does not expect the user to perform calculations.Readers do not need to do the calculations themselves.21
    8The material breaks or “chunks” information into short sections.This article breaks down large paragraphs of information into smaller paragraphs of content.32
    9The material’s sections have informative headers.The title of the article makes it easy to see what the article is about.42
    10The material presents information in a logical sequence.The content of the article is logical.31
    11The material provides a summary.The article has a summary.11
    12The material uses visual cues (eg, arrows, boxes, bullets, bold, larger font, and highlighting) to draw attention to key points.The article uses visual graphics (eg, arrows, etc.) to emphasize the main points of the article.22
    13The material uses visual aids whenever they could make content more easily understood (eg, illustration of healthy portion size).The article often uses diagrams (eg, a healthy diet nutrition chart) to help people understand the content more easily.21
    14The material’s visual aids reinforce rather than distract from the content.The diagrams emphasize the main content rather than distract the reader.11
    15The material’s visual aids have clear titles or captions.The diagrams have clear headings or notes.22
    16The material uses illustrations and photographs that are clear and uncluttered.The pictures and photos are clear and easy to recognize.21
    17The material uses simple tables with short and clear row and column headings.The diagrams are simple, with short and clear row and column headings.11
    18The material clearly identifies at least one action the user can take.The article clearly presents at least one action that the reader can take.21
    19The material addresses the user directly when describing actions.When the article recommends action, it clearly points out who the target audience is.31
    20The material breaks down any action into manageable, explicit steps.When recommending actions, the article further decomposes the actions into clear, actionable, concrete steps.32
    21The material provides a tangible tool (eg, menu planners, checklists) whenever it could help the user take action.The article provides simple, easy-to-plan, record-keeping tools (eg, recipe planners, checklists) for readers to take action.21
    22The material provides simple instructions or examples of how to perform calculations.The article gives simple examples of how to do the calculation.11
    23The material explains how to use the charts, graphs, tables, or diagrams to take actions.The article explains how to use various diagrams to take action.11
    24The material uses visual aids whenever they could make it easier to act on the instructions.The article uses diagrams as much as possible to make it easier for readers to follow the suggested actions.21

    Nineteen of the 24 items were rated as extremely similar, and the remaining 5 were rated as similar. Therefore, all 24 items achieved an acceptable degree of SI and did not need to be corrected in this aspect. Concerning CL, 19 of the 24 items were rated either as extremely comparable or as comparable, but 4 (items 8, 10, 19, and 20) were rated as not comparable, and 1 (item 9) was rated as not at all comparable. To make these 5 items in the back-translated version more comparable with the corresponding items in the original version, we revised these items in the Chinese version and back-translated them into English as follows:

    8. This article breaks down the whole passage into smaller sections.
    9. The section headings of this article make it easy to see what the article is about.
    10. The content of the article is presented logically.
    19. The article directly tells the reader the recommended actions.
    20. The article breaks down the recommended actions into clear, actionable clear steps.

    Considering cultural appropriateness [12], we avoided the literal translation of “chunks,” “information,” “header,” “address,” and “informative” in the revision of the Chinese version. We asked the same native English speaker to rate the 5 back-translated English items above. He rated all of them as comparable. As a result, all 24 items passed the translation equivalence testing.

    Psychometric Properties Testing

    Content Validity

    Four Chinese health educators evaluated the content validity of the C-PEMAT-P. For the whole C-PEMAT-P, the content validity index in clarity was 0.969, and the content validity index in relevance was also 0.969. Of the 24 items on the C-PEMAT-P scoring sheet, 3 items (4, 6, and 24) received a 0.75 rating for item clarity, and the remaining items received a 1.0 rating for item clarity. Three items (4, 6, and 24) received a 0.75 rating for content relevance, and the others received a 1.0 rating for content relevance. One of the 4 health educators gave items 4, 6, and 24 a score of 2, 1, and 2, respectively, for clarity. The same health educator gave a score of 2 to each of these 3 items for relevance. We did not further revise these items for 2 reasons: (1) the other 3 health educators all gave these 3 items a score of 4 for clarity and relevance, and (2) we concluded that these 3 items were not problematic after checking them against the original English version of the PEMAT-P.

    Reliability Testing
    Interrater Reliability

    Two raters assessed 15 printed health education materials on air pollution and health independently using the C-PEMAT-P. The Cohen coefficient for the interrater agreement was .944 (P=.0<.05). For the 15 materials rated, 19 (79%) of the 24 items in the C-PEMAT-P were given the same score by the 2 raters. Five (21%) of the 24 items (6, 7, 10, 12, and 21) received different scores in 8 materials out of those 15 materials assessed.

    Internal Consistency

    The Cronbach α was determined at .897 for the internal consistency of the overall Chinese scale. For comprehensibility, Cronbach α was determined at .847. For actionability, Cronbach α was determined at .751.


    Principal Findings

    Translation

    This study followed the cross-cultural translation model (comprising forward-translation, back-translation, and translation equivalence testing) that was proposed by Sperber et al [30] for health instrument translation. We found that the model was effective in the translation of the PEMAT-P into the C-PEMAT-P. This finding confirms the findings reported by some recent studies [34-37], which also attested to the effectiveness of this model in the cross-lingual, cross-cultural translation of health-related instruments and materials. We adopted back-translation to verify the translation equivalence of the C-PEMAT-P by requesting a native English speaker to compare the back-translated English version and the original English version of the PEMAT-P in terms of CL and SI, therefore, minimizing translation errors. The errors we identified and corrected in the translation stage confirmed the translation errors proposed by Capitulo et al [38], which were elaborated in the following subsection of Comparison with Previous Studies. Most of these errors stemmed from some cultural aspects that may not be captured through linguistic translation alone. Therefore, we propose some essential cultural aspects that need to be considered in the cross-cultural translation of health scales, as listed and glossed in Table 2.

    Table 2. Essential cultural aspects to be considered in the cross-cultural translation of health scales.
    Cultural aspectsGloss
    Cultural equivalenceFunctional equivalence between the source and target scales that is achieved through well considering the types of the original text, the importance of cultural color in the original text, the purpose of translation, and the reader type of the target text.
    Cultural appropriatenessCorrespondence of the core concepts in the materials with the logic, language, and experiences of the target culture and the use of positive cultural images and examples.
    Similarity of interpretabilityThe degree to which the source-language scale and the target-language scale would engender the same response even if the words are not the same.
    Item relevanceRelevance of the items of the translated scale to the corresponding subscales (if any) and the intended domain of the translated scale.
    Validity

    Through the content validity testing of the C-PEMAT-P, we obtained a high content validity index of 0.969, which verified the validity of this newly developed instrument for evaluating the comprehensibility and actionability of Chinese health education materials on air pollution and health.

    Reliability

    Cohen coefficient of .994 obtained for the interrater reliability implied a fair scoring agreement between the 2 raters. This interrater reliability was relatively higher than that in previous studies [9,18-23,28]. Analyzing the rating scores for the 24 items provided by the 2 raters, we found that high interrater agreement was associated with less subjective rating criteria. The reliability test of the C-PEMAT-P identified inconsistent scoring between the 2 raters on items 6, 7, 10, 12, and 21. These findings revealed that we could refine and reassess the subjective scoring criteria for the C-PEMAT-P that generated relatively lower interrater agreement [7].

    Comparison With Previous Studies

    Various methods were used to translate assessment tools in previous studies [29]. Regardless of the methods adopted, finding qualified translators is the first step in the translation process [7]. It is usually difficult to find competent bilingual translators familiar with the semantic content of the scale [31]. Translators are not always adequately knowledgeable in the subject area of a scale, and some even translate the original text literally without paying due attention to cultural nuances [7]. In this respect, colloquial phrases, slang, and jargon terms challenge translators the most [39]. In this study, the translator of the Chinese instrument is a competent bilingual of Chinese and English who has engaged in medical informatics and translation in this field for several years. Taking cultural nuances and appropriateness into account, she translated the original English version of the PEMAT-P into Chinese, achieving a high degree of semantic, pragmatic, and cultural equivalence.

    Back-translation is a helpful approach to detecting translation errors [31,40]. These errors included (1) the addition of words or phrases to the original, (2) the deletion of words or phrases from the original, (3) the alteration of the original meaning by changing words or phrases, and (4) the use of poor grammar and syntax impacting meaning and clarity negatively [39]. In the PEMAT-P, some technical terms, for example, “active voice,” “chunks,” “specialized terms,” “visual aids,” etc, were challenging to understand when translated literally into Chinese. Instead, the translator conveyed their meanings using culturally equivalent words, phrases, or sentences, which, however, possibly risked distorting the original intention of the source text [39]. Informed by Yu et al [40], we adopted back-translation, which allowed for comparing the 2 English PEMAT-P versions (original- and back-translated), to minimize the risk of translation distortion. As a result, we identified all potential problematic items, ensuring a finalized Chinese version of optimal translation equivalence, the C-PEMAT-P.

    A few previous studies that used the PEMAT to assess the comprehensibility and actionability of health education materials described interrater reliability. Studies using 2 or more raters usually calculated Cohen coefficient [9,18-23,28]. Cohen coefficients of <0.2, 0.21-0.4, 0.41-0.6, 0.61-0.8, and more than 0.8 are regarded as poor, fair, moderate, strong, and nearly complete agreement, respectively [41]. Because experience in assessing print health education materials can affect interrater reliability [42], we invited health educator raters experienced in assessing print health education materials, contributing to relatively higher interrater reliability than previous studies [9,18-23,28]. Considering that “there is latitude allowed in the interpretation of the criteria” that may cause subjectivity in rating materials [43], we propose that training programs should be carried out to increase the raters’ experience in evaluating health education materials to raise the interrater reliability for the C-PEMAT-P.

    Limitations

    This study has some limitations. First, although we went through rigorous procedures to translate, adapt, and validate the C-PEMAT-P, probably, we could not eliminate all potential linguistic and cultural discrepancies. Second, we used the C-PEMAT-P to assess the comprehensibility and actionability of health education materials from the health educators’ perspective, but we failed to verify these 2 aspects from the patients’ perspective. Therefore, future studies need to invite non–health care professionals as informants who will use the C-PEMAT-P to assess the comprehensibility and actionability of health education materials. The third limitation is directly associated with the second limitation. Future research needs to be conducted to measure the actual behavioral change and the resulting health outcomes by following up with patient informants for a given period after the intervention. Fourth, we used the newly developed Chinese tool to assess health education materials only on air pollution and health. Further studies need to be conducted to evaluate other health education materials to verify the usability of the new tool. Finally, concurrent validity was not assessed due to the lack of validated Chinese-language scales similar to the PEMAT-P. However, we adopted the back-translation strategy, the translation equivalence testing, the content validity testing, and the reliability testing to ensure the validity and reliability of the C-PEMAT-P.

    Conclusions

    To address the unavailability of patient education materials assessment tools in Chinese-speaking communities, we translated and cross-culturally adapted the PEMAT-P into the C-PEMAT-P and verified its reliability and validity for evaluating the comprehensibility and actionability of health education materials written in Chinese. The C-PEMAT-P is the first validated Chinese-language scale for assessing the comprehensibility and actionability of Chinese health education materials. It has the potential to allow health care educators and practitioners to provide patients and the public with Chinese health education materials that are easy to understand and act on. It can also enable them to improve or develop patient-friendly health education materials. As a result, patients and the public cannot only increase their perceived self-efficacy but also promote their actual behavioral change to achieve desired health outcomes. In the future, it is imperative to verify the generalizability of the findings of this study and further validate the C-PEMAT-P by conducting many similar studies.

    Data Availability

    Data are available upon reasonable request via victorsyhz@hotmail.com.

    Conflicts of Interest

    None declared.

    ‎
    Multimedia Appendix 1

    The Chinese version of the PEMAT-P (the original English version of the PEMAT-P can be consulted at [44]).

    XLSX File (Microsoft Excel File), 12 KB
    1. National action plan to improve health literacy. U.S. Department of Health and Human Services, Office of Disease Prevention and Health Promotion. 2010.   URL: http://www.health.gov/communication/hlactionplan/pdf/Health_Literacy_Action_Plan.pdf [accessed 2022-05-17]
    2. Shoemaker SJ, Wolf MS, Brach C. Development of the patient education materials assessment tool (PEMAT): a new measure of understandability and actionability for print and audiovisual patient information. Patient Educ Couns 2014;96(3):395-403 [FREE Full text] [CrossRef] [Medline]
    3. Berkman ND, Sheridan SL, Donahue KE, Halpern DJ, Crotty K. Low health literacy and health outcomes: an updated systematic review. Ann Intern Med 2011;155(2):97-107. [CrossRef] [Medline]
    4. Keller DL, Wright J, Pace HA. Impact of health literacy on health outcomes in ambulatory care patients: a systematic review. Ann Pharmacother 2008;42(9):1272-1281. [CrossRef] [Medline]
    5. Healthy people 2010. U.S. Department of Health and Human Services.   URL: https://www.cdc.gov/nchs/data/hpdata2010/hp2010_final_review.pdf [accessed 2023-04-08]
    6. Dewalt DA, Berkman ND, Sheridan S, Lohr KN, Pignone MP. Literacy and health outcomes: a systematic review of the literature. J Gen Intern Med 2004;19(12):1228-1239 [FREE Full text] [CrossRef] [Medline]
    7. Chang MC, Chen YC, Gau BS, Tzeng YF. Translation and validation of an instrument for measuring the suitability of health educational materials in Taiwan: suitability assessment of materials. J Nurs Res 2014;22(1):61-68. [CrossRef] [Medline]
    8. DuBay WH. The Principles of Readability. Costa Mesa, CA: Impact Information; 2004.
    9. Vishnevetsky J, Walters CB, Tan KS. Interrater reliability of the patient education materials assessment tool (PEMAT). Patient Educ Couns 2018;101(3):490-496 [FREE Full text] [CrossRef] [Medline]
    10. Toolkit for making written material clear and effective. Centers for Medicare & Medicaid Services. 2012.   URL: https:/​/www.​cms.gov/​Outreach-and-Education/​Outreach/​WrittenMaterialsToolkit/​index.html?redirect=/​WrittenMaterialsToolkit [accessed 2022-05-17]
    11. Brega AG, Freedman MA, LeBlanc WG, Barnard J, Mabachi NM, Cifuentes M, et al. Using the health literacy universal precautions toolkit to improve the quality of patient materials. J Health Commun 2015;20(suppl 2):69-76 [FREE Full text] [CrossRef] [Medline]
    12. Doak CC, Doak LG, Root JH. Teaching patients with low literacy skills. Am J Nurs 1996;96(12):16M. [CrossRef]
    13. Kaphingst KA, Kreuter MW, Casey C, Leme L, Thompson T, Cheng MR, et al. Health literacy INDEX: development, reliability, and validity of a new tool for evaluating the health literacy demands of health information materials. J Health Commun 2012;17(suppl 3):203-221. [CrossRef] [Medline]
    14. Helitzer D, Hollis C, Cotner J, Oestreicher N. Health literacy demands of written health information materials: an assessment of cervical cancer prevention materials. Cancer Control 2009;16(1):70-78 [FREE Full text] [CrossRef] [Medline]
    15. Clayton LH. TEMPtEd: development and psychometric properties of a tool to evaluate material used in patient education. J Adv Nurs 2009;65(10):2229-2238. [CrossRef] [Medline]
    16. Charnock D, Shepperd S, Needham G, Gann R. DISCERN: an instrument for judging the quality of written consumer health information on treatment choices. J Epidemiol Community Health 1999;53(2):105-111 [FREE Full text] [CrossRef] [Medline]
    17. CDC clear communication index. Centers for Disease Control and Prevention. 2013.   URL: https://www.cdc.gov/ccindex/widget.html [accessed 2022-05-17]
    18. Balakrishnan V, Chandy Z, Hseih A, Bui TL, Verma SP. Readability and understandability of online vocal cord paralysis materials. Otolaryngol Head Neck Surg 2016;154(3):460-464. [CrossRef] [Medline]
    19. Balakrishnan V, Chandy Z, Verma SP. Are online Zenker's diverticulum materials readable and understandable? Otolaryngol Head Neck Surg 2016;155(5):758-763. [CrossRef] [Medline]
    20. Zellmer C, Zimdars P, Parker S, Safdar N. How well do patient education materials for clostridium difficile infection score? A systematic evaluation. Int J Infect Control 2015;11(2). [CrossRef]
    21. Zellmer C, Zimdars P, Parker S, Safdar N. Evaluating the usefulness of patient education materials on surgical site infection: a systematic assessment. Am J Infect Control 2015;43(2):167-168. [CrossRef] [Medline]
    22. Morony S, McCaffery KJ, Kirkendall S, Jansen J, Webster AC. Health literacy demand of printed lifestyle patient information materials aimed at people with chronic kidney disease: are materials easy to understand and act on and do they use meaningful visual aids? J Health Commun 2017;22(2):163-170. [CrossRef] [Medline]
    23. Sarzynski E, Hashmi H, Subramanian J, Fitzpatrick L, Polverento M, Simmons M, et al. Opportunities to improve clinical summaries for patients at hospital discharge. BMJ Qual Saf 2017;26(5):372-380. [CrossRef] [Medline]
    24. Yiu A, Ng KK, Lee VW, Bajorek BV. Evaluating the understandability and actionability of web-based education materials for patients taking non-vitamin K oral anticoagulants. Ther Innov Regul Sci 2020;54(2):476-483. [CrossRef] [Medline]
    25. Jamil NA, Chau SH, Abdul Razak NI, Shamsul Kamar II, Mohd-Said S, Rani H, et al. Development and evaluation of an integrated diabetes-periodontitis nutrition and health education module. BMC Med Educ 2021;21(1):278 [FREE Full text] [CrossRef] [Medline]
    26. Wong ST, Saddki N, Arifin WN. Reliability of the Bahasa Malaysia version of patient education materials assessment tool. Malaysian J Public Health Med 2018;18(suppl 2):63-64.
    27. Lee H, Kim J, Yoo R, Lee JY. Development and evaluation of cardiovascular disease prevention education materials for middle-aged Korean-Chinese female workers: applying patient education materials assessment tool for printable materials (PEMAT-P). J Korean Acad Community Health Nurs 2016;27(3):284-298. [CrossRef]
    28. Furukawa E, Okuhara T, Okada H, Shirabe R, Yokota R, Iye R, et al. Translation, cross-cultural adaptation, and validation of the Japanese version of the patient education materials assessment tool(PEMAT). Int J Environ Res Public Health 2022;19(23):15763 [FREE Full text] [CrossRef] [Medline]
    29. Maneesriwongul W, Dixon JK. Instrument translation process: a methods review. J Adv Nurs 2004;48(2):175-186. [CrossRef] [Medline]
    30. Sperber AD, Devellis RF, Boehlecke B. Cross-cultural translation: methodology and validation. J Cross Cult Psychol 1994;25(4):501-524. [CrossRef]
    31. Wang WL, Lee HL, Fetzer SJ. Challenges and strategies of instrument translation. West J Nurs Res 2006;28(3):310-321. [CrossRef] [Medline]
    32. Mustafa N, Safii NS, Jaffar A, Sani NS, Mohamad MI, Abd Rahman AH, et al. Malay version of the mHealth app usability questionnaire (M-MAUQ): translation, adaptation, and validation study. JMIR Mhealth Uhealth 2021;9(2):e24457 [FREE Full text] [CrossRef] [Medline]
    33. Rubio DM, Berg-Weger M, Tebb SS, Lee ES, Rauch S. Objectifying content validity: conducting a content validity study in social work research. Soc Work Res 2003;27(2):94-104. [CrossRef]
    34. Shan Y, Xing Z, Dong Z, Ji M, Wang D, Cao X. Translating and adapting the DISCERN instrument into a simplified Chinese version and validating its reliability: development and usability study. J Med Internet Res 2023;25:e40733 [FREE Full text] [CrossRef] [Medline]
    35. Mohamad Marzuki MF, Yaacob NA, Yaacob NM. Translation, cross-cultural adaptation, and validation of the Malay version of the system usability scale questionnaire for the assessment of mobile apps. JMIR Hum Factors 2018;5(2):e10308 [FREE Full text] [CrossRef] [Medline]
    36. Logullo P, Torloni MR, de O C Latorraca C, Riera R. The Brazilian Portuguese version of the DISCERN instrument: translation procedures and psychometric properties. Value Health Reg Issues 2019;20:172-179. [CrossRef] [Medline]
    37. Zhao S, Cao Y, Cao H, Liu K, Lv X, Zhang J, et al. Chinese version of the mHealth app usability questionnaire: cross-cultural adaptation and validation. Front Psychol 2022;13:813309 [FREE Full text] [CrossRef] [Medline]
    38. Capitulo KL, Cornelio MA, Lenz ER. Translating the short version of the perinatal grief scale: process and challenges. Appl Nurs Res 2001;14(3):165-170. [CrossRef] [Medline]
    39. Sperber AD. Translation and validation of study instruments for cross-cultural research. Gastroenterology 2004;126(1 suppl 1):S124-S128. [CrossRef] [Medline]
    40. Yu DSF, Lee DTF, Woo J. Issues and challenges of instrument translation. West J Nurs Res 2004;26(3):307-320. [CrossRef] [Medline]
    41. Fleiss JL. Statistical methods for rates and proportions.2nd ed. New York, NY: Wiley; 1981.
    42. Hoffmann T, Ladner Y. Assessing the suitability of written stroke materials: an evaluation of the interrater reliability of the suitability assessment of materials (SAM) checklist. Top Stroke Rehabil 2012;19(5):417-422 [FREE Full text] [CrossRef] [Medline]
    43. Weintraub D, Maliski SL, Fink A, Choe S, Litwin MS. Suitability of prostate cancer education materials: applying a standardized assessment tool to currently available materials. Patient Educ Couns 2004;55(2):275-280. [CrossRef] [Medline]
    44. Department of Health and Human Services. Patient Education Materials Assessment Tool for Printable Materials (PEMAT-P). Agency for Healthcare Research and Quality. Rockville, MD; 2013 Oct.   URL: https://www.ahrq.gov/health-literacy/patient-education/pemat-p.html

    ‎
    CL: comparability of language
    C-PEMAT-P: a Chinese version of the Patient Education Materials Assessment Tool for printable materials
    PEMAT-P: the Patient Education Materials Assessment Tool for printable materials
    SI: similarity of interpretability

    Edited by G Eysenbach; submitted 24.05.22; peer-reviewed by K Wright , J Guendouzi, J Brooke, PF Chen, P Ding; comments to author 27.02.23; revised version received 02.03.23; accepted 30.03.23; published 18.05.23

    Copyright

    ©Yi Shan, Meng Ji, Zhaogang Dong, Zhaoquan Xing, Ding Wang, Xiangting Cao. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 18.05.2023.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.