This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
Developing accessible Web-based materials to support diabetes self-management in people with lower levels of health literacy is a continuing challenge.
The objective of this international study was to develop a Web-based intervention promoting physical activity among people with type 2 diabetes to determine whether audiovisual presentation and interactivity (quizzes, planners, tailoring) could help to overcome the digital divide by making digital interventions accessible and effective for people with all levels of health literacy. This study also aimed to determine whether these materials can improve health literacy outcomes for people with lower levels of health literacy and also be effective for people with higher levels of health literacy.
To assess the impact of interactivity and audiovisual features on usage, engagement, and health literacy outcomes, we designed two versions of a Web-based intervention (one interactive and one plain-text version of the same content) to promote physical activity in people with type 2 diabetes. We randomly assigned participants from the United Kingdom, Austria, Germany, Ireland, and Taiwan to either an interactive or plain-text version of the intervention in English, German, or Mandarin. Intervention usage was objectively recorded by the intervention software. Self-report measures were taken at baseline and follow-up (immediately after participants viewed the intervention) and included measures of health literacy, engagement (website satisfaction and willingness to recommend the intervention to others), and health literacy outcomes (diabetes knowledge, enablement, attitude, perceived behavioral control, and intention to undertake physical activity).
In total, 1041 people took part in this study. Of the 1005 who completed health literacy information, 268 (26.67%) had intermediate or low levels of health literacy. The interactive intervention overall did not produce better outcomes than did the plain-text version. Participants in the plain-text intervention group looked at significantly more sections of the intervention (mean difference –0.47, 95% CI –0.64 to –0.30, P<.001), but this did not lead to better outcomes. Health literacy outcomes, including attitudes and intentions to engage in physical activity, significantly improved following the intervention for participants in both intervention groups. These improvements were similar across higher and lower health literacy levels and in all countries. Participants in the interactive intervention group had acquired more diabetes knowledge (mean difference 0.80, 95% CI 0.65-0.94, P<.001). Participants from both groups reported high levels of website satisfaction and would recommend the website to others.
Following established practice for simple, clear design and presentation and using a person-based approach to intervention development, with in-depth iterative feedback from users, may be more important than interactivity and audiovisual presentations when developing accessible digital health interventions to improve health literacy outcomes.
International Standard Randomized Controlled Trial Number (ISRCTN): 43587048; http://www.isrctn.com/ISRCTN43587048. (Archived by WebCite at http://www.webcitation.org/6nGhaP9bv)
Health literacy has been defined as “knowledge, motivation and competences to access, understand, appraise, and apply health information” [
Barriers to accessing support for self-management of chronic health problems include disability, cost, work or family responsibilities, and lack of transport [
To date, interventions to reduce the literacy burden and improve health literacy have included using simple language, audiovisual or pictorial formats, interactivity, and tailoring of content to individuals’ needs (if the intervention is Web based). Reviews of the effectiveness of such interventions for the general public and mixed-patient populations [
This study addressed the evidence gap regarding how best to design Web-based materials for the growing population of patients with basic literacy and computer skills but lower levels of health literacy. We developed a Web-based intervention to promote physical activity in people with type 2 diabetes, following established best practice for designing accessible Web-based written content. We included a range of interactive elements (quiz, tailoring, a planner) and audiovisual modes of presentation, so that we could evaluate whether these improved usage and health literacy outcomes, particularly in those with lower levels of health literacy. We used our person-based approach to intervention development [
This paper reports on a subsequent large international quantitative study comparing this Web-based intervention with a static, plain-text presentation of identical content. The study evaluated engagement and heath literacy outcomes in people with varied levels of health literacy. We measured engagement by objectively recorded intervention usage and self-reported user experience (website satisfaction and whether participants would recommend the website to others) [
We developed 2 Web-based interventions using the LifeGuide software, an open access platform for developing Web-based behavior change interventions [
We designed the interactive version to assess the additional impact that interactivity, audiovisual features, and tailoring may have on engagement with the intervention and health literacy outcomes in people with varied health literacy levels. Audiovisual aspects of the interactive intervention were positive images throughout, and a series of audiovisual sequences demonstrating lifestyle and physical activities (tailored to age and sex). The interactive features of the website consisted of a quiz, a physical activity planner, and tailored advice, feedback, and images based on user responses to questions (such as current physical activity levels, attitudes to physical activity, age, and sex).
We first developed the intervention in English for testing in the United Kingdom, and then adapted and translated it for testing in Austria, Germany, Ireland, Taiwan, and the United States. Researchers in the United States did not take part in this subsequent trial. We followed our person-based approach to intervention development [
We carried out a multisite randomized trial in the United Kingdom, Austria, Germany, Ireland, and Taiwan to compare the interactive Web-based materials versus a plain-text Web-based version of the intervention. The plain-text intervention contained the same content and structure as the interactive version, but all tailoring, interactivity, and audiovisual features were removed. Ethics and research governance approvals were granted by the University of Southampton and UK National Health Service (NHS) research ethics committees (number 13/LO/0316).
Participants were invited to take part in the study if they were over 18 years old with a diagnosis of type 2 diabetes, had access to the Internet, were able to read the intervention language (English, German, or Mandarin), and give informed consent. We recruited participants from the United Kingdom, Austria, Germany, Ireland, and Taiwan between July 2014 and March 2015. Minor country differences in recruitment procedures were permitted to allow for differing health care systems and patient access. UK participants were recruited through 43 primary care practices specifically selected for being in areas of high deprivation in order to reach more people with low health literacy. Participants in Ireland and Taiwan were recruited opportunistically by health care professionals in diabetes outpatient clinics, and participants in Austria and Germany were recruited through national diabetes support group newsletters and advertisements placed on the Internet. Health care professionals in the United Kingdom, Ireland, and Taiwan screened potential participants to exclude patients with potential difficulties, including severe mental health problems, palliative care, recent bereavement, and inability to complete research measures (eg, learning disability, inability to read or speak an intervention language) before they were invited to the study.
Participants from all countries were presented with details of the study, research team contact details for more information, and a website URL where they could log in to the Web-based intervention on their own time. Participant information stated that we were comparing two types of webpages to see which was more helpful; it did not mention website features such as interactivity or audiovisual features. Participants were therefore blinded to what the differences between the 2 arms were. Consent was given online, and participants completed a very brief baseline questionnaire before being randomly assigned to 1 of the 2 groups (with a 50% ratio). Participants were then presented with either the interactive or plain-text Web-based materials, depending on randomization assignment. Participants were asked follow-up questions immediately after using the intervention. All recruitment and follow-up procedures (including full study information, obtaining informed consent, baseline and follow-up data collection, and randomization) were Web based using automated procedures carried out by the LifeGuide software [
We calculated the sample size a priori using the G*Power 3 (version 3.1.9.2) sample size calculation program [
Participants completed Web-based assessments at baseline (immediately before) and follow-up (immediately after using the intervention materials). We collected demographic variables at baseline, consisting of age, sex, time since diabetes diagnosis, and age they left full-time education. Participants’ levels of physical activity during the previous week were measured at follow-up using the International Physical Activity Questionnaire-Short Form (IPAQ-SF) self-administered questionnaire assessing the minutes spent doing vigorous and moderate activity and walking in the last 7 days [
We measured engagement with the Web-based intervention by intervention usage and self-reported measures of engagement. Intervention usage was measured by the number of intervention sections completed, as total time spent on the intervention was likely to be confounded with format (plain text vs interactive). Both the interactive and plain-text intervention were designed to comprise 5 distinct sections: knowledge of physical activity benefits (with/without interactive quiz); advice on selecting physical activities (with/without tailoring); advice on planning physical activity (with/without interactive planner); success stories (with/without audiovisual presentation); access to further information about undertaking physical activity. All intervention usage data was automatically recorded by the LifeGuide software [
Health literacy outcomes were (1) diabetes knowledge, (2) patient enablement, and (3) attitude, behavioral control, and intention to undertake physical activity. Diabetes knowledge was measured by a 9-item knowledge quiz based on the intervention content. Patient enablement was measured by 3 items from the Patient Enablement Instrument [
We measured health literacy at baseline by a validated single item: “How often do you have problems learning about your condition because of difficulty understanding written information?” [
We analyzed the data using IBM SPSS for Windows version 14.0 (IBM Corporation) and Stata statistical software Special Edition Release 2007 (version 13; StataCorp LP), following a prespecified data analysis plan developed with our statistician (BS) and approved by the whole Diabetes Literacy consortium. All comparisons of the plain-text and interactive versions of the website controlled for potential confounding effects of the covariates health literacy, education, age, sex, and illness duration. We allowed for clustering by country by including country as a random effect in the model.
Due to the small numbers of participants with low health literacy levels, we categorized health literacy as low/intermediate compared with high health literacy. To avoid undertaking too many between-country comparisons, analyses by country compared UK data with a pooled sample of all other countries, as the UK sample was the largest and the intervention materials were originally developed for testing in the United Kingdom, and then translated and adapted for other countries and cultures.
The primary research question asked whether an interactive, tailored, and audiovisual Web-based intervention can lead to better engagement than a plain-text version of the same content can. The primary analysis compared the number of intervention sections completed by participants randomly assigned to the interactive intervention versus the number completed by participants randomly assigned to the plain-text intervention to test the prediction that more sections of the interactive version of the Web-based intervention would be completed. We used linear regression to compare the mean difference between intervention groups. We then examined whether intervention usage was moderated by health literacy level or by country. For these analyses, we carried out linear regressions to look for group differences by health literacy level and country. Post hoc exploratory analyses of Web usage were carried out using visualization analyses to examine patterns of intervention usage. Intervention usage data were analyzed using the LifeGuide visualization tool [
Secondary research questions asked whether people with high and low health literacy found the materials engaging, and whether the intervention improved health literacy outcomes in people with lower and high levels of health literacy. In order to answer these questions, we analyzed self-report measures of engagement (website satisfaction; recommending the website to others) and health literacy outcomes (diabetes knowledge; patient enablement; and change in attitude, behavioral control, and intention to undertake physical activity) using linear regression models and then assessed for potential moderator effects by heath literacy level and country.
The main outcome for this study was intervention usage, which was automatically recorded by the intervention software for all participants and therefore had no missing data. We investigated levels of missing data for baseline and follow-up measures and compared the frequency of missing data between the 2 intervention groups. Levels of missing data were high for the diabetes knowledge quiz score (459/1041, 44.09% missing) and the single item measuring whether participants would recommend the intervention to others (231/1041, 22.19% missing data). We assumed that these were at random and applied a multiple imputation model of 100 imputations for missing secondary outcomes and key covariates. We present this analysis as a sensitivity analysis alongside the main analysis on complete cases.
In total, 1045 participants from the United Kingdom, Austria, Germany, Ireland, and Taiwan participated in the study and were randomly assigned to view either the interactive intervention or the plain-text intervention. Of these, 4 participants used the Back button on their Internet browsers to be rerandomized and were consequently excluded, resulting in 1041 participants in the final analysis. We successfully measured the primary outcome, intervention usage, for 100% of randomly assigned participants. See
Consolidated Standards of Reporting Trials (CONSORT) flow diagram.
Participants in this study were predominantly male (662/1041, 63.59%), with a mean age of 62 years. On average, participants left full-time education before the age of 18 years and had been diagnosed with type 2 diabetes for 9.2 years (this ranged from just a few months to 50 years). The majority of participants (737/1005, 73.33%) had high levels of health literacy, while 268/1005 (26.67%) had intermediate or low levels of health literacy. A total of 835/1041 (80.21%) of participants completed the IPAQ-SF physical activity questionnaire. Most of these participants reported being inactive (561/835, 67.2%), while some reported being minimally active (190/835, 22.8%) and a minority reported being highly active (84/835, 10.1%). Participant characteristics were similar across both groups at baseline, with the only slight difference being higher health literacy levels in the interactive group. See
Participant characteristics in the 2 arms of the Web-based intervention promoting physical activity among people with type 2 diabetes.
Characteristic | Group | ||
Plain-text (n=497) | Interactive (n=544) | ||
Female, n (%) | 182 (36.6) | 197 (36.2) | |
Age in years, mean (SD) | 61.5 (11.2) | 62.4 (11.4) | |
Years since diagnosis, mean (SD) | 9.1 (9.1) | 9.5 (9.3) | |
Age when left full-time education, mean (SD) | 17.8 (3.0) | 17.8 (3.0) | |
Low | 37/478 (7.7) | 30/527 (5.7) | |
Intermediate | 105/478 (22.0) | 96/527 (18.2) | |
High | 336/478 (70.3) | 401/527 (76.1) | |
IPAQ-SFa, mean (SD) | 15.1 (3.5) | 15.0 (3.7) | |
Highly active, n (%) | 35/431 (8.1) | 49/404 (12.1) | |
Minimally active, n (%) | 106/431 (24.6) | 84/404 (20.8) | |
Inactive, n (%) | 290/431 (67.3) | 271/404 (67.1) |
aIPAQ-SF: International Physical Activity Questionnaire-Short Form.
The primary outcome in this study was Web-based intervention usage to test whether the interactive intervention led to better engagement than the plain-text version. Analysis of usage data found a significant difference in intervention usage between the 2 groups, with participants in the interactive intervention group being likely to complete fewer of the 5 intervention sections than were participants in the plain-text intervention group (mean difference –0.47, 95% CI –0.64 to –0.30,
Moderator analysis examined intervention usage by health literacy level. Participants with higher levels of health literacy were significantly more likely to complete more sections of the intervention (mean difference 0.25, 95% CI 0.05-0.45,
Results of analysis of intervention usage as determined by number of sections completed, and results of self-reported measures of engagement and moderator analyses of self-reported engagement, by intervention group.
Analysis | Intervention group | Univariate difference | Multivariate differencea | Multivariate differencea based on 100 imputations | |||||
Plain text | Interactive | Mean (95% CI) | Mean (95% CI) | Mean (95% CI) | |||||
No. of sections completed, mean (SD) | 4.5 (1.3) | 4.0 (1.5) | –0.47 (–0.64 to –0.30) | <.001 | –0.49 (–0.67 to –0.31) | <.001 | N/Ab | N/A | |
Satisfied with website, mean (SD) | 4.1 (2.0) | 4.1 (1.9) | 0.03 (–0.24 to 0.30) | .82 | 0.05 (–0.22 to 0.33) | .70 | 0.08 (–0.19 to 0.35) | .54 | |
Would recommend to others, n (%) | 281/419 (67.1) | 248/391 (63.4) | 0.85 (0.64 to 1.14) | .28 | 0.85 (0.62 to 1.15) | .29 | 0.78 (0.58 to 1.05) | .10 |
aAll analyses controlled for possible confounding by age, sex, time since diagnosis, age when the participant left education, health literacy, and for clustering by country.
bN/A: not applicable.
Results of analysis of intervention usage as determined by number of sections completed, and results of self-reported measures of engagement and moderator analyses of self-reported engagement, by health literacy level.
Analysis | Health literacy level | Univariate difference | Multivariate differencea | Multivariate differencea based on 100 imputations | |||||
Lower | High | Mean (95% CI) | Mean (95% CI) | Mean (95% CI) | |||||
No. of sections completed, mean (SD) | 4.1 (1.5) | 4.3 (1.4) | 0.25 (0.05 to 0.45) | .02 | 0.28 (0.08 to 0.48) | .01 | N/Ab | N/A | |
Satisfied with website, mean (SD) | 4.1 (2.0) | 4.1 (2.0) | –0.03 (–0.34 to 0.29) | .87 | 0.05 (–0.27 to 0.37) | .76 | 0.04 (–0.28 to 0.35) | .82 | |
Would recommend to others, n (%) | 139/195 (71.3) | 372/591 (62.9) | 0.70 (0.48 to 0.97) | .04 | 0.64 (0.44 to 0.93) | .02 | 0.69 (0.48 to 1.01) | .05 |
aAll analyses controlled for possible confounding by age, sex, time since diagnosis, age when the participant left education, health literacy, and for clustering by country.
bN/A: not applicable.
Visualization of intervention usage by health literacy level and intervention. Blue: time spent on quiz; yellow: time spent on physical activity planner; red: time spent on reading personal tips; green: time spent on audiovisual sequences.
We carried out exploratory analyses to examine whether intervention usage differed by country (comparing the United Kingdom versus the other participating countries). Patterns of usage were similar in the United Kingdom and the other countries. See
The self-reported measures of engagement were website satisfaction and a single item measuring whether participants would recommend the website to others. We used these items to address the secondary research question asking whether Web-based materials can be developed to be engaging to people with low and high levels of health literacy. There were no significant group differences, with participants in both groups reporting high levels of website satisfaction and the majority of participants in both groups reporting that they would be likely to recommend the website to others.
We carried out exploratory analyses to evaluate whether self-reported measures of engagement varied by health literacy level. Participants with lower health literacy were significantly more likely to recommend the website to friends or family with diabetes (mean difference –0.70, 95% CI 0.48-0.97,
Secondary research questions asked whether the Web-based materials could improve health literacy outcomes in people with low health literacy and be effective for people with higher levels of health literacy. The health literacy outcomes in this study were (1) diabetes knowledge, (2) patient enablement, and (3) change in attitude, behavioral control, and intention to undertake physical activity. There was a significant group difference in participants’ diabetes knowledge, with participants in the interactive group scoring significantly higher than the plain-text intervention group (mean difference 0.80, 95% CI 0.65-0.94,
Health literacy outcomes by intervention group.
Outcome | Intervention group | Univariate difference | Multivariate differencea | Multivariate differencea based on 100 imputations | ||||
Plain text | Interactive | Mean (95% CI) | Mean (95% CI) | Mean (95% CI) | ||||
Diabetes knowledge, mean (SD) | 8.0 (1.1) | 8.8 (0.5) | 0.80 (0.65 to 0.94) | <.001 | 0.78 (0.63 to 0.92) | <.001 | 0.74 (0.50 to 0.88) | <.001 |
Diabetes knowledge score of 9 vs lower score, n (%) | 124/303 (40.9) | 228/279 (81.7) | 6.5 (4.4 to 9.4) | <.001 | 6.9 (4.6 to 10.3) | <.001 | 4.90 (3.35 to 7.17) | <.001 |
Patient Enablement Instrument, mean (SD) | 7.5 (3.1) | 7.6 (3.0) | 0.08 (–0.33 to 0.49) | .70 | 0.02 (–0.40 to 0.43) | .93 | 0.17 (–0.25 to 0.58) | .44 |
aAll analyses controlled for possible confounding by age, sex, time since diagnosis, age when the participant left education, health literacy, and for clustering by country.
Moderator analyses explored these results by health literacy level. There was a trend for people with higher levels of health literacy to score higher on the Patient Enablement Instrument (multivariate mean difference 0.53, 95% CI 0.04-1.02,
Participants were asked about their attitudes and intentions toward physical activity at baseline and again at follow-up, enabling an analysis to establish whether the score had changed within each group. In both intervention groups, and across all health literacy levels, the score at follow-up was significantly higher than at baseline, indicating that participants from all groups had more positive attitudes and intentions toward physical activity after viewing the intervention materials.
Moderator analyses of health literacy outcomes by health literacy levels.
Outcome | Health literacy level | Univariate difference | Multivariate differencea | Multivariate differencea based on 100 imputations | ||||
Lower | High | Mean (95% CI) | Mean (95% CI) | Mean (95% CI) | ||||
Diabetes knowledge, mean (SD) | 8.2 (1.1) | 8.4 (0.9) | 0.16 (–0.02 to 0.35) | .09 | 0.13 (–0.05 to 0.30) | .16 | 0.13 (–0.06 to 0.32) | .18 |
Diabetes knowledge score of 9 vs lower score, n (%) | 73/132 (55.3) | 270/434 (62.2) | 1.33 (0.90 to 1.97) | .16 | 1.31 (0.83 to 2.07) | .25 | 1.27 (0.84 to 1.92) | .26 |
Patient Enablement Instrument, mean (SD) | 7.3 (2.8) | 7.7 (3.1) | 0.39 (–0.09 to 0.87) | .11 | 0.53 (0.04 to 1.02) | .03 | 0.40 (–0.09 to 0.88) | .11 |
aAll analyses controlled for possible confounding by age, sex, time since diagnosis, age when the participant left education, and for clustering by country.
Change in attitude behavioral control and physical activity intentions from baseline to follow-up across all groups and literacy levels.
Outcome | Plain text group | Interactive group | Lower health literacy | High health literacy | ||||
Mean (95% CI) | Mean (95% CI) | Mean (95% CI) | Mean (95% CI) | |||||
Physical activity attitude | 0.10 (0.02-0.18) | .01 | 0.22 (0.11-0.34) | <.001 | 0.15 (0.02-0.27) | .02 | 0.15 (0.07-0.23) | <.001 |
Perceived behavioral control | 0.34 (0.24-0.45) | .001 | 0.35 (0.22-0.47) | <.001 | 0.33 (0.17-0.49) | <.001 | 0.34 (0.24-0.43) | <.001 |
Physical activity intention | 0.35 (0.24-0.45) | <.001 | 0.49 (0.35-0.63) | <.001 | 0.27 (0.10-0.44) | .002 | 0.46 (0.35-0.56) | <.001 |
The main finding of this study was that the interactive intervention overall did not produce better outcomes than those obtained by a plain-text version of the intervention. Participants in the plain-text intervention group showed higher levels of engagement by completing more sections of the intervention, although this did not lead to better health literacy outcomes, and participants in the interactive intervention group had better diabetes knowledge.
Health literacy outcomes significantly improved following the intervention to a very similar extent in both groups. These significant changes were reflected across all health literacy levels and all countries, with participants reporting increased beliefs in the benefits of physical activity, greater confidence in undertaking physical activity, and a stronger intention to increase physical activity as a result of the intervention. Given the low levels of physical activity reported by our sample, these changes in attitude to physical activity are positive, and it is encouraging that we observed these changes in those with lower levels of health literacy, since low self-confidence for physical activity has been shown to be a key mediator of the association between low health literacy and inactivity [
Analysis by health literacy level revealed few differences. Participants with high levels of health literacy completed more sections of the intervention, but this did not lead to better health literacy outcomes. Participants with high health literacy reported higher levels of enablement, and participants with lower health literacy were more likely to recommend the intervention to others, but these differences were not significant after correcting for missing data. Despite these minor group differences, there are encouraging signs that the intervention design was accessible and helpful for people with all health literacy levels. These findings are consistent with evidence from previous research that interventions designed to be accessible for people with lower health literacy can be suitable for people with higher health literacy [
However, more work is needed to engage hard-to-reach populations in Web-based interventions. Despite deliberately sampling in socially deprived populations, we attracted surprisingly few people with lower levels of health literacy.
This study did not succeed in recruiting many participants with very low levels of health literacy, and the results can therefore not be generalized to this group. It is also important to note that the results only refer to our version of interactivity, and others may be able to produce more engaging interactive materials. This study was not powered for examining interactions, and all subgroup analyses were exploratory and should be interpreted with caution. There were minor recruitment differences between countries, which should be taken into account when interpreting response rates. We did not undertake longer-term follow-up and therefore do not know the extent to which the immediate intervention effect will endure in this population. Since this study did not include a control group, we cannot draw firm conclusions regarding the effectiveness of the Web-based intervention content, since changes in attitudes before and after viewing the content could in theory have been due to other factors.
In this study, a good, clear design and person-based intervention development [
Participant characteristics by country.
Moderator analysis of intervention usage by country.
Moderator analyses of self-reported engagement by country.
Moderator analyses of health literacy outcomes by country.
CONSORT-EHEALTH checklist V1.6.2 [
Consolidated Standards of Reporting Trials
International Physical Activity Questionnaire-Short Form
National Health Service
This research was part of the Diabetes Literacy project supported by grant FP7-Health-2012-Innovation-1/306186 of the European Commission. We would like to thank the members of the Diabetes Literacy Study Group who assisted with data collection, in particular Peter Schwarz, Jürgen Pelikan, Sarah Gibney, and Becky Sun. We would especially like to thank our 2 lay advisors, Andy Hamson and Wendy Heath. We would also like to acknowledge the support of the National Institute of Health Research Clinical Research Network.
None declared.