Published on 05.11.20 in Vol 22, No 11 (2020): November
Preprints (earlier versions) of this paper are available at http://preprints.jmir.org/preprint/20322, first published May 17, 2020.
My Diabetes Coach, a Mobile App–Based Interactive Conversational Agent to Support Type 2 Diabetes Self-Management: Randomized Effectiveness-Implementation Trial
Background: Delivering self-management support to people with type 2 diabetes mellitus is essential to reduce the health system burden and to empower people with the skills, knowledge, and confidence needed to take an active role in managing their own health.
Objective: This study aims to evaluate the adoption, use, and effectiveness of the My Diabetes Coach (MDC) program, an app-based interactive embodied conversational agent, Laura, designed to support diabetes self-management in the home setting over 12 months.
Methods: This randomized controlled trial evaluated both the implementation and effectiveness of the MDC program. Adults with type 2 diabetes in Australia were recruited and randomized to the intervention arm (MDC) or the control arm (usual care). Program use was tracked over 12 months. Coprimary outcomes included changes in glycated hemoglobin (HbA1c) and health-related quality of life (HRQoL). Data were assessed at baseline and at 6 and 12 months, and analyzed using linear mixed-effects regression models.
Results: A total of 187 adults with type 2 diabetes (mean 57 years, SD 10 years; 41.7% women) were recruited and randomly allocated to the intervention (n=93) and control (n=94) arms. MDC program users (92/93 participants) completed 1942 chats with Laura, averaging 243 min (SD 212) per person over 12 months. Compared with baseline, the mean estimated HbA1c decreased in both arms at 12 months (intervention: 0.33% and control: 0.20%), but the net differences between the two arms in change of HbA1c (−0.04%, 95% CI −0.45 to 0.36; P=.83) was not statistically significant. At 12 months, HRQoL utility scores improved in the intervention arm, compared with the control arm (between-arm difference: 0.04, 95% CI 0.00 to 0.07; P=.04).
Conclusions: The MDC program was successfully adopted and used by individuals with type 2 diabetes and significantly improved the users’ HRQoL. These findings suggest the potential for wider implementation of technology-enabled conversation-based programs for supporting diabetes self-management. Future studies should focus on strategies to maintain program usage and HbA1c improvement.
Trial Registration: Australia New Zealand Clinical Trials Registry (ACTRN) 12614001229662; https://anzctr.org.au/Trial/Registration/TrialReview.aspx?ACTRN=12614001229662
J Med Internet Res 2020;22(11):e20322
Type 2 diabetes mellitus (T2DM) is a common chronic condition that places a significant burden on individuals and the health care system. The prevalence of diabetes has risen substantially over the past two decades worldwide . However, clinical outcomes have not improved significantly despite considerable investment and advances in treatments and technologies over this period [ ]. Delivering self-management support to people with T2DM is essential to reduce the health system burden and to empower people with the skills, knowledge, and confidence needed to take an active role in managing their own health [ ].
Health coaching programs, incorporating continuous feedback and reinforcement  delivered to people with diabetes by health care professionals, have been shown to be an effective strategy to improve glycemic management [ , ]. However, the human delivery of such programs on a large scale is very challenging, given the health system and workforce constraints. The emergence of information and communication technologies in recent years offers the potential to deliver such programs at scale and with relatively low costs [ ]. Our previous studies and others have demonstrated the effectiveness and cost-effectiveness of using technology-enabled programs, including computer-based or telephone-based coaching, in promoting behavior change and self-management among adults with T2DM [ - ]. The increasing ubiquity of smartphones and other smart devices now offers the potential to further optimize the delivery of such programs at scale by the use of apps.
Despite the increasing use of mobile apps for health purposes, reviews have found that existing digital health solutions are not generally able to meet the needs of people with diabetes, and more evidence is required before their wider scale-up [, ]. Most commercially available apps employ limited use of behavior change techniques and inadequate features, other than self-monitoring [ , ]. Furthermore, people with diabetes expect apps to be engaging; incorporating multiple functions; and covering a broad range of content, including psychological and emotional support [ ]. Although some recent systematic reviews and meta-analyses have indicated a modest effect for app-based interventions to support diabetes self-management [ , , ], there still remains a great deal of uncertainty about their feasibility, acceptability, and effectiveness in the real world [ , - ].
By adapting our team’s previous effective and cost-effective Telephone-Linked Care for Diabetes (TLC diabetes) program  to a more contemporary technology platform, we designed the My Diabetes Coach (MDC) program, which incorporates an app-based interactive embodied conversational agent, Laura, to provide more accessible and engaging self-management support, monitoring, and coaching to adults with T2DM in Australia. We undertook a hybrid effectiveness-implementation study [ ] with a randomized controlled design. The study aims to evaluate both the implementation of the MDC program over 12 months and its effectiveness. In this paper, we report on the program adoption and use during the trial and program effectiveness in terms of the coprimary outcomes and selected secondary outcomes. The report of the study is in line with the CONSORT (Consolidated Standards of Reporting Trials) guideline and the extension guideline of the CONSORT-eHealth guidelines.
Study Design Overview
This trial is a two-arm, open-label, randomized controlled trial with participants recruited between June 2016 and April 2017 in Australia. The trial was registered before recruitment (Australia New Zealand Clinical Trials Registry ID: ACTRN12614001229662). Full ethics approval was granted by the University of Melbourne’s Human Research Ethics Committee (Ethics ID 1442433). Participants provided written informed consent and returned the informed consent forms to the research team via email or fax, including permission for their general practitioners (GPs) to regularly share clinical data with the study team.
Adults (aged ≥18 years) diagnosed with T2DM, registered with the National Diabetes Service Scheme (NDSS) for less than 10 years, with basic English language skills, who had access to an internet-enabled smart device with an up-to-date operating system (at least iOS 8.0 for Apple and 4.2 for Android) were eligible to participate in the study. Participants were ineligible if they were pregnant or planning to become pregnant, had severe comorbid conditions that would compromise their participation, or did not have stable doses of diabetes-related medication over the previous 4 weeks or more.
To assist with recruitment, on behalf of the research team, the NDSS sent invitation letters to registered adults with T2DM living in the Australian states of Queensland, Victoria, and Western Australia. These 3 states comprise 54.5% of all Australians with diagnosed diabetes . Recruitment information was also posted on social media via the University of Melbourne website, Facebook, and Twitter and was made available via Diabetes Australia and state-based diabetes organizations (eg, newsletters or magazines). This approach enabled people from all Australian states or territories to enter the study. People who submitted an expression of interest were contacted by the research team by telephone within 48 hours to complete eligibility screening. Eligible participants who provided consent and completed the baseline survey were formally enrolled in the study.
Participants were randomly allocated to the intervention or control arm using a 2x4 block randomization sequence, programmed into a Redcap data management system , with the individual as the unit of randomization. The program coordinator informed participants and their GPs of the randomization allocation via telephone or email. Due to the nature of the intervention, participants, their GPs, and the program coordinator could not be blinded to participant study allocation. A statistician who verified the data analysis was blinded to the randomization.
Intervention and Control
Intervention: MDC Program
Participants allocated to the intervention arm received access to the MDC program for up to 12 months. The overall program comprises 5 components: the MDC app; a printed user guide; the MDC website; an optional blood glucose meter with Bluetooth capability (Accu-Chek Aviva Connect, Roche Diabetes Care); and a small number of brief, structured interactions with a program coordinator, primarily for technical assistance. The MDC app was adapted from the previous TLC diabetes program  and was co-designed with the support of the Bupa Foundation and a technology company named Clevertar. details the MDC program and these 5 components.
MDC delivers personalized support, monitoring, and motivational coaching via an embodied conversational agent, Laura, through a series of modules covering blood glucose monitoring, healthy eating, physical activity, medication taking, and foot care. The conversational scripts and algorithms guiding each individual’s progress were designed by applying behavior change theories and techniques, including the transtheoretical model , social cognitive theory [ ], gamification [ ], and other concepts derived from chronic disease self-management [ ]. Algorithms were further tailored according to the clinical targets and recommendations provided by each participant’s GP.
Participants were encouraged to use the app weekly to complete online modules by chatting with Laura or touching buttons on the screen. Each appointment module with Laura began with a review of progress with feedback, education and counseling on the chosen topic, and incorporated tips on overcoming barriers, followed by a short quiz and closing remarks.
In addition to the MDC app, participants were also encouraged to regularly access the user guide and the MDC website and to join the discussion forums on diabetes self-management topics posted fortnightly by the program coordinator on the website. The program coordinator, supported by a web-based user management portal, assisted participants in dealing with system-generated technical alerts by communicating with participants, their GPs, and Clevertar. The program coordinator also telephoned participants after 1, 4, 8, 12, and 24 weeks of program access to answer questions, to troubleshoot technical issues, and to encourage program use.
Participants in the control arm were encouraged to continue their routine diabetes self-care, including access to health care services, resources accessed via NDSS, and the diabetes not-for-profit organizations in their states. They received a quarterly project newsletter to maintain their interest in the study. Following the 12-month data collection, participants in the control arm received access to the MDC program if they wished.
Measurement and Outcomes
We used the Reach, Effectiveness, Adoption, Implementation, and Maintenance framework to evaluate the impact of the MDC intervention, which covered the 5 dimensions in terms of reach, effectiveness, adoption, implementation, and maintenance . This paper focuses on program adoption, use, and effectiveness.
Program Adoption and Use
Program adoption and use were tracked using the program management portal. The key metrics included the number of completed chats with Laura, the total duration of chats, and the number of blood glucose levels uploaded by participants. Other metrics included the number of technical and clinical alerts generated by the system for the project coordinator and the number of posts on the web-based discussion forum of the MDC website.
Program effectiveness was measured by both clinical and psycho-behavioral outcomes. The coprimary outcomes were changes (12 months compared with baseline) in glycated hemoglobin (HbA1c) and health-related quality of life (HRQoL), which were examined in terms of between-arm differences. The secondary time point of analysis examined the change between baseline and 6 month. HbA1c (reported as % and mmol/mol) was measured through a pathology blood test that each participant’s GP requested. HRQoL was assessed via participants’ completion of the Assessment of Quality of Life (AQoL)-8D scale, which is a 35-item multi-attribute utility instrument covering 8 dimensions focused on independent living, happiness, mental health, coping, relationships, self-worth, pain, and senses [, ]. The AQoL-8D provides a utility score ranging from −0.04 to 1, where higher scores indicate better utility value in HRQoL.
Selected secondary outcomes reported in this paper include anxiety and depressive symptoms, assessed by the Hospital Anxiety and Depression Scale (HADS) ; diabetes-specific distress, assessed by the Problem Areas in Diabetes scale [ ]; and clinical measurement of body weight. Secondary outcomes were assessed by examining between-arm differences at 6 and 12 months compared with baseline. Further secondary outcomes will be reported elsewhere.
Data Collection Procedures
Data were collected at 4 time points, including screening, baseline, 6 months, and 12 months post randomization. During the screening telephone calls, the research team contacted participants and recorded sociodemographic characteristics (age, sex, education, employment, and language capability), clinical characteristics (duration of NDSS registration and stabilization of the health condition), and app use variables in the REDcap data collection system . Assessments of participants’ clinical and psycho-behavioral data were conducted following a standard protocol at baseline and at 6 and 12 months. At each time point, the research team contacted participants in both arms via email to advise that data collection was required. The full data collection details are listed in . The email contained a link to the web-based self-administered questionnaire via REDcap. The research team also requested clinical measures from participants and their GPs. If clinical measures were taken within the past 3 months, GPs provided these to the research team; otherwise, participants made an appointment with their GPs to request new tests. GPs provided the clinical information to the research team via fax, and the research team entered the information into REDcap. Data related to program adoption and use were captured automatically by the program (throughout the study period, for all program users), and the information was provided to the research team by Clevertar.
Analytical and Statistical Approaches
On the basis of a previous study , we estimated that a sample size of 180 participants would provide 80% power at a two-tailed 5% significance level to detect a difference in HbA1c of 0.4% between the 2 arms with three repeated measures based on a previous study. This estimation assumed a 20% loss to follow-up, a SD change of 1% in HbA1c, and an intraclass correlation coefficient of 0.6 within measurements from the same participant.
All enrolled participants who completed the baseline assessment were included in the intention-to-treat analysis . Descriptive analyses were conducted for baseline variables, and t tests or chi-square tests were performed (as appropriate) to determine any differences in baseline characteristics between the 2 study arms. For indicators related to program use, mean with SD and range were reported for key metrics. A chat was included in the analysis of chat duration only if the whole appointment module with Laura was completed during the chat (ie, valid values were available in the program data). Durations were truncated when the records showed extreme values (larger than two interquartile ranges above the third quartile of the distribution), as they were likely to represent cases when participants had forgotten to exit the app and duration had been tracked beyond the end of the actual chat.
The main effectiveness analysis followed the intention-to-treat principle . The analysis fitted a linear mixed-effect model for repeated measures over the 3 time points of data collection, with a random intercept for the participant and fixed effects for study arm, time point, and study arm-by-time interaction. Two sets of sensitivity analyses were performed: (1) adjusted model: adjusted baseline covariates that were identified as being imbalanced between the two arms or significantly associated with lost-to-follow-up and (2) per-protocol analysis: intervention participants who had completed ≥7 completed chats with Laura. Posthoc subgroup analyses were conducted based on age, sex, baseline HbA1c, and NDSS registration duration by adding the subgroup variable and its interaction term with the variable of the interaction between intervention and assessment point as fixed effects to the mixed-effect models used for the main analysis. To further investigate the relationship between program use and effectiveness, mixed-effect models were fitted with a random intercept for the participant and fixed effect for levels of program use (number of chats as 0-6, 7-24, and 25+), assessment time point, and program use-by-time interaction. All analyses were conducted in Stata version 15.1 (Stata Corp) by EG and verified by a statistician (ML) who was blinded to the randomization allocation.
Participant Flow and Baseline Characteristics
Of the 697 individuals with T2DM who expressed interest in participating in the study, 187 were recruited, including 62 (33%) from Victoria, 21 (16.5%) from New South Wales, and 21 (16.5%) from Queensland (). The sample included 78 (41.7%) women, with a mean age of 57 (SD 10) years, 76 (40.7%) had a bachelor’s degree or above, 88 (47.1%) worked full time, and 72 (38.5%) were registered with the NDSS within the 12 months before recruitment ( ). The baseline mean HbA1c was 7.3% (56 mmol/mol, SD 1.5%). Individuals who were excluded were more likely to be females, have relatively lower education levels, be unemployed, and use mobile phones less or do not have access to a smartphone ( ).
The intervention and control arms (n=93 and n=94, respectively) were generally comparable, except that those in the intervention arm were slightly younger (P=.04) and had lower depression scores (P=.004) at baseline. Among the participants, 116 (62.0%) provided complete data for all 3 assessments (baseline, 6, and 12 months). Incomplete assessments were due to active withdrawal, loss to follow-up, incomplete surveys, or refusal to complete a clinical measure with blood tests. Participants without HbA1c measures provided by GPs at 6 months or 12 months were more likely to have lower baseline HRQoL (P=.01) and higher anxiety symptom scores (P=.03). Participants who had an incomplete survey without a valid AQoL-8D score were younger (P=.02) and more likely to be allocated to the intervention arm (P=.007).
|Baseline characteristics||Waitlist (n=94)||Intervention (n=93)||Total (N=187)||P value|
|Agea, (years), mean (SD)||58.4 (10.5)||55.4 (9.7)||56.9 (10.2)||.04|
|Sex, n (%)||.12|
|Male||60 (63.8)||49 (52.7)||109 (58.3)|
|Female||34 (36.2)||44 (47.3)||78 (41.7)|
|Education, n (%)||.25|
|Secondary high school or lower||29 (30.8)||25 (26.9)||54 (18.8)|
|Technical apprenticeship or diploma||30 (31.9)||27 (29.0)||57 (30.5)|
|Bachelor’s degree||23 (24.5)||17 (18.3)||40 (21.4)|
|Postgraduate degree or higher||12 (12.8)||24 (25.8)||36 (19.3)|
|Employment, n (%)||.99|
|Full time||43 (45.7)||45 (48.4)||88 (47.1)|
|Part time or casual||16 (17.0)||14 (15.1)||30 (16.0)|
|Retired||21 (22.3)||21 (22.6)||42 (22.5)|
|Unemployed or others||14 (14.9)||13 (14.0)||27 (14.4)|
|English as a secondary language, n (%)||11 (11.7)||6 (6.5)||17 (9.1)||.21|
|Aboriginal or Torres Strait Islander origin, n (%)||0 (0.0)||4 (4.3)||4 (2.1)||.08|
|General app use: frequent (multiple times per day), n (%)b||67 (71.3)||69 (74.2)||136 (72.7)||.38|
|Health-related quality of life: Assessment of Quality of Life-8 Dimensions score, mean (SD)||0.7 (0.2)||0.7 (0.2)||0.7 (0.2)||.13|
|Depressive symptoms: HADS-Dc score, mean (SD)a||4.7 (3.3)||3.3 (3.4)||4.0 (3.4)||.004|
|Cases (or likely cases) of depression, n (%)||24 (25.5)||12 (12.9)||36 (19.3)||.03|
|Anxiety symptoms: HADS-Ad score, mean (SD)||5.6 (3.3)||5.4 (3.8)||5.5 (3.5)||.67|
|Cases (or likely cases) of anxiety, n (%)||27 (28.7)||28 (30.1)||55 (29.4)||.84|
|Diabetes-specific distress: PAIDe score, mean (SD)||30.5 (19.9)||29.2 (21.4)||29.9 (20.6)||.67|
|Severe diabetes distress (PAID score >40), n (%)||31 (33.0)||29 (31.2)||60 (32.1)||.79|
|Years registered with NDSSf, n (%)b||.51|
|≤1 year||34 (38.2)||38 (42.7)||72 (40.5)|
|1-5 years||27 (30.3)||23 (25.8)||50 (28.1)|
|6-10 years||28 (31.5)||28 (31.5)||56 (31.5)|
|Diabetes medications, n (%)|
|Diabetes medication(s) prescribed||83 (88.3)||80 (86.0)||163 (87.2)||.64|
|Insulin prescribed||16 (17.0)||15 (16.1)||31 (16.6)||.06|
|Taking medicines daily as recommended||71 (75.5)||74 (79.6)||145 (77.5)||.33|
|Smoking, n (%)||7 (7.5)||4 (4.3)||11 (5.9)||.16|
|Self-monitoring of blood glucose (>5 days in past 7 days), n (%)b||49 (52.1)||53 (57.0)||102 (54.5)||.49|
|Daily foot checks, n (%)||23 (24.5)||21 (22.6)||44 (23.5)||.87|
|Weight (kg), mean (SD)b||95.7 (19.0)||97.1 (22.5)||96.4 (20.8)||.65|
|Glycated hemoglobin (%), mean (SD)||7.3(1.6)||7.3(1.5)||7.3 (1.5)||.86|
|Total cholesterol (mmol/L), mean (SD)b||4.5 (1.3)||4.6 (1.4)||4.6 (1.3)||.54|
|Systolic blood pressure (mm Hg), mean (SD)b||130.4 (13.6)||131.1 (14.6)||130.7 (14.1)||.72|
|Diastolic blood pressure (mm Hg), mean (SD)||78.5 (9.3)||78.4 (9.4)||78.5 (9.3)||.94|
|Triglyceride (mmol/L), mean (SD)||2.0 (1.3)||1.8 (0.8)||1.9 (1.1)||.26|
|High cholesterol, n (%)||59 (62.8)||64 (68.8)||123 (65.8)||.37|
|Hypertension, n (%)||52 (55.3)||56 (60.2)||108 (57.8)||.45|
|Arthritis (rheumatoid, osteoarthritis, or other), n (%)b||34 (36.2)||22 (23.7)||56 (30.0)||.11|
|Depression/anxiety/nervous disorder, n (%)||26 (27.7)||26 (28.0)||52 (27.8)||.60|
|Heart diseases, n (%)||17 (18.1)||17 (18.3)||34 (18.2)||.60|
|Diabetes-related eye complications, n (%)||12 (12.8)||12 (12.9)||24 (12.8)||.60|
|Lung diseases, n (%)||11 (11.7)||13 (14.0)||24 (12.8)||.53|
|Diabetes-related neuropathy, n (%)||11 (11.7)||11 (11.8)||22 (11.8)||.60|
|Stomach, duodenal, or gastro-intestinal ulcer, n (%)||11 (11.7)||10 (10.8)||21 (11.2)||.59|
|Cancer, n (%)||6 (6.4)||8 (8.6)||14 (7.5)||.50|
|Stroke, n (%)||4 (4.3)||10 (10.8)||14 (7.5)||.14|
|Peripheral vascular diseases, n (%)||8 (8.5)||6 (6.5)||14 (7.5)||.53|
|Kidney disease, n (%)||5 (5.3)||6 (6.5)||11 (5.9)||.57|
|Health care service utilization|
|Had an appointment with a general practitioner or specialist in the past 12 months, n (%)||93 (98.9)||92 (98.9)||185 (98.9)||.32|
|Had an appointment with any other health professional (eg, dietician) in the past 12 months, n (%)||54 (57.4)||63 (67.7)||117 (62.6)||.12|
|Admitted to hospital in the past 6 months, n (%)||21 (22.3)||12 (12.9)||33 (17.6)||.10|
|Used any other hospital service over the past 6 months that did not involve an admission, n (%)||20 (21.3)||21 (22.6)||41 (21.9)||.80|
aSignificant difference observed between the intervention and control arms.
bSome missing values exist: general app use (n=4), years registered with NDSS (n=9), weight (n=8), systolic blood pressure (n=2), diastolic blood pressure (n=2), total cholesterol (n=11), triglyceride (n=13), medication adherence (n=24), self-monitoring of blood glucose (n=6), diagnosed comorbidities (n=1), and health care service utilization (n=1).
cHADS-D: Hospital Anxiety and Depression Scale-Depression score.
dHADS-A: Hospital Anxiety and Depression Scale-Anxiety score.
ePAID: Problem Areas in Diabetes scale.
fNDSS: National Diabetes Service Scheme.
Program Adoption and Use
shows the program adoption and use among the 93 participants in the intervention arm. During the 12-month program access, 92 participants (98.9%) completed at least one chat with Laura, and 83 participants (89.2%) uploaded their blood glucose levels at least once to the app. These 92 participants completed a total of 1942 chats over 12 months of program access and reached a mean of 243 (SD 212) min of chats per person. In general, the program use, including the number of chats and number of blood glucose uploads, reduced over time of the program access ( ). During the 12-month program access, participants who had at least one chat per month with Laura reduced from 87% in the first month (n=81) to 15% (n=14) in the 12th month, and participants who kept recording their blood glucose in the app dropped from 78% (n=73) to 23% (n=21). In addition, the system generated 297 clinical alerts (related to issues on glucose levels, foot ulcer management, and medication side effects) and 179 technical alerts (eg, glucose uploading failed or user did not interact with the app for 14 days) to the program coordinator. A total of 19 discussion topics were posted on the web-based discussion forum covering a broad range of topics such as stigma and social support, depression, and stress management to facilitate the discussion among participants in the intervention arm.
|Indicators of program adoption and use||Intervention arm (n=93)|
|Uptake of the MDCaapp, n (%)|
|Participants who had at least one “appointment” with “Laura” over 12 months||92 (99)|
|Participants who had uploaded glucose data into the MDC app||83 (89)|
|Number of completed chats with “Laura” over 12 months, n (%)|
|25 or more||30 (32)|
|Total number of chats completed over 12 months||1942|
|Total number of valid chats completed over 12 monthsb||1641|
|Individual-level MDC app usagec, mean (SD); range|
|Number of chats completed per person||21.8 (16.7); 1-65|
|Number of valid chats completed per personb||18.4 (15.0); 1-53|
|Duration of valid chats per person (in minutes), mean (SD); ranged|
|Total duration of chats||242.7 (212.3); 0-1050|
|Mean duration of each valid chat||13.4 (4.8); 3-26.8|
|Glucose data uploaded, mean (SD); range|
|Number of glucose level uploads per person||181.8 (192.1); 1-966|
|Program delivery,total; mean per month (SD)|
|Clinical alerts (eg, abnormal glucose level)||297; 13.7 (8.8)|
|Technical alerts (eg, glucose uploading failed)||179; 8.3 (6.5)|
|Posts on the web-based discussion forum||19; 1.1|
aMDC: My Diabetes Coach.
bInvalid chats were defined as chats for which participants exited the app before the coach modules were fully completed with the closing remark.
cFor individual-level information, the estimation is based on 92 participants who had records of chat with Laura through the app and 83 participants who had uploaded their glucose levels into the app. Mean (SD) and range of number and duration of chats and glucose data uploads were reported.
dOnly completed chats have been included in the calculation of the total duration of chats. If the users did not exit the app after completing the chats, the duration would be continuously counted. Thus, we truncated the values if the duration of the chats were more than two interquartile ranges above the third quartile of the distribution.
There was a statistically significant between-arm difference at 12 months in the mean change in HRQoL (AQoL-8D utility value: 0.04, 95% CI 0.00 to 0.07; P=.04) but not in HbA1c (−0.04, 95% CI −0.45 to 0.36; P=.83;). The MDC arm had a significant improvement in HRQoL from baseline to 12 months (mean estimated change of AQoL-8D score: 0.04, 95% CI 0.01 to 0.06; P=.007). This was maintained from the increase at 6 months (mean estimated change: 0.05, 95% CI 0.01 to 0.08; P=.006). HRQoL remained stable for participants in the control arm between baseline and 12 months (mean estimated change: −0.00, 95% CI −0.03 to 0.02; P=.92). The adjusted model showed consistent results. The per-protocol analysis yielded a slightly larger effect size, as the 12-month between-arm difference in the AQoL-8D score was 0.06 (95% CI 0.02 to 0.09; P=.003). Compared with baseline, the intervention resulted in an increase in the score of the physical health and mental health subscales ( ). There were significant between-arm differences in the mean change of mental health subscore at 6 months as well as the mean change of physical health subscore at 12 months.
|Coprimary outcomes and analysis models and Arms||Between arm differences at 6 months (95% CI)||P value||Between arm differences at 12 months (95% CI)||P value||Estimated mean changes between baseline and 6 months (95% CI)a||P value||Estimated mean changes between baseline and 12 months (95% CI)a||P value|
|Primary (Intention-to-treat) modelb|
|Intervention||0.06 (−0.35 to 0.47)||.78||−0.04 (−0.45 to 0.36)||.84||−0.20 (−0.49 to 0.09)||.17||−0.33 (−0.62 to −0.04)||.03|
|Control||Reference||N/Ac||Reference||N/A||−0.26 (−0.55 to 0.03)||.08||−0.28 (−0.57 to 0.00)||.05|
|Intervention||0.06 (−0.35 to 0.46)||.79||−0.04 (−0.44 to 0.37)||.87||−0.20 (−0.49 to 0.09)||.18||−0.32 (−0.61 to −0.03)||.03|
|Control||Reference||N/A||Reference||N/A||−0.25 (−0.54 to 0.03)||.09||−0.28 (−0.57 to 0.00)||.05|
|Intervention||−0.05 (−0.47 to 0.37)||.81||−0.14 (−0.56 to 0.28)||.52||−0.26 (−0.59 to 0.07)||.12||−0.40 (−0.73 to −0.06)||.02|
|Control||Reference||—||Reference||—||−0.21 (−0.47 to 0.05)||.11||−0.26 (−0.51 to −0.00)||.05|
|Health Related Quality of Life: Assessment of Quality of Life-8D utility score|
|Primary (Intention-to-treat) modelb|
|Intervention||0.05 (0.01 to 0.08)||.006||0.04 (0.00 to 0.07)||.039||0.04 (0.01 to 0.07)||.002||0.04 (0.01 to 0.06)||.007|
|Control||Reference||N/A||Reference||N/A||−0.01 (−0.03 to 0.02)||.48||0.00 (−0.03 to 0.02）||.92|
|Intervention||0.05 (0.01 to 0.08)||.005||0.03 (0.00 to 0.07)||.047||0.04 (0.01 to 0.06)||.002||0.03 (0.01 to 0.06)||.009|
|Control||Reference||N/A||Reference||N/A||−0.01 (−0.03 to 0.01)||.46||−0.00 (−0.02 to 0.02)||.93|
|Intervention||0.06 (0.02 to 0.09)||.002||0.06 (0.02 to 0.09)||.003||0.05 (0.02 to 0.08)||.001||0.05 (0.02 to 0.08)||.001|
|Control||Reference||N/A||Reference||N/A||−0.01 (−0.03 to 0.01)||.49||−0.01 (−0.03 to 0.02)||.63|
aMean changes in outcomes were estimated based on the linear mixed-effect regression model.
bFor HbA1c, the intraclass correlation coefficient (ICC) for the primary model was 0.551 (95% CI 0.465-0.634). For HRQoL, the ICC for the unadjusted model was 0.847 (95% CI 0.806-0.880). Number of participants with valid data at each time point: n for HbA1c (intervention vs control): 93 vs 94 at baseline, 78 vs 78 at 6 months, and 77 vs 79 at 12 months. Number of participants at each time point for HRQoL (intervention vs control): 93 vs 94 at baseline, 67 vs 77 at 6 months, and 60 vs 78 at 12 months.
cN/A: not applicable.
dThe adjusted model adjusted baseline values of variables that were either imbalanced by intervention allocation by chance (baseline age and depression score) or associated with loss to follow-up (baseline AQoL-8D utility value and HADS Anxiety score).
eThe per-protocol analysis considered participants who had completed more than 6 chats with Laura as following the study protocol.
Compared with baseline, the mean estimated HbA1c decreased in both arms at 12 months (intervention arm: mean estimated change: −0.33%, 95% CI −0.62 to −0.04; P=.03 and control arm: −0.28%, 95% CI −0.57 to 0.00; P=.05). The results were consistent in the adjusted model. In the per-protocol model, a larger between-arm difference was observed at 12 months (between-arm difference: −0.14%, 95% CI −0.56 to 0.28; P=.52).
There was a dose-response relationship between the number of chats and the change in the HRQoL score (). Compared with people who completed chats less than 7 times, those who completed more than 24 chats with Laura during program access had a significantly greater improvement in AQoL-8D utility score (between-arm difference: 0.09, 95% CI 0.02 to 0.17; P=.02) at 12 months. There was no significant difference in HbA1c (−0.33%, 95% CI −1.11 to 0.44; P=.40).
Although this study was not powered for subgroup analyses, the results did show some statistically significant differences favoring the intervention for HbA1c. There were greater between-arm differences at 6 and 12 months in the mean change in HbA1c for those with higher baseline HbA1c and those registered on the NDSS within the previous year (). There were no significant differences in HRQoL at either 6 or 12 months in the sex and age subgroups. The results for secondary outcomes are presented in . A significant between-arm difference (−0.89, 95% CI −1.74 to −0.04; P=.04) was observed in the mean change in the HADS anxiety score at 6 months but not at 12 months or for other secondary outcomes reported here.
This study is among the very few randomized controlled trials that have evaluated the adoption, use, and effectiveness of a mobile app–based, interactive, embodied conversational agent to support diabetes self-management within home settings over a 12-month period. Our study adds new evidence to this emerging field by demonstrating the program use in a home setting and the effectiveness of app-based interactive conversational agents in supporting diabetes self-management. The MDC program was feasible and shown to be effective in improving participants’ HRQoL. Although HbA1c levels reduced during the trial, the between-arm difference was not statistically significant at 6 or 12 months.
Despite the growing number of studies using mobile technologies for diabetes management [, , ], the effectiveness of introducing apps to people with T2DM to support their self-management at large scale remains uncertain given the generally poor quality of apps [ , , ], unmet consumer needs [ ], and studies lacking robust designs with long-term evaluations [ , ]. The MDC program is highly innovative with its conversational element. The MDC app incorporated interactive voice recognition and an embodied conversational agent, Laura, with human-like characteristics who used a very conversational style of speech to provide people with T2DM with personalized coaching and support on a range of essential diabetes self-management activities in their home environment. The process evaluation that received response from 66 out of 93 participants at 6 months showed that more than 80% of them considered Laura as a helpful, friendly and competent assistant and 72% described Laura as trustworthy [ ].
The indicators for program use suggest successful uptake among individuals with T2DM, but maintaining long-term program use still remains a challenge. Interestingly, program exposure (an average of 18 appointments with Laura) is very similar to the TLC diabetes program previously developed and evaluated by our team in Australia ; however, the program is more intensive than a human-delivered program that is widely available in Australia (a maximum of eight face-to-face group sessions per calendar year) [ ]. Some gamification elements and human-like characteristics were utilized in the MDC program design, as suggested by previous studies [ ]. The recently published and separate evaluation from MDC users demonstrated that these techniques did increase users’ engagement with the program but were insufficient to ensure engagement among all users [ ]. Program users have diverse preferences in terms of program interaction, which is likely determined by their technology literacy, personal preference, and their existing self-management style [ ]. The decreasing trend in program use over time and the dose-response relationship between the level of app use and its effectiveness suggest that more efforts are still required to improve the maintenance of program use over time.
We observed a modest but statistically significant improvement in HRQoL at 6 months, which was maintained at 12 months. This finding is consistent with some previous research, which indicates a small but statistically significant benefit from mobile app–based interventions on HRQoL . The improvement in AQoL utility score was relatively small in scale (0.04 increase from baseline to 12 months), but previous literature indicates that even a small effect size for HRQoL may bring substantial well-being benefit in the long term [ , , ]. In addition, the magnitude of HRQoL improvement is associated with the dose of program use among individuals who accessed the program; this demonstrates the importance of increasing program use in future studies. Such HRQoL improvement was contributed by the improvement in both mental health and physical health, as indicated by the subscale analysis of AQoL. The reduced anxiety symptoms, indicated by the significant between-arm difference in anxiety scores at 6 months, further confirmed the impact of the program on mental health. Although the intervention reduced HbA1c, it did not achieve a significant reduction compared with previous programs [ , ], and there are several likely reasons for this. First, the baseline HbA1c level was not an inclusion criterion, and 52.4% of participants had an HbA1c in the recommended target range (<7% or 53 mmol/mol) at baseline. However, the significant difference by baseline HbA1c observed in the subgroup analysis suggests that the intervention may have significant benefits if the study had targeted only those individuals with HbA1c above target. Second, similar to some previous digital health trials [ ], there was a reduction in HbA1c among participants in the control arm. This could be due to the Hawthorne effect with participants sufficiently motivated to be involved in the trial are likely to seek better self-management through other options when they were not allocated to the active intervention. Third, participants’ lack of maintenance in program use may have led to a relatively low dose of program exposure. The larger between-arm differences observed in the per-protocol analysis and the greater impact among people with higher program exposure indicates that the effect of the MDC program could have been larger with better fidelity to the study protocol.
This study has several important strengths. First, study participants were recruited from across Australia, with broad inclusion criteria, and the program was delivered within participants’ home settings. Thus, the sample is broadly representative of Australians with T2DM, and the study findings are likely to be generalizable and scalable. Second, the study followed participants over 12 months, which is a relatively long term compared with many studies of this kind . Third, we applied intention-to-treat and per-protocol analysis and performed further analysis of the relationship between program use and effectiveness. As a pragmatic behavioral trial, applying both analysis frameworks provides evidence on the effectiveness of the intervention, both for the ideal situation based on the protocol design and for the real-world situation with adaptation among users.
There were also some limitations. First, due to the nature of the intervention, we were not able to blind participants or their GPs (who provided clinical measurements) to the study arm allocation. Without being able to blind participants, self-report bias and Hawthorne effects may exist. Second, we observed a higher rate of completed assessments among participants in the control arm than in the intervention arm, possibly due to their interest in gaining program access or because of higher attrition in the intervention arm. Third, due to the relatively small sample size, the subgroup analyses should be interpreted with caution. Although the main analyses were fully powered, the subgroup analysis was underpowered and multiple testing would have increased the likelihood of false positives. Overall, the sample was not dissimilar from previous trials in this field .
To summarize, this study presents findings concerning the effectiveness, adoption, and use of the MDC program, an app-based interactive embodied conversational agent, in supporting individuals with T2DM. Participants had good adoption of the program and completed a significant amount of chats with Laura over 12 months. The trial demonstrated benefits for HRQoL among individuals with T2DM in Australia, which remained apparent at 12 months. The lack of effect for HbA1c is likely due to the relatively low baseline HbA1c level and declining program use over time. Strategies that would increase the program engagement and maintenance in use and improve the effect on HbA1c are required before program scale-up.
The authors thank the study participants for volunteering their time and experience. The authors would also like to acknowledge the following members of the research team and thank them for their contributions: Jillian Zemanek, Fiona Cocker, Anna Scovelle, Suman Shetty, Mandy Cassimatis, Robin Zhou, Phillipa Dalach, Shuai Shao, Ameera Katar (all University of Melbourne), Trish Roderick (Diabetes Queensland), Carolyn Hines (Diabetes Victoria), Deborah Schofield (Diabetes WA), Ornella Care, Judith Ngai, and Audra Millis (Bupa Australia). The authors also appreciate the contributions from Dr Pilvikki Absetz for providing suggestions for revising this manuscript and the support from Diabetes Australia (NDSS) in assisting with participant recruitment. The authors also would like to acknowledge other researchers as part of the My Diabetes Coach Group who contributed to the study, including Professor Mark Harris, Professor Robert H Friedman, Ms Taryn Black, Ms Margaret Brand, Associate Professor Louisa Gordon, Ms Carolyn Hynes, Professor Greg Johnson, Mr Michael Skinner, Dr Marlien Varnfield, Mr Andrew Wagstaff, and Professor Rory Wolfe.
This research was supported by a National Health and Medical Research Council (NHMRC) Partnership Grant (ID1057411), with additional financial and in-kind support provided by Diabetes Australia, Diabetes Queensland, Diabetes Victoria, Diabetes WA, and Roche Diabetes Care. Development of the MDC app was made possible with generous financial and other support from Bupa Australia and the Bupa Foundation and collaboration with Clevertar. EG is supported by the Melbourne Graduate Research Scholarship. JS is supported by core funding from the Australian Centre for Behavioural Research in Diabetes provided by the collaboration between Diabetes Victoria and Deakin University. EG, PS, and BO are supported by the NHMRC Centre of Research Excellence in Digital Technology to Transform Chronic Disease Outcomes (ID 1170937).
Conflicts of Interest
BO and DB received some royalty payments for the development of the scripts for the MDC platform.
Introduction about My Diabetes Coach Program and its five components.DOCX File , 2729 KB
Supplementary tables and figures.DOCX File , 229 KB
CONSORT-eHEALTH checklist (V1.6.1).PDF File (Adobe PDF File), 1512 KB
- IDF Diabetes Atlaas. International Diabetes Federation. 2019. URL: https://www.diabetesatlas.org/en/resources/ [accessed 2020-06-30]
- Edelman SV, Polonsky WH. Type 2 diabetes in the real world: the elusive nature of glycemic control. Diabetes Care 2017 Nov;40(11):1425-1432. [CrossRef] [Medline]
- Chatterjee S, Davies MJ, Heller S, Speight J, Snoek FJ, Khunti K. Diabetes structured self-management education programmes: a narrative review and current innovations. Lancet Diabetes Endocrinol 2018 Feb;6(2):130-142. [CrossRef] [Medline]
- Lindner H, Menzies D, Kelly J, Taylor S, Shearer M. Coaching for behaviour change in chronic disease: a review of the literature and the implications for coaching as a self-management intervention. Aust J Prim Health 2003;9(3):177. [CrossRef]
- Pirbaglou M, Katz J, Motamed M, Pludwinski S, Walker K, Ritvo P. Personal health coaching as a type 2 diabetes mellitus self-management strategy: a systematic review and meta-analysis of randomized controlled trials. Am J Health Promot 2018 Sep;32(7):1613-1626. [CrossRef] [Medline]
- Oldenburg B, Taylor CB, O'Neil A, Cocker F, Cameron LD. Using new technologies to improve the prevention and management of chronic conditions in populations. Annu Rev Public Health 2015 Mar 18;36:483-505. [CrossRef] [Medline]
- Wolever R, Dreusicke M, Fikkan J, Hawkins T, Yeung S, Wakefield J, et al. Integrative health coaching for patients with type 2 diabetes: a randomized clinical trial. Diabetes Educ 2010;36(4):629-639. [CrossRef] [Medline]
- Williams ED, Bird D, Forbes AW, Russell A, Ash S, Friedman R, et al. Randomised controlled trial of an automated, interactive telephone intervention (TLC Diabetes) to improve type 2 diabetes management: baseline findings and six-month outcomes. BMC Public Health 2012 Aug 3;12:602 [FREE Full text] [CrossRef] [Medline]
- Gordon L, Bird D, Oldenburg B, Friedman R, Russell A, Scuffham P. A cost-effectiveness analysis of a telephone-linked care intervention for individuals with type 2 diabetes. Diabetes Res Clin Pract 2014 Apr;104(1):103-111. [CrossRef] [Medline]
- Aminuddin H, Jiao N, Jiang Y, Hong J, Wang W. Effectiveness of smartphone-based self-management interventions on self-efficacy, self-care activities, health-related quality of life and clinical outcomes in patients with type 2 diabetes: a systematic review and meta-analysis. Int J Nurs Stud 2019 Feb 8:103286. [CrossRef] [Medline]
- Hou C, Xu Q, Diao S, Hewitt J, Li J, Carter B. Mobile phone applications and self-management of diabetes: a systematic review with meta-analysis, meta-regression of 21 randomized trials and GRADE. Diabetes Obes Metab 2018 Aug;20(8):2009-2013. [CrossRef] [Medline]
- Hood M, Wilson R, Corsica J, Bradley L, Chirinos D, Vivo A. What do we know about mobile applications for diabetes self-management? A review of reviews. J Behav Med 2016 Dec;39(6):981-994. [CrossRef] [Medline]
- Gong E, Zhang Z, Jin X, Liu Y, Zhong L, Wu Y, et al. Quality, functionality, and features of chinese mobile apps for diabetes self-management: systematic search and evaluation of mobile apps. JMIR Mhealth Uhealth 2020 Apr 7;8(4):e14836 [FREE Full text] [CrossRef] [Medline]
- Baptista S, Trawley S, Pouwer F, Oldenburg B, Wadley G, Speight J. What do adults with type 2 diabetes want from the 'perfect' app? Results from the second diabetes miles: Australia (MILES-2) study. Diabetes Technol Ther 2019 Jul;21(7):393-399. [CrossRef] [Medline]
- Wu X, Guo X, Zhang Z. The efficacy of mobile phone apps for lifestyle modification in diabetes: systematic review and meta-analysis. JMIR Mhealth Uhealth 2019 Jan 15;7(1):e12297 [FREE Full text] [CrossRef] [Medline]
- Garabedian LF, Ross-Degnan D, LeCates RF, Wharam JF. Uptake and use of a diabetes management program with a mobile glucometer. Prim Care Diabetes 2019 Dec;13(6):549-555. [CrossRef] [Medline]
- Shen Y, Wang F, Zhang X, Zhu X, Sun Q, Fisher E, et al. Effectiveness of internet-based interventions on glycemic control in patients with type 2 diabetes: meta-analysis of randomized controlled trials. J Med Internet Res 2018 May 7;20(5):e172 [FREE Full text] [CrossRef] [Medline]
- Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C. Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Med Care 2012 Mar;50(3):217-226 [FREE Full text] [CrossRef] [Medline]
- Australian Diabetes Map. National Diabetes Service Scheme. 2019. URL: https://www.ndss.com.au/about-the-ndss/diabetes-facts-and-figures/australian-diabetes-map/ [accessed 2020-06-30]
- Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)--a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 2009 Apr;42(2):377-381 [FREE Full text] [CrossRef] [Medline]
- Prochaska J, Redding C, Evers K. The transtheoretical model and stages of change. In: Health Behavior: Theory, Research, and Practice. San Francisco, CA: Jossey-Bass; 2015:125-148.
- Crosby R, Salazar L, DiCelemente R. Social cognitive theory applied to health behavior. In: Health Behavior Theory for Public Health: Principles, Foundations, and Applications. Burlington, UK: Jones & Bartlett Learning; 2013:163-186.
- Cugelman B. Gamification: what it is and why it matters to digital health behavior change developers. JMIR Serious Games 2013 Dec 12;1(1):e3 [FREE Full text] [CrossRef] [Medline]
- Grady PA, Gough LL. Self-management: a comprehensive approach to management of chronic conditions. Am J Public Health 2014 Aug;104(8):e25-e31. [CrossRef] [Medline]
- Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health 1999 Sep;89(9):1322-1327. [CrossRef] [Medline]
- Richardson J, Iezzi A, Khan MA, Maxwell A. Validity and reliability of the assessment of quality of life (AQoL)-8D multi-attribute utility instrument. Patient 2014;7(1):85-96 [FREE Full text] [CrossRef] [Medline]
- Hawthorne G, Osborne R. Population norms and meaningful differences for the assessment of quality of life (AQoL) measure. Aust N Z J Public Health 2005 Apr;29(2):136-142. [CrossRef] [Medline]
- Zigmond AS, Snaith RP. The hospital anxiety and depression scale. Acta Psychiatr Scand 1983 Jun;67(6):361-370. [CrossRef] [Medline]
- Welch GW, Jacobson AM, Polonsky WH. The problem areas in diabetes scale. An evaluation of its clinical utility. Diabetes Care 1997 May;20(5):760-766. [CrossRef] [Medline]
- Cui M, Wu X, Mao J, Wang X, Nie M. T2DM self-management via smartphone applications: a systematic review and meta-analysis. PLoS One 2016;11(11):e0166718 [FREE Full text] [CrossRef] [Medline]
- Newell DJ. Intention-to-treat analysis: implications for quantitative and qualitative research. Int J Epidemiol 1992 Oct;21(5):837-841. [CrossRef] [Medline]
- Brzan PP, Rotman E, Pajnkihar M, Klanjsek P. Mobile applications for control and self management of diabetes: a systematic review. J Med Syst 2016 Sep;40(9):210. [CrossRef] [Medline]
- Baptista S, Wadley G, Bird D, Oldenburg B, Speight J, My Diabetes Coach Research Group. Acceptability of an embodied conversational agent for type 2 diabetes self-management education and support via a smartphone app: mixed methods study. JMIR Mhealth Uhealth 2020 Jul 22;8(7):e17038 [FREE Full text] [CrossRef] [Medline]
- Bird D, Oldenburg B, Cassimatis M, Russell A, Ash S, Courtney MD, et al. Randomised controlled trial of an automated, interactive telephone intervention to improve type 2 diabetes self-management (telephone-linked care diabetes project): study protocol. BMC Public Health 2010 Oct 12;10:599 [FREE Full text] [CrossRef] [Medline]
- Group Allied Health Services Under Medicare For Patients With Type 2 Diabetes Patient Information. Australian Government Department of Health. 2013. URL: https://www1.health.gov.au/internet/main/publishing.nsf/Content/D89B8F9BD812E58CCA257BF000209C70/$File/Group%20Allied%20Health%20Services-Type%202%20Diabetes-Patient%20Info.pdf [accessed 2020-06-30]
- Priesterroth L, Grammes J, Holtz K, Reinwarth A, Kubiak T. Gamification and behavior change techniques in diabetes self-management apps. J Diabetes Sci Technol 2019 Sep;13(5):954-958 [FREE Full text] [CrossRef] [Medline]
- Baptista S, Wadley G, Bird D, Oldenburg B, Speight J, My Diabetes Coach Research Group. User experiences with a type 2 diabetes coaching app: qualitative study. JMIR Diabetes 2020 Jul 17;5(3):e16692 [FREE Full text] [CrossRef] [Medline]
- Cochran J, Conn VS. Meta-analysis of quality of life outcomes following diabetes self-management training. Diabetes Educ 2008;34(5):815-823 [FREE Full text] [CrossRef] [Medline]
- Captieux M, Pearce G, Parke HL, Epiphaniou E, Wild S, Taylor SJ, et al. Supported self-management for people with type 2 diabetes: a meta-review of quantitative systematic reviews. BMJ Open 2018 Dec 14;8(12):e024262. [CrossRef] [Medline]
- Karhula T, Vuorinen A, Rääpysjärvi K, Pakanen M, Itkonen P, Tepponen M, et al. Telemonitoring and mobile phone-based health coaching among Finnish diabetic and heart disease patients: randomized controlled trial. J Med Internet Res 2015 Jun 17;17(6):e153 [FREE Full text] [CrossRef] [Medline]
|AQoL-8D: Assessment of Quality of Life-8 Dimensions|
|CONSORT: Consolidated Standards of Reporting Trials|
|GP: general practitioner|
|HADS: Hospital Anxiety and Depression Scale|
|HbA1c: glycated hemoglobin|
|HRQoL: health-related quality of life|
|MDC: My Diabetes Coach|
|NDSS: National Diabetes Service Scheme|
|NHMRC: National Health and Medical Research Council|
|T2DM: type 2 diabetes mellitus|
|TLC diabetes: Telephone-Linked Care for Diabetes|
Edited by G Eysenbach; submitted 17.05.20; peer-reviewed by C Hou, Y Zhang; comments to author 24.06.20; revised version received 21.07.20; accepted 03.08.20; published 05.11.20
©Enying Gong, Shaira Baptista, Anthony Russell, Paul Scuffham, Michaela Riddell, Jane Speight, Dominique Bird, Emily Williams, Mojtaba Lotfaliany, Brian Oldenburg. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 05.11.2020.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.