Published on 14.02.20 in Vol 22, No 2 (2020): February
Preprints (earlier versions) of this paper are available at http://preprints.jmir.org/preprint/15424, first published Aug 12, 2019.
Feasibility of an Electronic Health Tool to Promote Physical Activity in Primary Care: Pilot Cluster Randomized Controlled Trial
Background: Physical inactivity is associated with increased health risks. Primary care providers (PCPs) are well positioned to support increased physical activity (PA) levels through screening and provision of PA prescriptions. However, PCP counseling on PA is not common.
Objective: This study aimed to assess the feasibility of implementing an electronic health (eHealth) tool to support PA counseling by PCPs and estimate intervention effectiveness on patients’ PA levels.
Methods: A pragmatic pilot study was conducted using a stepped wedge cluster randomized trial design. The study was conducted at a single primary care clinic, with 4 pre-existing PCP teams. Adult patients who had a periodic health review (PHR) scheduled during the study period were invited to participate. The eHealth tool involved an electronic survey sent to participants before their PHR via an email or a tablet; data were used to automatically produce tailored resources and a PA prescription in the electronic medical record of participants in the intervention arm. Participants assigned to the control arm received usual care from their PCP. Feasibility was assessed by the proportion of completed surveys and patient-reported acceptability and fidelity measures. The primary effectiveness outcome was patient-reported PA at 4 months post-PHR, measured as metabolic equivalent of task (MET) minutes per week. Secondary outcomes assessed determinants of PA, including self-efficacy and intention to change based on the Health Action Process Approach behavior change theory.
Results: A total of 1028 patients receiving care from 34 PCPs were invited to participate and 530 (51.55%) consented (intervention [n=296] and control [n=234]). Of the participants who completed a process evaluation, almost half (88/178, 49.4%) stated they received a PA prescription, with only 42 receiving the full intervention including tailored resources from their PCP. A cluster-level linear regression analysis yielded a non–statistically significant positive difference in MET-minutes reported per week at follow-up between intervention and control conditions (mean difference 1027; 95% CI −155 to 2209; P=.09). No statistically significant differences were observed for secondary outcomes.
Conclusions: Our results suggest that it is feasible to build an eHealth tool that screens and provides tailored resources for PA in a primary care setting but suboptimal intervention fidelity suggests greater work must be done to address PCP barriers to resource distribution. Participant responses to the primary effectiveness outcome (MET-minutes) were highly variable, reflecting a need for more robust measures of PA in future trials to address limitations in patient-reported data.
Trial Registration: ClinicalTrials.gov NCT03181295; https://clinicaltrials.gov/ct2/show/NCT03181295
J Med Internet Res 2020;22(2):e15424
Physical inactivity is the fourth leading risk factor for global morbidity and mortality, responsible for 6% of deaths annually . The Canadian Physical Activity Guidelines recommend at least 150 min of moderate-to-vigorous activity per week for adults aged 18 to 64 years [ ]. In those who achieve recommended levels of physical activity (PA), all-cause mortality is decreased by 19% to 30% [ , ], with a dose response identified [ , ]. Despite this evidence, it is estimated that only 16% of adults aged 18 to 79 years in Canada meet current PA guidelines [ ].
Primary care physicians (PCPs) are ideally positioned to positively affect levels of PA among their patients . Multiple clinical guidelines recommend PCPs screen patients for current activity levels and offer targeted counseling during routine visits [ - ]. Evidence indicates that a tailored PA prescription from PCPs can improve overall activity levels [ - ]. Unfortunately, this is rarely implemented in real-world clinical practice [ - ], with reported barriers including lack of time, knowledge, and training in PA counseling and a perceived inability to change patient behavior [ , ].
Electronic screening of health behaviors can save time for PCPs and has been highly accepted by patients as a method to share information with their care team [- ], and there is evidence supporting the use of digital health tools to improve PA [ ]. Furthermore, using computers to deliver tailored messaging and resources to patients can have a positive impact on behavior change, including PA, relative to more traditional methods of health counseling [ - ]. Integrating screening and tailored information provision into one intervention may help change PA levels by addressing the complex needs of both providers and patients [ , ].
In this pilot study, we tested the feasibility of implementing an electronic health (eHealth) tool to support PA counseling in routine primary care and produced a preliminary estimate of intervention effectiveness on changing PA levels. Our aim was to optimize the intervention, evaluate recruitment and retention of participants, and assess suitability of the primary outcome for a subsequent, larger definitive trial [, ].
This pilot study has been reported in accordance with extensions to the Consolidated Standards of Reporting Trials 2010 statement for both randomized pilot studies  and stepped wedge cluster randomized trials (SW-CRT) [ ] and the standards for reporting implementation studies statement [ ]. Research ethics approval was obtained from the Women's College Hospital Research Ethics Board (registered on ClinicalTrials.gov as NCT03181295).
We conducted a pilot study using a pragmatic SW-CRT design to identify potential issues with implementation or analysis that might challenge the feasibility of future trials involving more clusters . PCP teams functioning as naturally occurring clusters of clinicians and patients were randomized to allow gradual implementation of the tool and prevent intervention contamination across clusters [ - ]. The study was divided into 5 periods, each one 6 weeks in length. Initially, no teams were exposed to the intervention [ ], then 1 team was randomly assigned to begin the intervention at the start of each period [ ]. Randomization occurred using computer-generated random numbers produced by an independent statistician [ ]. Participants and researchers could not be blinded due to the nature of the intervention.
The study was conducted at the Women’s College Hospital Family Practice Health Centre (FPHC), an academic, multidisciplinary family health team located in Toronto, Ontario, Canada, between February 20, 2017, and March 17, 2018. The FPHC has 39 PCPs and over 50,000 clinical visits per annum. The FPHC is divided into 4 teams for operational convenience with minimal clinician or patient crossover among teams.
Participants and Recruitment
All PCPs (N=39) at the FPHC were eligible to participate in the study, exempting learners and PCPs who were not expected to be present for the entire study period. Patients rostered to a participating PCP were eligible if they attended a periodic health review (PHR) during the study period and were aged 18 to 79 years at the time of the PHR. PHRs were considered appropriate opportunities to deliver the intervention, as they focus on preventative care counseling . Patients deemed unable to safely or effectively complete the intervention at the time of their PHR were excluded. This included those with dementia or cognitive impairment, with major active illness, or who were pregnant. Non-English speakers were also excluded because of a lack of resources to appropriately accommodate other languages. A research assistant was responsible for regularly reviewing the FPHC schedule and assessing potential eligibility via electronic medical record (EMR) review. The PCP confirmed eligibility when the research assistant was uncertain.
The Health Action Process Approach (HAPA) is a theory of behavior change used to inform the design of successful behavior change interventions, including those targeting PA . It aligns with factors such as goal-setting, which has been shown to improve PA in some digital health interventions [ ]. In general, HAPA suggests that individuals who have not yet developed an intention to change behavior (preintenders) may benefit from interventions that target risk perception and outcome expectancy. Those who have developed an intention to modify behaviors but have not yet shown change (intenders) may benefit from interventions that target action planning and coping planning. Those who have achieved certain health behaviors (actors) may benefit from interventions focused on relapse prevention. Movement through these phases is fluid, affected by social support and/or contextual barriers, and is mediated by self-efficacy in action, maintenance, and recovery [ ]. This approach was used to categorize participants, customize intervention materials for each participant, and analyze outcomes as described further on.
The intervention was refined using principles of user-centered design. This approach emphasizes the use of iterative product design with ongoing feedback from the end user to drive improvements and optimize the acceptance and use of the tool [- ]. This involved multiple interviews with potential end users, as described in another paper [ ].
Treatment Group: Intervention
All patients deemed eligible for the study received an email 2 weeks before their visit with a link to a secure electronic survey (e-survey). Those who did not complete the survey before their appointment were approached in the clinic, and the e-survey was completed using a digital tablet in the waiting room. The e-survey collected informed consent, assessed baseline PA, and assessed perceived barriers and motivators for PA.
The intervention included 3 key components that were automatically generated based on the baseline survey. First, responses were summarized in the patient’s EMR along with a statement comparing the results with current PA guidelines of 150 min of moderate-to-vigorous PA per week [, , ]. Second, the EMR was populated with a link to 1 out of 5 toolkits that included online and community-based resources tailored to the patient’s current PA levels and perceived barriers, and an additional condition-specific PA toolkit if the patient reported any other condition (eg, cardiovascular disease). Third, a customized PA prescription was generated based on current PA levels and patient-identified motivators to increase PA. During the PHR, the prescription could be edited by the PCP based on discussions with the patient and then printed along with the toolkit for the patient to take home. Each patient’s toolkit was also sent to them 2 weeks after the PHR via mail or email. A full description and examples of the prescription and toolkit can be found in [ ].
To encourage intervention fidelity, one of the principal investigators (PA or NI) spoke with each of their PCP colleagues for 5 to 15 min before their cluster switching to the intervention arm. The intervention, including EMR outputs, was demonstrated using a test patient chart in the EMR, and then a handout was reviewed, which addressed both workflow integration and evidence for PA counseling (seefor the handout).
Control Group: Usual Care
Participants in the usual care group completed the same baseline questionnaire as the intervention group, but no EMR outputs or patient toolkits were produced. Participating PCPs were encouraged to provide PA advice (or not) as per their normal routines, for example, no attempt was made to standardize usual care. PCPs received education about the intervention only in the week before the intervention being activated for their team.
Outcomes and Data Collection
After exposure to the intervention, every intervention participant received a paper survey immediately after their appointment, or an e-survey 1 day following their appointment if they did not complete the paper version, to collect process measures (see). Acceptability was measured using a 5-point Likert scale ranging from very dissatisfied to very satisfied. Participants were also asked about the number of min of PA counseling they received (no discussion, less than 2 min, 2-5 min, 5-10 min, or more than 10 min) and if they received a PA prescription (yes/no). Feasibility was also assessed in part by the number of eligible patients who completed a baseline survey and the frequency of missing or inaccurate data [ ].
The primary effectiveness outcome was patient-reported PA at 4 months post-PHR, measured as metabolic equivalent of task (MET) minutes per week using the international physical activity questionnaire-short form (IPAQ-SF) . The IPAQ-SF was selected for its short length, ease of administration, good test-retest reliability, and low cost [ ].
Secondary outcomes, also collected 4 months post-PHR, assessed attitudes toward PA using the HAPA constructs to guide the assessment of proximal changes . Specifically, 3 subdimensions of self-efficacy (action, recovery, and maintenance) were assessed, each measured via 2 questions (using a 4-point Likert scale ranging from strongly disagree to strongly agree) [ - ]. A score for each subdimension of self-efficacy was calculated by summing the 2 questions, dividing by the maximum possible score and multiplying by 100 (for self-efficacy scores ranging from 0-100). The total self-efficacy score was the average of all subdimension scores.
Participants’ intention regarding PA was measured in a 2-step process. Those meeting recommended PA guidelines of 150 min of moderate to vigorous activity a week were defined as actors . Participants not meeting the recommended guidelines were defined as nonactors. This group was further subdivided into intenders and preintenders. Those who agreed with the statement, “I have made the decision to take part in a new kind of physical activity or increase my amount or intensity of physical activity soon” were deemed to be intenders, while those who disagreed were labeled preintenders. An e-survey collected responses for both primary and secondary outcomes and data were securely transferred and collated into a single, study-specific database (see for the survey).
Analysis of the pilot data was mainly descriptive . The distribution of patient- and PCP-level baseline characteristics were summarized by team via means and standard deviations (median and interquartile range when skewed) and frequencies and proportions, respectively [ , ].
To understand the feasibility of our study protocol, the frequency and proportion of patients that were assessed as eligible, recruited, randomized, and who had responded to both baseline and follow-up surveys were independently summarized. Additionally, responses to the process evaluation survey were summarized by patients exposed to the intervention—via counts and proportions for categorical responses and means and standard deviations for continuous responses—to elucidate patient satisfaction with their PA counseling and patient-reported impressions of PCP acceptability and adherence to the intervention. We assessed significant differences between those who received and those who did not receive the intervention using a chi-square test of independence.
The presence of few clusters in our study limits options for estimating the preliminary effectiveness of the intervention. Specifically, it precludes the use of conventional analytic approaches for stepped wedge trials [, , ] that model patient-level responses while accounting for clustering via random effects, which require observations on many clusters to yield unbiased estimates and accurate standard errors [ , ]. Correspondingly, patient-level responses to the primary outcome (measured in MET-minutes per week) were aggregated to the cluster-period level, removing the need to adjust for patient- or PCP-level characteristics or clustering of patient responses within teams [ ]. To obtain a preliminary estimate of intervention effectiveness on the primary outcome, the cluster-period mean response was then regressed as the outcome using linear regression with intervention exposure as the primary independent variable and the following fixed effects included as covariates: team (cluster), period, and mean baseline (or pretest) value [ ]. To assess the robustness of our findings to statistical outliers, a sensitivity analysis was conducted involving regression analysis as specified for the primary outcome; however, before aggregating patient-level responses to the cluster-period level, those patient responses in the top 5% by primary outcome value were excluded.
Secondary outcomes were analyzed similarly. Preliminary treatment effect estimates on each self-efficacy measure (action, recovery, maintenance, and overall) were obtained using multivariable linear regression with the unit of analysis as the cluster-period and adjustment for team, period, and baseline response as covariates. With regard to intention to change PA levels, the proportion of participants meeting criteria as an actor or intender at follow-up was calculated per cluster-period and expressed as a percentage. This value was then regressed as the outcome in a negative binomial regression model with intervention exposure as the primary independent variable, adjusting for period, team, and for the proportion meeting the outcome at baseline. A similar model focused only on the proportion of participants meeting criteria as an actor.
For each primary and secondary outcome, the analysis was limited to patients who were randomized, attended their PHR, and provided baseline and follow-up data for that outcome. Statistical significance was assessed, where applicable, using a two-sided P value of .05. SAS 9.4 (SAS Institute) was used to perform all analyses.
Sample Size Calculation
Following standard calculations for stepped wedge trials , assuming an average of 30 patients per cluster period for 5 periods total (4 steps), an estimated intracluster coefficient of 0.05 and cluster autocorrelation of 0.8, significance level at 5%, and assuming a standard deviation of 300 MET-min and a conservative loss to follow-up of 20%, we would have 80% power to observe a mean difference (MD) of 150 MET-min (over the past week) between the intervention and control condition [ , ]. This corresponds to recruiting a total of 440 patients across all periods. To maximize our ability to recruit the necessary amount in each period, the time interval for each period was set at 6 weeks.
Participants and Recruitment
In total, 34 out of 39 eligible PCPs participated across the 4 teams in the clinic. Of 1277 eligible patients, 1028 were invited to participate and 948 consented. Randomization proceeded based on cluster allocation (see). In total, 46.3% (296/640) and 60.3% (234/388) individuals randomized to the intervention and control groups, respectively, completed the baseline survey and received their allocated treatment. Most participants (307/530, 57.9%) completed the baseline survey via email before their PHR; 15.1% (80/530) participants completed the survey via a tablet (because no email address was informed in their EMR) and 27.0% (143/530) completed the survey via a tablet (after being sent the survey via an email).
describes baseline patient- and PCP-level characteristics within the 4 clusters. In terms of PCP characteristics, teams 1 and 4 were largely composed of female PCPs, and team 4 had a substantially lower median number of years since graduation. Most patient-level characteristics were balanced among teams, including baseline PA levels; patients in teams 2 and 4 reported a greater number of barriers to PA and a larger proportion of patients in team 1 were deemed actors at baseline, particularly when compared with patients in team 3 or 4. Maintenance self-efficacy was the measure with the highest degree of missingness (50/530, 9.4% missing).
|Female, n (%)||8 (100)||6 (66)||5 (62)||8 (88)||27 (79)|
|Years since graduation, median (Q1, Q3)||18 (14, 28)||26 (0, 32)||30 (7, 30)||8 (2, 16)||15 (4, 30)|
|Patients with baseline data, n||131||143||118||138||530|
|Age (years), mean (SD)||52 (13.3)||50 (14.8)||52 (12.7)||53 (12.8)||52 (13.4)|
|Female, n (%)||110 (84.0)||95 (66.4)||80 (67.8)||122 (88.4)||407 (76.8)|
|Total METb-min, median (Q1, Q3)||2502 (1453, 5028)||2866 (1499, 5172)||2974 (1273, 4973)||2601 (1634, 4986)||2768 (1453, 5028)|
|Total MET-min, mean (SD)||3719 (3466)||4242 (4586)||3811 (3818)||3672 (3293)||3868 (3832)|
|Cardiovascular disease, n (%)||17 (13.0)||19 (13.3)||15 (12.7)||24 (17.4)||75 (14.2)|
|Respiratory disease, n (%)||7 (5.3)||9 (6.3)||6 (5.1)||13 (9.4)||35 (6.6)|
|Diabetes, n (%)||3 (2.3)||7 (4.9)||3 (2.5)||9 (6.5)||22 (4.2)|
|Mental health issues, n (%)||25 (19.1)||26 (18.2)||19 (15.1)||19 (13.8)||89 (16.8)|
|Musculoskeletal disorder, n (%)||26 (19.9)||32 (22.4)||28 (23.7)||38 (27.5)||124 (23.4)|
|Neurological disorder, n (%)||1 (0.7)||3 (2.1)||3 (2.5)||2 (1.5)||9 (1.7)|
|Cancer, n (%)||7 (5.3)||8 (5.6)||8 (6.8)||3 (2.2)||26 (4.9)|
|No history of above diseases, n (%)||65 (49.6)||69 (48.3)||56 (47.5)||62 (44.9)||252 (47.5)|
|Number of motivators, mean (SD)||5.69 (2.30)||5.50 (2.41)||6.02 (2.18)||5.36 (2.44)||5.62 (2.38)|
|Number of barriers, mean (SD)||0.94 (1.23)||1.14 (1.32)||0.88 (1.16)||1.28 (1.28)||1.07 (1.26)|
|Behavior change category based on HAPAc theory of behavior change|
|Actor, n (%)||80 (61.5)||69 (48.6)||32 (27.1)||35 (25.4)||216 (40.8)|
|Intender, n (%)||32 (24.6)||52 (36.6)||57 (48.3)||61 (44.2)||202 (38.1)|
|Preintender, n (%)||18 (13.9)||21 (14.8)||29 (24.6)||42 (30.4)||110 (20.8)|
|Missing, n||1||1||0||0||2 (0.4)|
|Task self-efficacy, mean (SD)||79.3 (16)||79.3 (15)||79.2 (14.2)||79.8 (17.9)||79.4 (15.8)|
|Maintenance self-efficacy, mean (SD)||80.8 (15.3)||80.3 (14.3)||80.7 (15.5)||81.8 (15.4)||80.9 (15.1)|
|Recovery self-efficacy, mean (SD)||82.3 (13.9)||82.1 (13.9)||83 (13)||81.3 (15.1)||82.1 (14)|
|Overall self-efficacy, mean (SD)||80.7 (13.2)||80.4 (12.7)||80.8 (12.3)||80.7 (14.6)||80.7 (13.2)|
aPCP: primary care provider.
bMET: metabolic equivalent of task.
cHAPA: Health Action Process Approach.
In total, 61.8% (183/296) patients exposed to the intervention handed in a process evaluation survey following their PHR, with 63.2% (112/183) completing fully. Overall, fewer than half of respondents (88/178, 49.4%) stated they received at least a PA prescription from their PCP. A chi-square test of independence indicated no significant difference in the proportion of patients who received at least a PA prescription versus no materials between teams (χ23=3.0; P=.39). Among the 88 patients who received a PA prescription, just under half (42/88, 47%) also received tailored resources to take home. The proportion of intervention patients who completed the process evaluation receiving both a PA prescription and resources ranged from a low of 9% (6/64 patients) for team 1 to a high of 45% (15/33 patients) for team 3.
Only 6.6% (12/183) patients completing a process evaluation indicated that no PA discussion occurred during their appointment. Nearly half (86/176, 48.9%) of the participants who estimated the length of their PA discussion reported a length of 2 to 5 min, and patients in team 4 were more likely to report a talk of less than 2 min. Most patients reported being satisfied with their PA discussion irrespective of team, with no patients indicating they were dissatisfied. Of the process evaluation questions, patient satisfaction with their PA counseling (if applicable) was most prone to missing responses, with only 62.3% (114/183) providing a response. Seefor a full summary of process evaluation results.
Preliminary Effectiveness Outcomes
The primary outcome (ie, total MET-minutes) was obtained for 82.5% (437/530) participants, with similar response rates among intervention (240/296, 81%) and control groups (197/234, 84.2%; see). Several baseline characteristics, including having a respiratory disease and lower number of motivators, were associated with a patient’s odds of having a missing follow-up response for the primary outcome.
Before the preliminary effectiveness analysis, we independently aggregated patient responses at baseline (pre) and follow-up (post) to the cluster-period level.summarizes the resulting 20 follow-up observations, each of which represents the mean number of MET-minutes per week reported among patients within a team (cluster) at a specific time (period) at follow-up. Comparing intervention with control within teams (ignoring time), a positive, albeit non–statistically significant difference in total MET-minutes per week was found at 4 months (team 1, MD 1412, 95% CI −2023 to 4846; team 2, MD 732, 95% CI −1059 to 2522; team 3, MD 292, 95% CI −1550 to 2133; and team 4, MD 1370, 95% CI −650 to 3391). After adjusting for time (period) and mean number of MET-minutes at baseline, cluster-level linear regression yielded a non–statistically significant difference in the grand mean number of MET-minutes reported per week at follow-up between intervention and control conditions (MD 1027, 95% CI −155 to 2209, P=.09).
A sensitivity analysis was conducted where any participants with a follow-up response in the top 5% (ie, ≥12,780 MET-min) were identified as statistical outliers. In total, 22 participants were flagged as outliers (intervention, n=14, 5%; control, n=8, 4%). Outliers were, on average, more likely to self-report a significantly greater number of MET-minutes at baseline versus nonoutliers (MD 5650, 95% CI 4082 to 7218); otherwise, the distribution of all other baseline characteristics was statistically equivalent between outliers and nonoutliers. After excluding outliers, the subsequent linear regression yielded a non–statistically significant and less positive (closer to the null) difference in the grand mean number of MET minutes reported per week between intervention and control conditions (MD 487, 95% CI −298 to 1273; P=.22).
There were no significant treatment effects on action self-efficacy (n=392; MD intervention-control −1.73, 95% CI −5.56 to 2.11, P=.38), maintenance self-efficacy (n=361; MD intervention-control −1.92, 95% CI −5.68 to 1.85, P=.32), recovery self-efficacy (n=420; MD intervention-control 2.28, 95% CI −1.39 to 5.94, P=.22), and overall self-efficacy (n=413; MD intervention-control 1.13, 95% CI −1.73 to 4.00, P=.44). There were also no significant differences in the mean proportion (PR) of subjects who were in the volitional phase at 4 months (PR intervention/control 0.95, 95% CI 0.14 to 6.66; P=.96), or those who were classified as actors at 4 months (PR intervention/control 0.88, 95% CI 0.11 to 7.12; P=.91).
|Team||Posttest PAa mean in METb, minutes per week (95% CI)|
|Study period 1c||Study period 2d||Study period 3e||Study period 4f||Study period 5g|
|1||2636 (1432-3840)||3535 (2208-4861)h||6727 (2463-10992)h||3418 (2405-4431)h||2941 (1989-3893)h|
|2||2277 (1280-3274)||3942 (2289-5595)||3996 (2379-5613)h||3702 (1571-5832)h||4545 (2407-6682)h|
|3||4889 (2293-7484)||2774 (1597-3951)||3867 (1802-5932)||5223 (2166-8281)h||3365 (1993-4737)h|
|4||3918 (2220-5615)||4129 (2237-6020)||2596 (1965-3226)||4256 (1805-6708)h||4936 (2272-7600)h|
aPA: physical activity.
bMET: metabolic equivalent of task.
hExposure to intervention.
This study assessed the feasibility of implementing a primary care-based eHealth tool to screen for PA levels and provide tailored, evidence-based resources for both providers and patients. Over the course of 6 months, 530 patients were enrolled, with limited investment in personnel. Results show a trend toward improvement in PA levels for those who received the intervention, although the unexpectedly high variability limited statistical power. Few prior studies have successfully implemented a tool that both screens and provides tailored resources for PA in a primary care setting [, , ]. This study demonstrates the feasibility and potential of impact of using eHealth technology to deliver tailored, evidence-based care in primary care; a model that can be adapted to many health-promotion behaviors.
The high recruitment rate, including completion of emailed e-surveys, aligns with previous studies that suggest high acceptability among patients for using e-surveys to collect primary care data and integrate it into the EMR [- ]. The process evaluation indicates that almost all patients in the intervention arm received counseling about PA, almost half received a PA prescription, and most of them were highly satisfied with the counseling they received. This suggests a potential for eHealth interventions to reduce barriers to screening, counseling, and self-management for health behaviors and to improve adherence to evidence-based treatment guidelines in a manner that is patient-centered [ , , ].
However, intervention fidelity was not ideal: only a quarter of intervention patients received the customized toolkit with tailored messaging and resources from their PCP. These results suggest the existence of barriers to clinicians’ distribution of resource toolkits; this may include low perceived benefit of toolkit, poor intervention design, lack of education, or competing time pressures. Although all participating physicians received training before joining the intervention arm, it is possible that it was not sufficient, and further efforts to remind providers of the intervention would be required. It is possible that sending tailored information directly to patients before an appointment may facilitate shared decision-making on PA during the clinical encounter [, ]. Exploring factors related to patient engagement and contextual factors impacting use in the clinic and patient context is also an important consideration that will be useful to evaluate in a larger trial [ ].
The IPAQ-SF tool was selected to measure the primary outcome of this study, because of its frequent use and feasibility. The study attempted to control for previously documented concerns with high measurement variability with a large sample size . However, PA levels as measured by the IPAQ-SF tool in our study exhibited higher than expected levels of variability in the data, making it difficult to attribute intervention effects. Although there is some evidence that accelerometers provide complementary or even superior PA tracking to self-reported tools [ ], the limited resources of this pilot study precluded us from their use. Future studies of similar interventions may benefit from a composite outcome of PA levels to reduce variability and improve accuracy, including both self-reported measures and tracking using an accelerometer [ - ].
In addition to careful consideration of outcome measures, appropriate patient selection is an important consideration for future work in this area. It is possible that patients who attend clinic for a PHR may be systematically different (ie, biased toward interest or willingness to engage in healthy lifestyle behaviors) from the general population. PHR visits were used as they present a highly feasible time to incorporate structured counseling on PA. However, this potential bias may explain why patients in our study had much higher than expected levels of PA. It is also possible that focusing on these types of visits limits the potential for effectiveness of the intervention if PA is already routinely discussed during usual care. Unfortunately, resources were not available to capture process data from usual care patients in this pilot trial.
This pilot feasibility study has several important limitations including the low number of participating clusters resulting in few random assignments. Randomizing a small number of clusters can undermine the conventional benefits of randomization, resulting in increased risk of chance imbalances, increased type I and II error, and limited external validity . Furthermore, within many cluster-periods, a limited number of participants were enrolled, which was reflected in the substantial observed variability in the primary outcome at the patient-level. In recognizing these limitations, we opted to use a cluster-level analysis to circumvent issues regarding patient- and PCP-level baseline imbalances and clustering of patient responses within teams. In aggregating to this level, however, the number of observations and corresponding statistical power was substantially reduced. As a SW-CRT design is suitable to test the effect of an intervention on PA, future studies must recruit a large number of clusters to minimize the aforementioned issues. This would enable using more conventional, mixed-effects regression that accounts for clustering via random effects and involves a greater number of observations (via avoiding aggregation) that can result in more power to detect treatment effects, if truly present [ ], and adjust for baseline imbalances with reduced concern of overfitting. Furthermore, our process evaluation had a high level of missing data, particularly on the overall satisfaction question, increasing the risk of bias in the reported results. It is possible that those who were not satisfied with the intervention were less likely to respond. Steps to increase response rates to process measure surveys, including electronic delivery, should be considered for future work.
Our pilot study demonstrates that it is feasible to build an eHealth tool that integrates both screening and tailored resource provision in primary care with good patient acceptability. Further work to better understand and address clinician barriers to resource distribution is needed. Future studies should include a greater number of clusters, improved methods for collecting process measures to reduce missing data, and more accurate measures for capturing PA levels.
This research was funded by the Academic Health Sciences Centre Alternate Funding Innovation Fund. The project funder had no role in study design; collection, management, analysis, and interpretation of data; writing of the report; and the decision to submit the report for publication. The authors would like to thank Sherry Teeter and Ameet Mathrani from Women’s College Hospital Family Practice Health Centre for their significant contributions in algorithm development, participant recruitment, and data collection and Doug Kavanagh from Ocean by CognisantMD for assistance in developing the eHealth tool.
Conflicts of Interest
Intervention description.DOCX File , 1927 KB
Training material.PNG File , 336 KB
Process evaluation.DOCX File , 16 KB
Physical activity survey.DOCX File , 20 KB
Process evaluation survey results.DOCX File , 21 KB
Cross-sectional sample size per team and measurement period.DOCX File , 66 KB
CONSORT-EHEALTH checklist (V 1.6.1).PDF File (Adobe PDF File), 496 KB
- World Health Organization. Global Recommendations on Physical Activity for Health. Geneva: WHO Press; 2010.
- Tremblay MS, Warburton DE, Janssen I, Paterson DH, Latimer AE, Rhodes RE, et al. New Canadian physical activity guidelines. Appl Physiol Nutr Metab 2011 Feb;36(1):36-46; 47. [CrossRef] [Medline]
- Leitzmann MF, Park Y, Blair A, Ballard-Barbash R, Mouw T, Hollenbeck AR, et al. Physical activity recommendations and decreased risk of mortality. Arch Intern Med 2007 Dec 10;167(22):2453-2460. [CrossRef] [Medline]
- Woodcock J, Franco OH, Orsini N, Roberts I. Non-vigorous physical activity and all-cause mortality: systematic review and meta-analysis of cohort studies. Int J Epidemiol 2011 Feb;40(1):121-138. [CrossRef] [Medline]
- Samitz G, Egger M, Zwahlen M. Domains of physical activity and all-cause mortality: systematic review and dose-response meta-analysis of cohort studies. Int J Epidemiol 2011 Oct;40(5):1382-1400. [CrossRef] [Medline]
- Wen CP, Wai JP, Tsai MK, Yang YC, Cheng TY, Lee M, et al. Minimum amount of physical activity for reduced mortality and extended life expectancy: a prospective cohort study. Lancet 2011 Oct 1;378(9798):1244-1253. [CrossRef] [Medline]
- Statistics Canada. Table 13-10-0388-01: Household Population Meeting/Not Meeting the Canadian Physical Activity Guidelines URL: https://www150.statcan.gc.ca/t1/tbl1/en/tv.action?pid=1310038801 [accessed 2019-06-15]
- Sanchez A, Bully P, Martinez C, Grandes G. Effectiveness of physical activity promotion interventions in primary care: a review of reviews. Prev Med 2015 Jul;76:S56-S67. [CrossRef] [Medline]
- Jacobson DM, Strohecker L, Compton MT, Katz DL. Physical activity counseling in the adult primary care setting: position statement of the American College of Preventive Medicine. Am J Prev Med 2005 Aug;29(2):158-162. [CrossRef] [Medline]
- National Institute for Health and Care Excellence (UK). National Institute for Health and Care Excellence: Clinical Guidelines. London: National Institute for Health and Care Excellence (UK); 2003.
- World Health Organization. Physical Activity Strategy for The WHO European Region 2016–2025. In: Proceedings of the 65th session of the WHO Regional Committee for Europe. Vilnius, Lithuania: World Health Organization; 2015 Presented at: WHO/Europe'15; September 14-17, 2015; Vilnius, Lithuania URL: http://www.euro.who.int/__data/assets/pdf_file/0010/282961/65wd09e_PhysicalActivityStrategy_150474.pdf?ua=1
- Piercy KL, Troiano RP, Ballard RM, Carlson SA, Fulton JE, Galuska DA, et al. The Physical Activity Guidelines for Americans. J Am Med Assoc 2018 Nov 20;320(19):2020-2028. [CrossRef] [Medline]
- Grandes G, Sanchez A, Sanchez-Pinilla RO, Torcal J, Montoya I, Lizarraga K, PEPAF Group. Effectiveness of physical activity advice and prescription by physicians in routine primary care: a cluster randomized trial. Arch Intern Med 2009 Apr 13;169(7):694-701. [CrossRef] [Medline]
- Petrella RJ, Koval JJ, Cunningham DA, Paterson DH. Can primary care doctors prescribe exercise to improve fitness? The Step Test Exercise Prescription (STEP) project. Am J Prev Med 2003 May;24(4):316-322. [CrossRef] [Medline]
- Petrella RJ, Lattanzio CN. Does counseling help patients get active? Systematic review of the literature. Can Fam Physician 2002 Jan;48:72-80 [FREE Full text] [Medline]
- Welcome to CDC stacks. 2012. Trends in Adults Receiving Recommendation for Exercise or Other Physical Activity From a Physician or Other Health Professional URL: https://stacks.cdc.gov/view/cdc/12065 [accessed 2020-01-03]
- Eakin EG, Brown WJ, Marshall AL, Mummery K, Larsen E. Physical activity promotion in primary care: bridging the gap between research and practice. Am J Prev Med 2004 Nov;27(4):297-303. [CrossRef] [Medline]
- Podl TR, Goodwin MA, Kikano GE, Stange KC. Direct observation of exercise counseling in community family practice. Am J Prev Med 1999 Oct;17(3):207-210. [CrossRef] [Medline]
- Glasgow RE, Eakin EG, Fisher EB, Bacak SJ, Brownson RC. Physician advice and support for physical activity: results from a national survey. Am J Prev Med 2001 Oct;21(3):189-196. [CrossRef] [Medline]
- Hébert ET, Caughy MO, Shuval K. Primary care providers' perceptions of physical activity counselling in a clinical setting: a systematic review. Br J Sports Med 2012 Jul;46(9):625-631. [CrossRef] [Medline]
- Kennedy A, Bower P, Reeves D, Blakeman T, Bowen R, Chew-Graham C, Salford National Institute for Health Research Gastrointestinal programme Grant Research Group. Implementation of self management support for long term conditions in routine primary care settings: cluster randomised controlled trial. Br Med J 2013 May 13;346:f2882 [FREE Full text] [CrossRef] [Medline]
- Frank E, Segura C, Shen H, Oberg E. Predictors of Canadian physicians' prevention counseling practices. Can J Public Health 2010;101(5):390-395. [Medline]
- Lin JS, O'Connor E, Whitlock EP, Beil TL. Behavioral counseling to promote physical activity and a healthful diet to prevent cardiovascular disease in adults: a systematic review for the U.S. Preventive Services Task Force. Ann Intern Med 2010 Dec 7;153(11):736-750. [CrossRef] [Medline]
- Carey M, Noble N, Mansfield E, Waller A, Henskens F, Sanson-Fisher R. The role of eHealth in optimizing preventive care in the primary care setting. J Med Internet Res 2015 May 22;17(5):e126 [FREE Full text] [CrossRef] [Medline]
- Flores Mateo G, Granado-Font E, Ferré-Grau C, Montaña-Carreras X. Mobile phone apps to promote weight loss and increase physical activity: a systematic review and meta-analysis. J Med Internet Res 2015 Nov 10;17(11):e253 [FREE Full text] [CrossRef] [Medline]
- Lustria ML, Cortese J, Noar SM, Glueckauf RL. Computer-tailored health interventions delivered over the web: review and analysis of key components. Patient Educ Couns 2009 Feb;74(2):156-173. [CrossRef] [Medline]
- Free C, Phillips G, Galli L, Watson L, Felix L, Edwards P, et al. The effectiveness of mobile-health technology-based health behaviour change or disease management interventions for health care consumers: a systematic review. PLoS Med 2013;10(1):e1001362 [FREE Full text] [CrossRef] [Medline]
- Kreuter MW, Wray RJ. Tailored and targeted health communication: strategies for enhancing information relevance. Am J Health Behav 2003;27(Suppl 3):S227-S232. [CrossRef] [Medline]
- Noar SM, Chabot M, Zimmerman RS. Applying health behavior theory to multiple behavior change: considerations and approaches. Prev Med 2008 Mar;46(3):275-280. [CrossRef] [Medline]
- Noar SM, Benac CN, Harris MS. Does tailoring matter? Meta-analytic review of tailored print health behavior change interventions. Psychol Bull 2007 Jul;133(4):673-693. [CrossRef] [Medline]
- Broekhuizen K, Kroeze W, van Poppel MN, Oenema A, Brug J. A systematic review of randomized controlled trials on the effectiveness of computer-tailored physical activity and dietary behavior promotion programs: an update. Ann Behav Med 2012 Oct;44(2):259-286 [FREE Full text] [CrossRef] [Medline]
- Horne E, Lancaster GA, Matson R, Cooper A, Ness A, Leary S. Pilot trials in physical activity journals: a review of reporting and editorial policy. Pilot Feasibility Stud 2018;4:125 [FREE Full text] [CrossRef] [Medline]
- Eldridge SM, Chan CL, Campbell MJ, Bond CM, Hopewell S, Thabane L, PAFS consensus group. CONSORT 2010 statement: extension to randomised pilot and feasibility trials. Pilot Feasibility Stud 2016;2:64 [FREE Full text] [CrossRef] [Medline]
- Hemming K, Taljaard M, McKenzie JE, Hooper R, Copas A, Thompson JA, et al. Reporting of stepped wedge cluster randomised trials: extension of the CONSORT 2010 statement with explanation and elaboration. Br Med J 2018 Nov 9;363:k1614 [FREE Full text] [CrossRef] [Medline]
- Pinnock H, Barwick M, Carpenter CR, Eldridge S, Grandes G, Griffiths CJ, StaRI Group. Standards for Reporting Implementation Studies (StaRI) Statement. Br Med J 2017 Mar 6;356:i6795 [FREE Full text] [CrossRef] [Medline]
- Thabane L, Ma J, Chu R, Cheng J, Ismaila A, Rios LP, et al. A tutorial on pilot studies: the what, why and how. BMC Med Res Methodol 2010 Jan 6;10:1 [FREE Full text] [CrossRef] [Medline]
- Hussey MA, Hughes JP. Design and analysis of stepped wedge cluster randomized trials. Contemp Clin Trials 2007 Feb;28(2):182-191. [CrossRef] [Medline]
- Hemming K, Haines TP, Chilton PJ, Girling AJ, Lilford RJ. The stepped wedge cluster randomised trial: rationale, design, analysis, and reporting. Br Med J 2015 Feb 6;350:h391 [FREE Full text] [CrossRef] [Medline]
- Liddy C, Hogg W, Singh J, Taljaard M, Russell G, Armstrong CD, et al. A real-world stepped wedge cluster randomized trial of practice facilitation to improve cardiovascular care. Implement Sci 2015 Oct 28;10:150 [FREE Full text] [CrossRef] [Medline]
- Ridley J, Ischayek A, Dubey V, Iglar K. Adult health checkup: Update on the Preventive Care Checklist Form. Can Fam Physician 2016 Apr;62(4):307-313 [FREE Full text] [Medline]
- Schwarzer R. Modeling health behavior change: how to predict and modify the adoption and maintenance of health behaviors. Appl Psychol 2008 Jan;57(1):1-29. [CrossRef]
- Eckerstorfer LV, Tanzer NK, Vogrincic-Haselbacher C, Kedia G, Brohmer H, Dinslaken I, et al. Key elements of mHealth interventions to successfully increase physical activity: meta-regression. JMIR Mhealth Uhealth 2018 Nov 12;6(11):e10076 [FREE Full text] [CrossRef] [Medline]
- van der Weegen S, Verwey R, Spreeuwenberg M, Tange H, van der Weijden T, de Witte L. The development of a mobile monitoring and feedback tool to stimulate physical activity of people with a chronic disease in primary care: a user-centered design. JMIR Mhealth Uhealth 2013 Jul 2;1(2):e8 [FREE Full text] [CrossRef] [Medline]
- van der Weegen S, Verwey R, Spreeuwenberg M, Tange H, Weijden T, Witte LD. User-centred development of a monitoring and feedback tool to stimulate physical activity. Int J Integr Care 2012;12(4). [CrossRef]
- Block G, Sternfeld B, Block CH, Block TJ, Norris J, Hopkins D, et al. Development of Alive! (A Lifestyle Intervention Via Email), and its effect on health-related quality of life, presenteeism, and other behavioral outcomes: randomized controlled trial. J Med Internet Res 2008 Nov 19;10(4):e43 [FREE Full text] [CrossRef] [Medline]
- Harte R, Glynn L, Rodríguez-Molinero A, Baker PM, Scharf T, Quinlan LR, et al. A human-centered design methodology to enhance the usability, human factors, and user experience of connected health systems: a three-phase methodology. JMIR Hum Factors 2017 Mar 16;4(1):e8 [FREE Full text] [CrossRef] [Medline]
- Reddeman L, Bourgeois N, Angl EN, Heinrich M, Hillier L, Finn H, et al. How should family physicians provide physical activity advice? Qualitative study to inform the design of an e-health intervention. Can Fam Physician 2019 Sep;65(9):e411-e419 [FREE Full text] [Medline]
- Sparling PB, Howard BJ, Dunstan DW, Owen N. Recommendations for physical activity in older adults. Br Med J 2015 Jan 21;350:h100. [CrossRef] [Medline]
- Canadian Society for Exercise Physiology. Canadian Society for Exercise Physiology. 2011. Canadian Physical Activity Guidelines - Clinical Practice Guideline Development Report URL: https://csepguidelines.ca/wp-content/uploads/2018/04/CPAGuideline_Report_JAN2011.pdf [accessed 2020-01-03]
- Hoffmann TC, Glasziou PP, Boutron I, Milne R, Perera R, Moher D, et al. Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. Br Med J 2014 Mar 7;348:g1687. [CrossRef] [Medline]
- Craig CL, Marshall AL, Sjöström M, Bauman AE, Booth ML, Ainsworth BE, et al. International physical activity questionnaire: 12-country reliability and validity. Med Sci Sports Exerc 2003 Aug;35(8):1381-1395. [CrossRef] [Medline]
- Sechrist KR, Walker SN, Pender NJ. Development and psychometric evaluation of the exercise benefits/barriers scale. Res Nurs Health 1987 Dec;10(6):357-365. [CrossRef] [Medline]
- Lippke S, Ziegelmann JP, Schwarzer R. Stage-specific adoption and maintenance of physical activity: testing a three-stage model. Psychol Sport Exerc 2005;6(5):585-603. [CrossRef]
- Lippke S, Schwarzer R, Ziegelmann JP, Scholz U, Schüz B. Testing stage-specific effects of a stage-matched intervention: a randomized controlled trial targeting physical exercise and its predictors. Health Educ Behav 2010 Aug;37(4):533-546. [CrossRef] [Medline]
- Schwarzer R, Lippke S, Luszczynska A. Mechanisms of health behavior change in persons with chronic illness or disability: the Health Action Process Approach (HAPA). Rehabil Psychol 2011 Aug;56(3):161-170. [CrossRef] [Medline]
- Thompson SB. Simple formulas for standard errors that cluster by both firm and time. J Financial Econ 2011;99(1):1-10. [CrossRef]
- Snijders TA, Bosker RJ. Multilevel Analysis: An Introduction To Basic And Advanced Multilevel Modeling. Second Edition. London: Sage Publications Ltd; 1999.
- McGuinness SL, O'Toole JE, Boving TB, Forbes AB, Sinclair M, Gautam SK, et al. Protocol for a cluster randomised stepped wedge trial assessing the impact of a community-level hygiene intervention and a water intervention using riverbank filtration technology on diarrhoeal prevalence in India. BMJ Open 2017 Mar 17;7(3):e015036 [FREE Full text] [CrossRef] [Medline]
- Hemming K, Taljaard M. Sample size calculations for stepped wedge and cluster randomised trials: a unified approach. J Clin Epidemiol 2016 Jan;69:137-146 [FREE Full text] [CrossRef] [Medline]
- Stenman E, Leijon ME, Calling S, Bergmark C, Arvidsson D, Gerdtham U, et al. Study protocol: a multi-professional team intervention of physical activity referrals in primary care patients with cardiovascular risk factors--the Dalby lifestyle intervention cohort (DALICO) study. BMC Health Serv Res 2012 Jun 22;12:173 [FREE Full text] [CrossRef] [Medline]
- Maddison R, Roberts V, McRobbie H, Bullen C, Prapavessis H, Glover M, et al. Exercise counseling to enhance smoking cessation outcomes: the Fit2Quit randomized controlled trial. Ann Behav Med 2014 Oct;48(2):194-204. [CrossRef] [Medline]
- Krebs P, Prochaska JO, Rossi JS. A meta-analysis of computer-tailored interventions for health behavior change. Prev Med 2010;51(3-4):214-221 [FREE Full text] [CrossRef] [Medline]
- Davies CA, Spence JC, Vandelanotte C, Caperchione CM, Mummery WK. Meta-analysis of internet-delivered interventions to increase physical activity levels. Int J Behav Nutr Phys Act 2012 Apr 30;9:52 [FREE Full text] [CrossRef] [Medline]
- Lofters AK, Telner D, Kalia S, Slater M. Association between adherence to cancer screening and knowledge of screening guidelines: feasibility study linking self-reported survey data with medical records. JMIR Cancer 2018 Nov 1;4(2):e10529 [FREE Full text] [CrossRef] [Medline]
- Paul CL, Carey M, Yoong SL, D'Este C, Makeham M, Henskens F. Access to chronic disease care in general practice: the acceptability of implementing systematic waiting-room screening using computer-based patient-reported risk status. Br J Gen Pract 2013 Sep;63(614):e620-e626 [FREE Full text] [CrossRef] [Medline]
- Hess R, Santucci A, McTigue K, Fischer G, Kapoor W. Patient difficulty using tablet computers to screen in primary care. J Gen Intern Med 2008 Apr;23(4):476-480 [FREE Full text] [CrossRef] [Medline]
- Bozic KJ, Belkora J, Chan V, Youm J, Zhou T, Dupaix J, et al. Shared decision making in patients with osteoarthritis of the hip and knee: results of a randomized controlled trial. J Bone Joint Surg Am 2013 Sep 18;95(18):1633-1639. [CrossRef] [Medline]
- Stacey D, Légaré F, Lewis K, Barry MJ, Bennett CL, Eden KB, et al. Decision aids for people facing health treatment or screening decisions. Cochrane Database Syst Rev 2017 Apr 12;4:CD001431 [FREE Full text] [CrossRef] [Medline]
- Agarwal P, Mukerji G, Desveaux L, Ivers NM, Bhattacharyya O, Hensel JM, et al. Mobile app for improved self-management of type 2 diabetes: multicenter pragmatic randomized controlled trial. JMIR Mhealth Uhealth 2019 Jan 10;7(1):e10321 [FREE Full text] [CrossRef] [Medline]
- Prince SA, Adamo KB, Hamel ME, Hardt J, Gorber SC, Tremblay M. A comparison of direct versus self-report measures for assessing physical activity in adults: a systematic review. Int J Behav Nutr Phys Act 2008 Nov 6;5:56 [FREE Full text] [CrossRef] [Medline]
- Reilly JJ, Penpraze V, Hislop J, Davies G, Grant S, Paton JY. Objective measurement of physical activity and sedentary behaviour: review with new data. Arch Dis Child 2008 Jul;93(7):614-619. [CrossRef] [Medline]
- Matthews CE, Ainsworth BE, Thompson RW, Bassett DR. Sources of variance in daily physical activity levels as measured by an accelerometer. Med Sci Sports Exerc 2002 Aug;34(8):1376-1381. [CrossRef] [Medline]
- Skender S, Ose J, Chang-Claude J, Paskow M, Brühmann B, Siegel EM, et al. Accelerometry and physical activity questionnaires - a systematic review. BMC Public Health 2016 Jun 16;16:515 [FREE Full text] [CrossRef] [Medline]
- Bort-Roig J, Gilson ND, Puig-Ribera A, Contreras RS, Trost SG. Measuring and influencing physical activity with smartphone technology: a systematic review. Sports Med 2014 May;44(5):671-686. [CrossRef] [Medline]
|eHealth: electronic health|
|EMR: electronic medical record|
|FPHC: Family Practice Health Centre|
|HAPA: health action process approach|
|IPAQ-SF: international physical activity questionnaire-short form|
|MD: mean difference|
|MET: metabolic equivalent of task|
|PA: physical activity|
|PCP: primary care provider|
|PHR: periodic health review|
|PR: mean proportion|
|SW-CRT: stepped wedge cluster randomized trial|
Edited by G Eysenbach; submitted 12.08.19; peer-reviewed by G Signorelli, J Alvarez Pitti; comments to author 30.08.19; revised version received 07.11.19; accepted 15.12.19; published 14.02.20
©Payal Agarwal, Natasha Kithulegoda, Zachary Bouck, Beth Bosiak, Ilana Birnbaum, Lindsay Reddeman, Liane Steiner, Liora Altman, Robin Mawson, Roni Propp, Jane Thornton, Noah Ivers. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 14.02.2020.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.