Published on 09.11.16 in Vol 18, No 11 (2016): November
Preprints (earlier versions) of this paper are available at http://preprints.jmir.org/preprint/6171, first published Jun 08, 2016.
mHealth or eHealth? Efficacy, Use, and Appreciation of a Web-Based Computer-Tailored Physical Activity Intervention for Dutch Adults: A Randomized Controlled Trial
Background: Until a few years ago, Web-based computer-tailored interventions were almost exclusively delivered via computer (eHealth). However, nowadays, interventions delivered via mobile phones (mHealth) are an interesting alternative for health promotion, as they may more easily reach people 24/7.
Objective: The first aim of this study was to compare the efficacy of an mHealth and an eHealth version of a Web-based computer-tailored physical activity intervention with a control group. The second aim was to assess potential differences in use and appreciation between the 2 versions.
Methods: We collected data among 373 Dutch adults at 5 points in time (baseline, after 1 week, after 2 weeks, after 3 weeks, and after 6 months). We recruited participants from a Dutch online research panel and randomly assigned them to 1 of 3 conditions: eHealth (n=138), mHealth (n=108), or control condition (n=127). All participants were asked to complete questionnaires at the 5 points in time. Participants in the eHealth and mHealth group received fully automated tailored feedback messages about their current level of physical activity. Furthermore, they received personal feedback aimed at increasing their amount of physical activity when needed. We used analysis of variance and linear regression analyses to examine differences between the 2 study groups and the control group with regard to efficacy, use, and appreciation.
Results: Participants receiving feedback messages (eHealth and mHealth together) were significantly more physically active after 6 months than participants in the control group (B=8.48, df=2, P=.03, Cohen d=0.27). We found a small effect size favoring the eHealth condition over the control group (B=6.13, df=2, P=.09, Cohen d=0.21). The eHealth condition had lower dropout rates (117/138, 84.8%) than the mHealth condition (81/108, 75.0%) and the control group (91/127, 71.7%). Furthermore, in terms of usability and appreciation, the eHealth condition outperformed the mHealth condition with regard to participants receiving (t182=3.07, P=.002) and reading the feedback messages (t181=2.34, P=.02), as well as the clarity of the messages (t181=1.99, P=.049).
Conclusions: We tested 2 Web-based computer-tailored physical activity intervention versions (mHealth and eHealth) against a control condition with regard to efficacy, use, usability, and appreciation. The overall effect was mainly caused by the more effective eHealth intervention. The mHealth app was rated inferior to the eHealth version with regard to usability and appreciation. More research is needed to assess how both methods can complement each other.
Trial Registration: Netherlands Trial Register: NTR4503; http://www.trialregister.nl/trialreg/admin/rctview.asp?TC=4503 (Archived by WebCite at http://www.webcitation.org/6lEi1x40s)
J Med Internet Res 2016;18(11):e278
Insufficient physical activity is considered to be a major public health issue worldwide [, ]. The Dutch public health guidelines recommend adults to engage in moderate- to vigorous-intensity physical activity for at least 30 minutes on at least 5 days per week [ , ]. Studies suggest that sufficient physical activity can effectively prevent numerous chronic diseases and mental health issues [ , - ]. Lee et al [ ] argued that 6% to 10% of worldwide deaths caused by noncommunicable diseases, such as cancer, cardiovascular diseases, and diabetes, can be attributed to physical inactivity. Therefore, there is a need for interventions that increase the level of physical activity and can reach a broad population cost effectively [ ].
Empirical research suggests that Web-based computer-tailored interventions are a promising solution . These interventions provide tailored information and feedback via the Internet and therefore have some important advantages. First, Web-based computer-tailored interventions can adapt intervention materials according to the specific situation, characteristics, and needs of an individual and accordingly make information more personally relevant for the individual [ - ]. Second, research has shown that tailored messages are more likely to be read, understood, discussed with others, and remembered by the receiver [ - ]. Third, due to the fact that more and more people are using the Internet to search for health-related information and health advice [ - ], Web-based computer-tailored health interventions offer an effective method to reach a broad population cost effectively [ - ]. Fourth, even though a broad population is targeted simultaneously, each individual can make use of the intervention privately at any given point in time or place [ , ].
Until a few years ago, Web-based computer-tailored interventions were almost exclusively delivered via computer. This medium of delivery has formed the term eHealth (electronic Health). The concept of eHealth has been described as the use of the Internet and related technologies to deliver health-related information and interventions . Even though eHealth has been shown to be an efficient strategy to lower costs and deliver health messages more interactively, it also has several disadvantages. One of the major problems with eHealth interventions is the high percentage of dropout [ , ].
To make interventions even more accessible, and thereby decrease chances of dropout, health promotion professionals are increasingly interested in the use of mHealth (mobile Health). mHealth refers to the delivery of health messages and interventions via mobile phones or tablets by making use of telecommunication and multimedia technologies [- ]. In the Netherlands, almost 70% of Dutch households use the Internet via mobile phones and approximately 45% use tablets [ ]. Based on the increasing usage of mobile phones as a lifestyle device, it has been argued that mHealth might increase the use of interventions and thereby also their efficacy [ , ]. Whereas computers and laptops are relatively stationary, mobile phones and tablets can be carried and used everywhere [ ]. People are able to use mHealth independent of time or space, which could improve the usage and evaluation of interventions compared with eHealth [ , , ].
Most people already use their phones for a variety of personal and work-related matters, such as social networking, calendaring, financial tracking, or emailing . This leads to the assumption that the inclusion of health-related information would be advisable. However, previous research shows some pitfalls of mHealth. First, mobile phone technology is a rapidly changing field that introduces new apps, communication possibilities, and additional gadgets nearly by the day. This makes it difficult for intervention developers to keep up with the newest technologies and interests of their users [ , ]. Second, although using text messaging can be a very effective way of communicating, some intervention messages might be too long or difficult to be presented in such a short manner. This restricted communication can lead to more misunderstandings between the participant and health professional, which in turn can influence the effectiveness of the intervention [ ]. And third, both participants and health professionals claim to feel unsure about the safety of private and sensitive information. Although this concern can also arise in the eHealth sector, the inferior but rapidly growing mHealth sector evokes skepticism on both sides [ ].
To examine whether mHealth can improve the use and efficacy and reduce dropout rates of Web-based computer-tailored interventions, this study examined the effects of an mHealth and eHealth intervention on physical activity compared with a control group. Both interventions were identical with regard to content but differed in the medium of delivery. The main aim of the study was to examine the efficacy of the 2 versions on physical activity and to compare them with a control group. A secondary aim was to study potential differences in dropout and appreciation of the mHealth and eHealth intervention.
The study was a 3-armed randomized controlled trial consisting of a no-treatment control group and 2 experimental conditions (eHealth and mHealth). We recruited participants from a Dutch online research panel and randomly assigned them to 1 of 3 conditions (eHealth, mHealth, or control). Participants were excluded from the study in case of (1) physical conditions hindering engagement in physical activity, (2) pregnancy at the time of recruitment, (3) having a holiday scheduled for more than 5 working days during the study period, and (4) participation in another intervention during the study period.
The baseline measurement took place in April 2014 and the follow-up measurement took place 6 months after baseline (in October 2014). All participants (control, eHealth, and mHealth) were informed about the study by email and asked to complete online questionnaires at 5 points in time: at baseline (T0), 7 days after baseline (T1), 7 days after T1 (T2), 7 days after T2 (T3), and 6 months after baseline (T4: follow-up questionnaire). When a questionnaire had not been completed within 7 days after the invitation email, a reminder was sent. The reminder was sent to prevent dropout and stimulate participants to continue with the intervention. It was not possible for participants to skip sessions, and the next session could only be accessed when the previous one was completed. So when participants received a reminder and accessed the intervention, they continued with the session that followed their last completed session; for example, after session 3, participants could not continue with session 5 until they had completed session 4. Participants received 2 bonus points amounting to €2.50 as an incentive for completing the intervention (the first bonus point after T3, the second one after T4). The 2 intervention groups (eHealth and mHealth) received, additionally to the questionnaires, feedback messages and advice based on their answers to the questionnaires at T0, T1, and T2. Participants allocated to the control condition were also asked to complete all questionnaires but did not receive any feedback or information.
To determine the sample size, we conducted a power analysis using G*Power (version 3.1; Heinrich-Heine University Dusseldorf, Germany) [, ] taking into account an effect size of 0.20, a power of 0.80, and an alpha of 5%. Based on this calculation, a minimum total sample size of 423 (141 participants per condition) was required.
Both the eHealth and the mHealth versions of the intervention were developed using the TailorBuilder software (OverNite Software Europe, Geleen, the Netherlands). Both interventions had exactly the same content. The mHealth intervention was specifically developed for use with a mobile phone, while the eHealth version was developed for use with a computer. Therefore, the intervention within the eHealth condition was delivered via email, whereas in the mHealth condition advice was delivered via short text messages (short message service; SMS). Questionnaires for both groups were sent via email; however, participants allocated to the mHealth group were requested to complete this questionnaire via their mobile phone. Participants in the control condition received an email to inform and remind them that they could assess a questionnaire.
Before starting, participants were clearly instructed that they should use the intervention only via the medium that belonged to their study condition. Participants in the eHealth condition were asked to use the intervention only via the computer and participants in the mHealth version were asked to use the intervention only via their mobile phone or tablet.
We assessed this adherence (use of the intervention) by means of a question in the follow-up questionnaire that asked participants which medium they had used for the intervention. It should be noted that this adherence is correspondingly based on self-reports. It unfortunately was not possible to use the logs of the intervention to assess the medium of use. Hence, we cannot 100% guarantee that the self-reported answers are actually in line with the medium of use. The visual format of the feedback messages was the same in the eHealth and mHealth interventions. In both interventions the feedback messages were merely provided by means of text, without any additional visual content.
The intervention (named SmartMobiel) was specifically focused on physical activity as a healthy lifestyle behavior. It was built on an existing eHealth intervention  and framed by the I-Change model [ , ] and the health action process approach [ , ]. The main goal of SmartMobiel was to stimulate participants’ awareness, ability factors (ie, action plans and goal action), and self-efficacy (see ) to engage in more physical activity. The intervention consisted of 5 successive rounds.
|Determinant||Theoretical method||Practical application||Intervention components|
|Awareness||Consciousness raising and feedback on performance||Compare baseline physical activity level with physical activity recommendation and current physical activity level||Feedback on participants’ physical activity pattern and sedentary behavior compared with physical activity guideline and additional information on their progress on a weekly basis|
|Ability factors||Action planning (active learning)||Encourage to formulate action plans||Example of action plan to help formulate appropriate action plans (what, when, where, with whom)|
|Preparatory planning (active learning)||Invite to formulate preparatory plans||Suggestion to organize social support (eg, to find a buddy, inform people in the social environment, ask for support, choose a start date)|
|Coping planning (active learning)||Encourage to formulate coping plans||Example of coping plan to help formulate appropriate coping plans (if-then)|
|Self-efficacy||Reinforcement||Compare baseline level in planning, enactment of plans, satisfaction with physical activity, and increased physical activity with current level||Feedback included compliments if planning, etc, were improved; if not successfully improved, feedback included questions stimulating self-reflection|
Round 1 Feedback: Messages 1-3
The intervention started with a baseline questionnaire (T0) consisting of 38 items concerning demographics, physical activity, sedentary behavior, and psychosocial factors (action planning, intention, satisfaction, and self-efficacy). All measurements were used as input for the tailored feedback messages, which were sent 2 days apart. The main aim of this first round was to inform participants how to successfully plan behavior change regarding physical activity. Based on the baseline questionnaire, participants received 3 feedback messages. The first message provided feedback about participants’ physical activity level. Depending on their reported physical activity level at baseline, the message indicated how their behavior compared with the standards and how they could improve their physical activity level. The second feedback message addressed participants’ intention to engage in physical activity. Finally, the last feedback message of step 1 was focused on planning precisely when, where, and in what type of physical activity participants planned to engage in the following week.
Round 2 Feedback: Messages 4-6
Respondents received the second questionnaire (T1) 1 week after baseline, which consisted of questions on physical activity and sedentary behavior (ie, the same questions as in the baseline measurement), intention, and self-efficacy. The main aim of this round was to give participants an overview of their physical activity level and ideas about how to overcome difficulties regarding their behavior change. In this round, 3 tailored feedback messages were sent (message 4, 5, and 6). The fourth feedback message compared participants’ physical activity level with their baseline physical activity level. After 2 days, respondents received the fifth feedback message, which focused on their sedentary behavior and indicated how many hours they sat per week and how they could decrease the time spent sitting. Respondents received a sixth feedback message focusing on self-efficacy with regard to overcoming situations in which it was difficult to be physically active, 5 days after the first follow-up questionnaire had been filled in.
Round 3 Feedback: Messages 7-9
During the third round, participants filled in the second follow-up questionnaire (T2). It assessed items regarding physical activity, sedentary behavior, satisfaction, plan enactment, intention, and self-efficacy. The main aim of this round was to encourage participants to act on their plans. Participants received a motivating feedback SMS or email 1 day after the second follow-up questionnaire. After 2 days, respondents received the eighth feedback message, which focused on participants’ habits and goal enactment. Respondents received a last feedback message about their physical activity progress during the intervention, 5 days after the second follow-up questionnaire had been filled in.
Round 4 Follow-Up Measurement and Progress Evaluation
The posttest served as a short-term follow-up measurement (T3). This measurement contained 41 items measuring physical activity, sedentary behavior, plan enactment, planning, intention, and self-efficacy. Additionally, we invited both experimental groups to fill in an evaluation questionnaire, consisting of 10 items, which focused on their appreciation of the content of the intervention.
Round 5 Final Follow-Up Measurements
This final 6-month follow-up questionnaire contained 35 items and assessed the effects of the intervention on physical activity, sedentary behavior, plan enactment, planning, intention, and self-efficacy.
At baseline (T0), respondents were asked to indicate their age, sex (1=male; 2=female), marital status (0=no relationship: unmarried without relationship, divorced without new relationship, widowed without new relationship; 1=relationship: married, unmarried in relationship, divorced in new relationship, widowed in new relationship), educational level (1=primary or basic vocational school; 2=secondary vocational school or high school; 3=high vocational school or university), work status (1=student; 2=job: employed, self-employed; 3=no job: unemployed, nonworking, retired), and height (in meters) and weight (in kilograms) to calculate the body mass index (BMI).
We measured physical activity both at baseline (T0) and at follow-up (T4) with the International Physical Activity Questionnaire (IPAQ) [- ]. The IPAQ consists of 6 items with a reference period of the past 7 days; participants were asked to indicate how many days per week they had engaged in, respectively, low, moderate, and vigorous physical activity. Additionally, they were asked for how many minutes they usually engaged in these activities on those days. In order to acquire an accurate measure of total physical activity per day, we multiplied the frequency and average duration of vigorous, moderate, and low physical activity and then divided the result by 7.
We measured all sociocognitive variables (ie, intention, self-efficacy, and action planning) at baseline (T0) and follow-up (T4) using adapted measures from previous studies [- ] and a 5-point Likert answering scale (1= low to 5= high). Assessment of these variables served as the basis for the feedback messages, as well as correction for potential confounders within the effect analyses. For each variable, we calculated a mean score.
Intention to engage in physical activity was assessed with 4 items (Cronbach alpha=.72). Participants were asked to indicate to what extent they intended to be physically active during the following week; for example, “I intend to be regularly physically active the upcoming week.” The subsequent questions concerned their intention to perform vigorous activities or moderate activities, and finally their intention to walk regularly.
Self-efficacy was measured by means of 6 items (Cronbach alpha=.86). Participants were asked to indicate to what extent they thought they were able to engage in physical activity when encountering difficult situations; for example, “I am going to be physically active next week even though I am stressed.”
Planning was measured by means of 4 items (Cronbach alpha=.89). Plans were related to the participants’ actual planned physical activity; that is, which type of activity, where to be performed, on which days, and for how long. The item stem “I have made a detailed plan regarding...” was followed by the items (1) “which type of physical activity,” (2) “where to exercise,” (3) “on which days to exercise,” and (4) “for how long to exercise.”
Action planning was assessed by 8 items (Cronbach alpha=.85) measuring whether participants planned to execute each of the 8 predefined plans. Action planning included plans that are likely to facilitate physical activity, such as “During the next week, I will buy the necessary equipment to be physically active.”
Plan enactment (T3) was assessed using 8 items (Cronbach alpha=.88) asking participants to indicate the extent to which they actually had executed the 8 actions plans on a 5-point scale. Plan enactment was directly related to the action planning items; for example, “During the last week, I have bought the necessary equipment to be physically active.”
We measured intervention completion using log file data in order to assess whether participants had completed the separate questionnaires. These scores were summed in order to calculate a total score for intervention use ranging from 0 completed rounds per questionnaire to a maximum of 4 completed rounds per questionnaire.
At T3, we asked both experimental groups to complete a process evaluation questionnaire. This questionnaire consisted of 10 items that assessed their appreciation of the intervention. One item measured the overall grade of the SmartMobiel intervention by asking respondents to give an overall score from 1 (very bad) to 10 (very good). Additionally, we assessed the appreciation of the feedback messages by means of 5 items (1=disagree; 5=agree) to investigate whether the feedback messages were (1) “convincing,” (2) “interesting,” (3) “informative,” (4) “clear,” and (5) “helpful.” Furthermore, we included 1 item using a 5-point scale (1=not appealing at all; 5=very appealing) to measure participants’ appreciation of the intervention design.
All statistical analyses were performed using IBM SPSS Statistics version 20 (IBM Corporation). We used multiple imputation with 25 iterations to replace missing values on sociocognitive and outcome variables at T0. Additionally, we replaced missing values on BMI and physical activity at T4.
Descriptive statistics and frequencies described the characteristics of the study population. We analyzed differences at baseline using analyses of variance (ANOVAs) with Tukey post hoc tests for continuous variables and chi-square tests with Bonferroni correction for categorical variables.
We analyzed attrition using logistic regression, with attrition at follow-up (T4) as the outcome variable (0=not completed; 1=completed whole intervention), and intervention condition and all baseline variables (ie, age, sex, educational level, BMI, baseline physical activity, and baseline sedentary behavior) as predictors. Process evaluation was analyzed using ANOVA with Tukey post hoc tests to assess the differences between the experimental conditions with regard to usability and appreciation.
Effect analyses were performed using linear regression analyses with the ENTER method. Analyses examined 3 independent effects: (1) intervention (eHealth and mHealth) versus control condition, (2) eHealth versus control condition, and (3) mHealth versus control condition. To analyze the last 2 effects, we recoded the study condition variable into 2 different dummies. We compared each intervention group only with the control group to examine their independent efficacy. All effect analyses were corrected for potential confounders (ie, baseline behavior, baseline differences, and predictors of attrition). We calculated Cohen d to assess the size of the possible effects.
The study was approved by the Ethical Committee Psychology of the Faculty of Psychology and Neuroscience at Maastricht University, the Netherlands (ECP-138 08_03_2014) and registered at the Netherlands Trial Register (NTR4503).
shows the characteristics of the total sample and the baseline differences between the 3 study conditions in terms of demographics, total minutes of physical activity per day, and minutes of moderate to vigorous physical activity. Comparison of baseline variables between groups showed no statistically significant differences.
|Baseline characteristics||Overall sample |
|F value||df||P value|
|Sex (female), n (%)||258 (69.2)||98 (71.0)||77 (71.3)||83 (65.4)||0.74||2||.48|
|Educational level, n (%)||0.36||2||.70|
|Low||23 (6.2)||8 (5.8)||4 (3.7)||11 (8.7)|
|Medium||121 (32.4)||48 (34.8)||42 (38.9)||31 (24.4)|
|High||224 (60.1)||81 (58.7)||58 (53.7)||85 (66.9)|
|Age in years, mean (SE)||38.69 (11.99)||39.32 (12.10)||38.03 (12.23)||38.55 (11.74)||0.36||.70|
|Self-efficacy, mean (SD)||3.40 (0.77)||3.35 (0.76)||3.47 (0.78)||3.39 (0.77)||0.73||2||.48|
|Intention, mean (SD)||3.70 (0.61)||3.72 (0.54)||3.70 (0.65)||3.68 (0.65)||0.10||2||.90|
|Physical activity level (low, moderate, and high), mean (SD)||54.12 (35.07)||52.72 (36.28)||55.29 (35.20)||54.69 (34.00)||0.18||2||.83|
|Baseline characteristics||Odds ratio||df||P value||95% CI|
|Condition (eHealth, mHealth, control)a|
|Sex (female, male)||2.16||1||.02||1.12–4.14|
|Educational level (low, middle, high)b|
|Educational level (low)||1.40||1||.54||0.47–4.14|
|Educational level (middle)||1.57||1||.15||0.84–2.92|
|Body mass index||0.98||1||.67||0.91–1.07|
|Physical activity (low, moderate, and high)||0.98||1||.18||0.99–1.00|
|Physical activity (moderate and high)||0.99||1||.049||0.97–1.001|
aReference category was the control group.
bReference group was high educational level.
shows the flow of respondents for the overall sample and separately for the 3 study conditions (see [ ] for the CONSORT eHealth checklist). Analysis showed that the overall participation rate at follow-up (T4) was 77.5% (289/373). When comparing dropout rates between the 3 conditions, the highest dropout rate was in the control group, in which 71.7% (91/127) of the participants at baseline completed the last follow-up questionnaire. The lowest dropout rate was in the eHealth condition, with a participation rate of 84.8% (117/138).
Attrition analysis () showed that respondents were more likely to complete the follow-up assessment when they were in the eHealth condition (compared with the control condition; odds ratio [OR] 2.43, P=.007), they were female (OR 2.16, P=.02), they were younger (OR 0.97, P=.009), and they had lower levels of daily moderate to vigorous physical activity (OR 0.99, P=.049). We included the significant predictors of dropout in all further analyses as potential confounders.
Results of the process analysis indicate that participants in the eHealth condition evaluated the intervention significantly better than did respondents in the mHealth condition for 3 items: receiving messages, reading messages, and the general clarity of the messages (see). For the other items, we found no significant differences between the 2 groups.
|Variable||Overall sample |
|t value||df||P value|
|Grade given for the whole intervention (range 1–10)||6.35 (1.63)||6.36 (1.60)||6.33 (1.68)||0.10||182||.92|
|Did you receive the 9 feedback messages?||4.52 (0.84)||4.67 (0.58)||4.29 (1.08)||3.07||182||.002|
|Did you read the 9 feedback messages you received?||4.60 (0.85)||4.72 (0.59)||4.43 (1.11)||2.34||181||.02|
|Were the feedback messages believable?||3.43 (0.87)||3.51 (0.83)||3.32 (0.92)||1.46||181||.15|
|Were the feedback messages interesting?||2.97 (1.05)||2.94 (1.09)||3.01 (0.99)||–0.44||181||.66|
|Were the feedback messages informative?||3.11 (1.03)||3.02 (1.08)||3.24 (0.96)||–1.43||181||.15|
|Were the feedback messages clear?||3.90 (0.74)||3.99 (0.74)||3.77 (0.71)||1.99||181||.049|
|Did the feedback messages help you to be physically active?||2.52 (1.06)||2.48 (1.07)||2.59 (1.05)||–0.66||181||.51|
|How attractive was the layout of the intervention for you?||3.02 (0.93)||2.99 (0.96)||3.05 (0.88)||–0.45||181||.66|
|Interventiona,b||Bc||SE||df||P value||95% CI|
|eHealth (1) versus control (0)||6.13||3.61||2||.09||–0.98 to 13.23|
|mHealth (1) versus control (0)||1.92||4.00||2||.63||–5.95 to 9.79|
|Intervention (1) versus control (0)||8.48||3.77||2||.03||1.06 to 15.90|
aIn the linear regression analyses the following covariates were included: baseline behavior, sex, age, and baseline moderate and vigorous physical activity.
bOutcome variable is average daily physical activity (light, moderate, and vigorous).
cB: unstandardized regression coefficient.
Regression analyses showed statistically significant differences between the intervention conditions and the control group for the total amount of physical activity (see). After 6 months, participants who used the intervention (ie, mHealth and eHealth together) were significantly more physically active than were participants in the control group (intervention groups: mean 56.35 minutes/day; control group: mean 47.79 minutes/day; B=8.48, df=2, P=.03, Cohen d=0.27). We found a small effect that was borderline significant for the difference between the eHealth group and control condition (eHealth: mean 57.91 minutes/day; control group: mean 47.79 minutes/day; B=6.13, df=2, P=.09, Cohen d=0.21) with regard to total physical activity. We found no effect between the mHealth group and the control group (mHealth: mean 54.78 minutes/day; control group: mean 47.79 minutes/day; B=1.92, df=2; P=.63, Cohen d=0.04) with regard to total physical activity. Secondary analyses with complete cases revealed similar results.
The aim of this study was to evaluate the efficacy, use, usability, and appreciation of 2 different versions (eHealth vs mHealth) of a Web-based computer-tailored physical activity intervention. Contradicting our hypothesis, the eHealth intervention resulted in better usability and appreciation than did the mHealth intervention. Further, we found no significant differences in use and effects between the mHealth and eHealth versions when compared with a control group. These findings imply that mHealth is not necessarily more suitable than eHealth interventions and even suggest that eHealth should still be preferred.
The effect analyses revealed a significant difference in physical activity when comparing the eHealth and mHealth versions against the control condition. Yet we found no differences in effect between the eHealth and mHealth versions. The effect size for the eHealth version suggests a small effect, but the significance level was only borderline significant due to the small sample size. Recent studies also suggested that the use of mobile phone-based interventions may have positive effects on physical activity and weight loss but did not compare the efficacy of mHealth versus that of eHealth [, , ]. In line with our findings, it has been suggested that mHealth may be less suitable to achieve behavior change, since participants in an mHealth condition can use the intervention wherever they are at any given time [ ]. One explanation may be that mHealth participants may be more prone to distractions than eHealth users. eHealth users may be more committed to take the time to complete their tasks, whereas mHealth users may have been in distracting surroundings and situations such as supermarkets or public transport, which may lead to skipping or misreading messages. However, this explanation needs more research to demonstrate its applicability. The explanation is in line with the assumption of the elaboration likelihood model of Petty and Cacioppo [ ]. The model explains that distraction can result in peripheral route processing rather than in more central processing, which is associated with more (enduring) behavior change [ , ].
The higher dropout rate in the mHealth condition can possibly also be explained by the fact that people are more easily distracted when using their mobile phone. A recent study showed that people tend to use their mobile phones during short waiting times (eg, waiting for the bus, waiting in line at a checkout) . This means, on the one hand, that they use the device frequently; on the other hand, it implies that its use can be short and with many interruptions. Previous studies demonstrated that mobile phone use can distract people from other activities such as driving a car [ , ]. However, ongoing activities and the surroundings might also distract the person from the task he or she is doing on the mobile phone. Distraction might lead to worse performance, as well as to forgetting or neglecting the task completely [ , ]. Furthermore, the possibility of distraction might also explain the finding that the eHealth group evaluated the intervention significantly better than did the mHealth group regarding receiving and reading feedback messages, as well as the clarity of the feedback messages. Elaboration likelihood model research has shown that when information is processed via the peripheral route it is less appreciated by the receiver [ , ]. Furthermore, peripheral route processing can lead to lower motivation to engage with the context of the intervention, which would lead to the lower levels of appreciation [ ].
Another explanation could be that, while using a mobile phone is often spontaneous and a direct action that is driven by technology, the use of eHealth might be much more user driven. This means that, whereas participants in the mHealth group might have felt obligated to check their message the moment they received it, regardless of time, place, and concentration, eHealth participants consciously chose to start their computer to check their emails. This feeling of autonomously choosing when to engage in the intervention can lead to more intrinsic motivation and appreciation of the intervention . A different explanation for the low usability and appreciation could be the difference in the technology itself. The intervention was message based, which might have led to more misunderstanding of the feedback messages within the mHealth group than within the eHealth group [ ].
Strengths and Limitations
An important strength is that, to our knowledge, this is the first study that compared an eHealth intervention with an mHealth intervention with regard to efficacy, use, and appreciation.
The first limitation is that all outcome measures were self-reported . Research has shown that self-reported measures, in comparison with objective measures, have both a lower reliability and less validity. However, the IPAQ has been proven to be a reliable and valid measurement of physical activity [ ]. Yet replication with other, more objective assessments for measuring physical activity, such as accelerometers, is recommended.
The second limitation is that it was necessary to replace missing values with multiple imputations. Although multiple imputations are often used, there is discussion about how to correctly apply this technique . However, we found the same results regardless of whether we performed the analyses with the multiple imputation or with the completers-only dataset.
The third limitation is that our process analyses were not accompanied by qualitative measurements. For example, by asking participants why they found messages less clear, we could have gained insight into whether the difference between groups was based on technical difficulties only or could be attributed to the other factors.
The fourth limitation is that, because the tested intervention was message based, the results are difficult to generalize to the broader field of mHealth and eHealth.
The fifth limitation is that participants were all recruited from an online panel and were a random sample from the panel. This might make it difficult to generalize the findings from participants who are used to participating in scientific research to the broader population.
Lastly, as pointed out above as well, our sample size was limited. Each condition had approximately 100 participants, and power analyses revealed that we needed at least 141 participants per group to detect standardized effects of 0.20. As the results showed that effect sizes were indeed roughly 0.20, replication of this study with a larger sample is recommended to be able to demonstrate more statistically significant results.
Based on our results, we can conclude that the eHealth version outperformed the mHealth version of a Web-based computer-tailored physical activity intervention with regard to usability and appreciation, but not with regard to effectiveness.
The eHealth intervention excelled with regard to usability and appreciation compared with the mHealth intervention, and there are indications that the eHealth intervention may have been used more often. However, a study by Morrison et al  showed the advantages of combining mHealth and eHealth. They reported that, although their mHealth version did not function as an alternative to eHealth, it enhanced the intervention with regard to perceived accessibility, mobility, and on-the-go gadgets.
We recommend performing more research to assess and develop interventions that combine mHealth and eHealth technologies.
The authors thank Sander Matthijs Eggers for his valuable comments and his statistical feedback during the writing of the manuscript. The contents of this publication are solely the responsibility of the author and do not necessarily represent the views of any institution or funding agency.
Conflicts of Interest
Multimedia Appendix 1
CONSORT-EHEALTH checklist V1.6.1.PDF File (Adobe PDF File), 1005KB
- World Health Organization. Physical Inactivity: A Global Public Health Problem. Geneva, Switzerland: WHO; 2015. URL: http://www.who.int/dietphysicalactivity/factsheet_inactivity/en/ [accessed 2016-10-14] [WebCite Cache]
- World Health Organization. Global Recommendations on Physical Activity for Health. Geneva, Switzerland: WHO; 2010. URL: http://apps.who.int/iris/bitstream/10665/44399/1/9789241599979_eng.pdf [accessed 2016-10-14] [WebCite Cache]
- Hildebrandt VH, Bernaards CM, Stubbe JH. Trendrapport Bewegen en Gezondheid 2010/2011. 2010. URL: https://www.volksgezondheidenzorg.info/sites/default/files/hildebrandt_trendrapport_bewegen_gezondheid_2010_2011.pdf [accessed 2016-10-14] [WebCite Cache]
- Van den Brink C, Savelkoul M. Volksgezondheid Toekomst Verkenning, Nationaal Kompas Volksgezondheid. 2013. Gezondheidsmonitor GGD'en, CBS en RIVM URL: https://www.volksgezondheidenzorg.info/ [accessed 2016-10-18] [WebCite Cache]
- Scully D, Kremer J, Meade M, Graham R, Dudgeon K. Physical exercise and psychological well being: a critical review. Br J Sports Med 1998 Jun;32(2):111-120 [FREE Full text] [Medline]
- Warburton DE, Nicol CW, Bredin SS. Health benefits of physical activity: the evidence. CMAJ 2006 Mar 14;174(6):801-809 [FREE Full text] [CrossRef] [Medline]
- Lee I, Shiroma E, Lobelo F, Puska P, Blair S, Katzmarzyk P, Lancet Physical Activity Series Working Group. Effect of physical inactivity on major non-communicable diseases worldwide: an analysis of burden of disease and life expectancy. Lancet 2012 Jul 21;380(9838):219-229 [FREE Full text] [CrossRef] [Medline]
- Wantland DJ, Portillo CJ, Holzemer WL, Slaughter R, McGhee EM. The effectiveness of Web-based vs. non-Web-based interventions: a meta-analysis of behavioral change outcomes. J Med Internet Res 2004 Nov 10;6(4):e40 [FREE Full text] [CrossRef] [Medline]
- de Vries H, Brug J. Computer-tailored interventions motivating people to adopt health promoting behaviours: introduction to a new approach. Patient Educ Couns 1999 Feb;36(2):99-105. [Medline]
- Schulz D, Fleig L, Kremers S, De Vries H. How effective are tailored mobile phone short-messages in increasing exercise among Dutch and German adults? Psychol Health 2012;27:320-321.
- Stanczyk NE, Bolman C, Muris JW, de Vries H. Study protocol of a Dutch smoking cessation e-health program. BMC Public Health 2011;11:847 [FREE Full text] [CrossRef] [Medline]
- Kroeze W, Werkman A, Brug J. A systematic review of randomized trials on the effectiveness of computer-tailored education on physical activity and dietary behaviors. Ann Behav Med 2006 Jun;31(3):205-223. [CrossRef] [Medline]
- Smeets T, Brug J, de Vries H. Effects of tailoring health messages on physical activity. Health Educ Res 2008 Jun;23(3):402-413 [FREE Full text] [CrossRef] [Medline]
- van Beelen ME, Beirens TM, den Hertog P, van Beeck EF, Raat H. Effectiveness of web-based tailored advice on parents' child safety behaviors: randomized controlled trial. J Med Internet Res 2014;16(1):e17 [FREE Full text] [CrossRef] [Medline]
- AlGhamdi KM, Moussa NA. Internet use by the public to search for health-related information. Int J Med Inform 2012 Jun;81(6):363-373. [CrossRef] [Medline]
- Beck F, Richard J, Nguyen-Thanh V, Montagni I, Parizot I, Renahy E. Use of the Internet as a health information resource among French young adults: results from a nationally representative survey. J Med Internet Res 2014;16(5):e128 [FREE Full text] [CrossRef] [Medline]
- Siliquini R, Ceruti M, Lovato E, Bert F, Bruno S, De Vito V, et al. Surfing the Internet for health information: an Italian survey on use and population choices. BMC Med Inform Decis Mak 2011;11:21 [FREE Full text] [CrossRef] [Medline]
- Bennett G, Glasgow R. The delivery of public health interventions via the Internet: actualizing their potential. Annu Rev Public Health 2009;30:273-292. [CrossRef] [Medline]
- Krebs P, Prochaska JO, Rossi JS. A meta-analysis of computer-tailored interventions for health behavior change. Prev Med 2010;51(3-4):214-221 [FREE Full text] [CrossRef] [Medline]
- Lustria MLA, Cortese J, Noar SM, Glueckauf RL. Computer-tailored health interventions delivered over the Web: review and analysis of key components. Patient Educ Couns 2009 Feb;74(2):156-173. [CrossRef] [Medline]
- Schulz DN, Smit ES, Stanczyk NE, Kremers SP, de Vries H, Evers SM. Economic evaluation of a Web-based tailored lifestyle intervention for adults: findings regarding cost-effectiveness and cost-utility from a randomized controlled trial. J Med Internet Res 2014;16(3):e91 [FREE Full text] [CrossRef] [Medline]
- Smit ES, Evers SM, de Vries H, Hoving C. Cost-effectiveness and cost-utility of Internet-based computer tailoring for smoking cessation. J Med Internet Res 2013;15(3):e57 [FREE Full text] [CrossRef] [Medline]
- Escoffery C, McCormick L, Bateman K. Development and process evaluation of a Web-based smoking cessation program for college smokers: innovative tool for education. Patient Educ Couns 2004 May;53(2):217-225. [CrossRef] [Medline]
- Eysenbach G. The law of attrition. J Med Internet Res 2005;7(1):e11 [FREE Full text] [CrossRef] [Medline]
- Christensen H, Griffiths KM, Farrer L. Adherence in Internet interventions for anxiety and depression. J Med Internet Res 2009;11(2):e13 [FREE Full text] [CrossRef] [Medline]
- PricewaterhouseCoopers. Emerging mHealth: Paths for Growth. New York, NY: PwC; Jun 2012.
- Bert F, Giacometti M, Gualano MR, Siliquini R. Smartphones and health promotion: a review of the evidence. J Med Syst 2014 Jan;38(1):9995. [CrossRef] [Medline]
- Dennison L, Morrison L, Conway G, Yardley L. Opportunities and challenges for smartphone applications in supporting health behavior change: qualitative study. J Med Internet Res 2013;15(4):e86 [FREE Full text] [CrossRef] [Medline]
- Lathia N, Pejovic V, Rachuri KK, Mascolo C, Musolesi M, Rentfrow PJ. Smartphones for large-scale behavior change interventions. IEEE Pervasive Computing 2013;12:66-73.
- Free C, Phillips G, Felix L, Galli L, Patel V, Edwards P. The effectiveness of M-health technologies for improving health and health services: a systematic review protocol. BMC Res Notes 2010 Oct 6;3:250 [FREE Full text] [CrossRef] [Medline]
- Smith JJ, Morgan PJ, Plotnikoff RC, Dally KA, Salmon J, Okely AD, et al. Smart-phone obesity prevention trial for adolescent boys in low-income communities: the ATLAS RCT. Pediatrics 2014 Sep;134(3):e723-e731 [FREE Full text] [CrossRef] [Medline]
- Centraal Bureau voor de Statistiek. Documentatierapport ICT bij huishoudens en personen (ICThhpers) 2013V1. 2013. URL: https://www.cbs.nl/NR/rdonlyres/A296DD08-845B-4852-ACED-C693A87380C8/0/icthhpersmicrodata.pdf [accessed 2016-10-18] [WebCite Cache]
- Klasnja P, Pratt W. Healthcare in the pocket: mapping the space of mobile-phone health interventions. J Biomed Inform 2012 Feb;45(1):184-198 [FREE Full text] [CrossRef] [Medline]
- Whittaker R, Merry S, Dorey E, Maddison R. A development and evaluation process for mHealth interventions: examples from New Zealand. J Health Commun 2012;17 Suppl 1:11-21. [CrossRef] [Medline]
- O'Reilly GA, Spruijt-Metz D. Current mHealth technologies for physical activity assessment and promotion. Am J Prev Med 2013 Oct;45(4):501-507 [FREE Full text] [CrossRef] [Medline]
- Mechael P, Batavia H, Kaonga N, Searle S, Kwan A, Goldberger A, et al. Barriers and Gaps Affecting mHealth in Low and Middle Income Countries: Policy White Paper. New York, NY: Center for Global Health and Economic Development, Earth Institute, Columbia University; 2010 May. URL: http://www.globalproblems-globalsolutions-files.org/pdfs/mHealth_Barriers_White_Paper.pdf [accessed 2016-10-18] [WebCite Cache]
- Levin D. MHealth: promise and pitfalls. Frontiers Health Serv Manage 2011;29(2):33-39.
- Faul F, Erdfelder E, Buchner A, Lang A. Statistical power analyses using G*Power 3.1: tests for correlation and regression analyses. Behav Res Methods 2009 Nov;41(4):1149-1160. [CrossRef] [Medline]
- Faul F, Erdfelder E, Lang A, Buchner A. G*Power 3: a flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behav Res Methods 2007 May;39(2):175-191. [Medline]
- de Vries H, Mudde A, Leijs I, Charlton A, Vartiainen E, Buijs G, et al. The European Smoking Prevention Framework Approach (EFSA): an example of integral prevention. Health Educ Res 2003 Oct;18(5):611-626 [FREE Full text] [Medline]
- Vries HD, Mudde AN. Predicting stage transitions for smoking cessation applying the attitude-social influence-efficacy model. Psychol Health 1998;13(2):369-385.
- Schwarzer R, Renner B. Social-cognitive predictors of health behavior: action self-efficacy and coping self-efficacy. Health Psychol 2000 Sep;19(5):487-495. [Medline]
- Schwarzer R, Lippke S, Luszczynska A. Mechanisms of health behavior change in persons with chronic illness or disability: the Health Action Process Approach (HAPA). Rehabil Psychol 2011 Aug;56(3):161-170. [CrossRef] [Medline]
- Kohl HW, Craig CL, Lambert EV, Inoue S, Alkandari JR, Leetongin G, Lancet Physical Activity Series Working Group. The pandemic of physical inactivity: global action for public health. Lancet 2012 Jul 21;380(9838):294-305. [CrossRef] [Medline]
- Hagströmer M, Oja P, Sjöström M. The International Physical Activity Questionnaire (IPAQ): a study of concurrent and construct validity. Public Health Nutr 2006 Sep;9(6):755-762. [Medline]
- Arends S, Hofman M, Kamsma YP, van der Veer E, Houtman PM, Kallenberg CG, et al. Daily physical activity in ankylosing spondylitis: validity and reliability of the IPAQ and SQUASH and the relation with clinical assessments. Arthritis Res Ther 2013;15(4):R99 [FREE Full text] [CrossRef] [Medline]
- Allen JK, Stephens J, Dennison Himmelfarb CR, Stewart KJ, Hauck S. Randomized controlled pilot study testing use of smartphone technology for obesity treatment. J Obes 2013;2013:151597 [FREE Full text] [CrossRef] [Medline]
- Stephens J, Allen J. Mobile phone interventions to increase physical activity and reduce weight: a systematic review. J Cardiovasc Nurs 2013;28(4):320-329 [FREE Full text] [CrossRef] [Medline]
- Smith A. U.S. Smartphone Use in 2015. Washington, DC: Pew Research Center; 2015 Apr 01. URL: http://www.pewinternet.org/2015/04/01/us-smartphone-use-in-2015/ [accessed 2016-10-14] [WebCite Cache]
- Eysenbach G, CONSORT-EHEALTH Group. CONSORT-EHEALTH: improving and standardizing evaluation reports of Web-based and mobile health interventions. J Med Internet Res 2011;13(4):e126 [FREE Full text] [CrossRef] [Medline]
- Petty R, Cacioppo J. The elaboration likelihood model of persuasion. Adv Exp Social Psychol 1986;19:123-205.
- Petty R, Barden J, Wheeler, SC. The elaboration likelihood model of persuasion: health promotions that yield sustainable behavioral change. In: DiClemente RJ, Crosby RA, Kegler MC. editors. Emerging Theories in Health Promotion Practice and Research. San Francisco, CA: Josey-Bass; 2009:185-214.
- Basacik D, Reed N, Robbins R. Smartphone use while driving: a simulator study. Published Project Report PPR592. Transport Research Laboratory; 2011. URL: https://merritt.cdlib.org/d/ark:%252F13030%252Fm5pr7t9s/1/producer%252F779410002.pdf [WebCite Cache]
- Gill P, Kamath A, Gill T. Distraction: an assessment of smartphone usage in health care work settings. Risk Manag Healthc Policy 2012;5:105-114 [FREE Full text] [CrossRef] [Medline]
- Katz-Sidlow RJ, Ludwig A, Miller S, Sidlow R. Smartphone use during inpatient attending rounds: prevalence, patterns and potential for distraction. J Hosp Med 2012 Oct;7(8):595-599. [CrossRef] [Medline]
- Petty R, Cacioppo J. Communication and Persuasion: Central and Peripheral Routes to Attitude Change. New York, NY: Springer Science & Business Media; 2012.
- Petty R, Cacioppo J, Schumann D. Central and peripheral routes to advertising effectiveness: the moderating role of involvement. J Consumer Res 1983;10(2):135-146.
- Ryan RM, Deci EL. Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. Am Psychol 2000 Jan;55(1):68-78. [Medline]
- Biddle S, Gorely T, Pearson N, Bull FC. An assessment of self-reported physical activity instruments in young people for population surveillance: Project ALPHA. Int J Behav Nutr Phys Act 2011;8:1 [FREE Full text] [CrossRef] [Medline]
- Zhang P. Multiple imputation: theory and method. Int Stat Rev 2003;71(3):581-592.
- Morrison LG, Hargood C, Lin SX, Dennison L, Joseph J, Hughes S, et al. Understanding usage of a hybrid website and smartphone app for weight management: a mixed-methods study. J Med Internet Res 2014 Oct 22;16(10):e201 [FREE Full text] [CrossRef] [Medline]
|ANOVA: analysis of variance|
|BMI: body mass index|
|IPAQ: International Physical Activity Questionnaire|
|OR: odds ratio|
|SMS: short message service|
Edited by G Eysenbach; submitted 08.06.16; peer-reviewed by L Morrison, I Montagni; comments to author 10.08.16; revised version received 22.08.16; accepted 22.08.16; published 09.11.16
©Stefanie Gomez Quiñonez, Michel Jean Louis Walthouwer, Daniela Nadine Schulz, Hein de Vries. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 09.11.2016.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.