Published on 09.12.20 in Vol 22, No 12 (2020): December
Preprints (earlier versions) of this paper are available at http://preprints.jmir.org/preprint/21687, first published Jun 22, 2020.
Design Features for Improving Mobile Health Intervention User Engagement: Systematic Review and Thematic Analysis
Background: Well-designed mobile health (mHealth) interventions support a positive user experience; however, a high rate of disengagement has been reported as a common concern regarding mHealth interventions. To address this issue, it is necessary to summarize the design features that improve user engagement based on research over the past 10 years, during which time the popularity of mHealth interventions has rapidly increased due to the use of smartphones.
Objective: The aim of this review was to answer the question “Which design features improve user engagement with mHealth interventions?” by summarizing published literature with the purpose of guiding the design of future mHealth interventions.
Methods: This review followed the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) checklist. Databases, namely, PubMed, Web of Science, Cochrane Library, Ovid EMBASE, and Ovid PsycINFO, were searched for English and Chinese language papers published from January 2009 to June 2019. Thematic analysis was undertaken to assess the design features in eligible studies. The Mixed Methods Appraisal Tool was used to assess study quality.
Results: A total of 35 articles were included. The investigated mHealth interventions were mainly used in unhealthy lifestyle (n=17) and chronic disease (n=10) prevention programs. Mobile phone apps (n=24) were the most common delivery method. Qualitative (n=22) and mixed methods (n=9) designs were widely represented. We identified the following 7 themes that influenced user engagement: personalization (n=29), reinforcement (n=23), communication (n=20), navigation (n=17), credibility (n=16), message presentation (n=16), and interface aesthetics (n=7). A checklist was developed that contained these 7 design features and 29 corresponding specific implementations derived from the studies.
Conclusions: This systematic review and thematic synthesis identified useful design features that make an mHealth intervention more user friendly. We generated a checklist with evidence-based items to enable developers to use our findings easily. Future evaluations should use more robust quantitative approaches to elucidate the relationships between design features and user engagement.
J Med Internet Res 2020;22(12):e21687
Mobile health (mHealth) is a means of providing medical and public health support to health care consumers via mobile devices, such as mobile phones, portable computers, and personal digital assistants . mHealth interventions involve the adoption of mobile technologies to provide educational information, help users manage their own conditions and behaviors, and deliver health care to improve the health of users. Compared to traditional delivery models, mobile interventions can be more cost-effective [ , ]; help users overcome demographic, socioeconomic, and geographic barriers to access [ - ]; allow the privacy of users to be protected [ ]; and allow a high level of customization, self-management, and communication [ - ]. With the popularity of smartphones, mHealth technology has grown rapidly in the past 10 years and has been used in many health fields. Mobile interventions have been shown to improve healthy behaviors (eg, weight loss [ ], balanced diet consumption [ ], and smoking cessation [ ]) and disease management [ ] (eg, tuberculosis [ ] and AIDS [ ]).
mHealth programs require autonomous use , which depends on user involvement and self-management, and engagement is related to behavior changes and health improvements [ ]. User engagement refers to high uptake, high-quality user experience, and good adherence over long periods of time [ ]. However, a pressing concern regarding mHealth interventions is the high reported rate of disengagement [ ]. For example, in a web-based weight loss study, respondents did not use the app as intended, and only 64% of the intervention group actually used the intervention at least once [ ]. A mobile phone text message smoking cessation program also suffered from a high attrition rate: nearly half of the subscribers did not complete the entire program in the real-world implementation phase, and the majority of opt-outs occurred in the first 2 weeks [ ]. With regard to web-based physical activity interventions, reported dropout attrition rates vary between 0% and 62% [ , ]. This is a common issue related to mHealth interventions; poor user engagement makes intervention effectiveness difficult because the users are not exposed to enough of the intervention content [ ].
The development steps of an mHealth intervention tool can influence user engagement. The first step is intervention content development. User engagement will be low if the content does not adhere to what has been shown to be effective [, ], and there are already well-established guidelines for health intervention content development [ , , ]. The second step is design of the mode by which the intervention content is delivered (eg, information architecture, screen appearance, and interactive features). Poor design features, such as complicated navigation and difficult-to-read screen presentations, are poorly tolerated by users in real-world settings [ ]. Good mHealth design is readily distinguishable from its competitors, leading users to feel more favorably disposed toward the product and have a positive user experience [ , , , ]. Several studies analyzed which design features should be included in mHealth interventions [ - ]; however, they did not clearly describe how to specifically deliver the interventions, and the contributions of most of the design features mentioned have not been tested in empirical studies. In addition, Morrison et al [ ] developed a hypothetical framework to define the design features through a review, but the framework only focused on 4 interactive design features (social context and support, contact with the intervention, tailoring, and self-management) and provided simple definitions of the other 8 features that were difficult to implement in subsequent studies. Webb et al [ ] developed a coding scheme for design features in a meta-analysis; however, this scheme was proposed before a review of the literature was performed rather than derived from the literature reviewed, so it inevitably missed features that are important but not reported.
While these studies have provided some important guidance for the design features of mHealth interventions, none has included comprehensive design features based on the literature or experiences. Additionally, except for Crutzen et al , other researchers did not focus on the relationships between the design features mentioned and user engagement. Furthermore, the data on which these studies were based were obtained more than 10 years ago, and in the past 10 years, the popularity of mHealth interventions has rapidly expanded due to the use of smartphones; data from more recent studies need to be considered. The objectives of our study were to systematically review studies published in the past 10 years regarding design features that improve user engagement with mHealth interventions and generate a checklist that can easily be used during the design of future mHealth interventions.
This systematic review followed the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) checklist . The protocol was registered in the International Prospective Register of Systematic Reviews (CRD42020140282).
Inclusion and Exclusion Criteria
Articles were selected if they met the following criteria: (1) The study was empirical. The study population was composed of users or potential users of mHealth. If a study focused on special populations, such as children and older adults, it was excluded. (2) The study focused on the mode of delivery of health intervention content via mobile devices, for example, aesthetics, message phrasing, and interactive features. Articles that mentioned design features but did not explain them, making it unclear which features were being referenced, were excluded. Studies on the selection of intervention content or theory, and those providing a general description of the process of designing an mHealth intervention were excluded. (3) The study reported quantitative and qualitative analyses of the effectiveness of the design features with regard to increasing user engagement or user acceptance of the design features. (4) The article was published in a peer-reviewed journal from January 1, 2009 to June 13, 2019. (5) The article was published in English or Chinese.
Search and Screening Strategy
Five databases, namely, PubMed, Web of Science, Cochrane Library, Ovid EMBASE, and Ovid PsycINFO, were searched in June 2019. The search strategy was “mobile health” or mHealth or m-health or eHealth or e-health combined with design or feature* or principle* or “mode* of delivery” or model and combined with engage* or adhere* or maintain* or retention or sustain* or usage* or satisf* or prefer or preference* or accept* or reliable.
The search results were uploaded to EndNote (Version X9; Clarivate Analytics) for screening.shows the process of identifying the eligible articles. Duplicates were identified with the sorting function in EndNote. Peer-reviewed journal articles were checked by searching Ulrich's Periodicals Directory; some journals could not be found there, so we visited the journal website to find evidence of the peer-review process. All remaining articles were assigned to 1 of 3 reviewers and were screened by titles, abstracts, and then full texts according to inclusion and exclusion criteria. YW screened all journal articles independently again. Any disagreement was discussed among the reviewers.
Data Extraction and Synthesis
All selected articles were imported into NVivo (version 11; QSR International). The following study characteristics were extracted: (1) article characteristics, including country and year of publication; (2) health topic; (3) participant characteristics, including sampling methods, sample size, sex, and age; (4) data collection method; and (5) mobile technology. Study characteristics were analyzed using descriptive statistics.
The thematic synthesis analysis method developed by Thomas and Harden  has 3 steps: (1) line-by-line coding of the articles to record the components, (2) the development of descriptive themes, and (3) the creation of analytical themes. We performed the first 2 steps together. YW coded each line of the records according to its meaning, translated the concepts among the records, and developed the descriptive themes. HD validated the results by comparing each assigned code to the full texts of the articles. Then, analytical themes were developed by answering the review question using the existing descriptive themes. Each reviewer did this independently, and the results were discussed among all authors. The coding process was iterative.
Assessing Study Quality
The Mixed Methods Appraisal Tool (MMAT)  was used to assess the methodological quality of the included studies, including the data collection methods, participant sampling, interpretation of results, consideration of confounders, and risk of bias. In the MMAT, there are 5 criteria with responses “yes,” “no,” and “cannot tell” for each research design. The retained studies were assessed, and the results recorded by 2 authors (YW and HD), independently.
All 35 articles included in the analysis were published between 2011 and 2019. They were primarily from the United States (n=14), the United Kingdom (n=9), and Australia (n=5). In terms of the health topics discussed in these studies, 17 articles focused on unhealthy lifestyles (eg, smoking, excessive alcohol consumption, sleep disturbances, and poor sexual health), 10 studies focused on chronic diseases (eg, diabetes, breast cancer, chronic arthritis, and asthma), 4 studies focused on mental health problems, and 4 studies focused on other health issues. Qualitative studies (n=22) and mixed methods (n=9) were the most common data collection methods used, accounting for 89% (31/35) of all studies. The sample sizes ranged from 8 to 1865 in the 35 studies, and the age of the participants ranged from 14 to 74 years old. The mobile technology used in the majority of studies was a mobile phone app (n=24), followed by a website platform (n=6), and text messages (n=5). More detailed information is presented in.
A total of 3 articles meet the criteria of all 5 items, with the remaining articles meeting 4 criteria (n=18), 3 criteria (n=13), and 2 criteria (n=1). The most common reasons low scores in each research design were a lack of coherence between the data collection and analysis and the explanations in qualitative studies; the quality of different components was low in the mixed methods studies; and there were poor sampling strategies and a high risk of bias in the quantitative studies ().
Themes of Design Features
In total, 7 analytical themes were generated to describe the design features that can improve user engagement with mHealth interventions, and each of these can be explained by several descriptive themes. With regard to the 3 types of mobile technology used in the studies, mobile phone apps and website platforms share common design features, while text messages lack 2 analytical themes: interface aesthetic and reinforcement (). For health topics, unhealthy lifestyles, chronic diseases, and mental health problems share common design features, and the topic of other health issues lacks interface aesthetic and message presentation ( ).
The interface refers to the appearance of the screen, which was reported to directly impact the user's impression of the intervention and impact user engagement in 7 of the 35 studies (20%). Two descriptive themes that could improve interface aesthetics were identified: (1) attention-grabbing and (2) simple and consistent style.
An aesthetically appealing screen easily attracts user attention . Users preferred that the screen show graphics rather than too much text because the latter can be overwhelming [ , ]. They also preferred a pleasing color scheme. Bright colors (eg, light green, white) were considered attention grabbing, while dark and neon colors discouraged further use [ , ].
Simple and Consistent Style
A simple and clean screen was praised most frequently in the studies, while users disliked complex and overcrowded pages, which rapidly made them lose interest [, , , ]. Many users appreciated the use of a consistent style, with a coherent presentation in terms of colors, pictures, and themes throughout an mHealth intervention [ , ].
Navigation describes how users move to different areas of content within mHealth intervention apps. There were 2 descriptive themes pertaining to navigation: (1) ease of use and (2) automation; these themes were proposed by users in 17 of the 35 studies (49%).
Easy to Use
The users highlighted the importance of minimum input and efficient access to information, such as a simple log-in process, fewer required tasks, or fewer buttons on the screen [- ]. An interactive process that confused the users or that took them many iterations to understand prevented them from continuing to engage with the app. Explanation of how the mHealth intervention worked (ie, clarifying what to do next) promoted continued use [ , , , ].
Users did not want to spend much time scrolling to find the information they wanted. The search bar and menu bar, which provide options to the user, were thought to facilitate usage [, , - ].
Personalization is a design feature that makes mobile technology act in a particular way depending on user preferences. Personalization was achieved by the following 3 elements according to 29 of the 35 studies (83%): (1) assessment, (2) feedback, and (3) manipulation.
Users expected to be assessed with metrics pertaining to the health problem that was the focus of the intervention in as much detail as possible to create an accurate profile [, - ]. Some parameters, such as sociodemographic characteristics, basic health status, and individual preferences and habits, could be measured with a quick survey within an app [ , , , , ]. Users also liked continuous monitoring features that allowed them to record their progress toward their goal on a daily basis (eg, health and behavior changes and adherence to an intervention) [ , , , , , - ] or diary entries or notes that helped them track their progress [ , , ]. Moreover, users also indicated a preference for sensor-based automated tracking as opposed to self-reported data, which they often forgot to input and found were not as convenient [ , , , , ].
Studies indicated the importance of building an assessment on the basis of feedback on the acquired data; users quickly lost interest when they did not receive feedback that was customized [, ]. The preference was for the mobile device to provide personalized information, including tailored intervention content matched to their basic characteristics [ , , , , , , , - , , , , ] and feedback on continuous monitoring data, for example, their health and behavior progress over time, predicted possible causes and consequences of a health problem and advice on the behavior under investigation [ - , , , , , , , , ]. There was a strong interest among users for visualization of continuous monitoring data, for example, presenting data as graphs and tables [ , , , , , , ].
Users not only wanted to obtain automatically tailored information but also wanted to be able to customize the mHealth intervention themselves. Users highlighted the importance of being able to choose when and how they receive reminders [, , , , - , ], set goals for the future use of the tool [ , , , , , , , ], and select preferred styles, such as the color and font [ , , ].
Reinforcement is the provision of a stimulus to strengthen the likelihood of a user continuing to exhibit a certain behavior in the future. There were 2 descriptive themes extracted from 23 of the 35 studies (66%), namely, (1) rewards and (2) reminders, that helped enhance reinforcement.
The reward feature could increase user motivation to engage with the intervention, and users expressed desire for confirmation when they completed a task. The reward features extracted from the studies included material incentives (eg, cash or gifts), intangible rewards (eg, virtual badges, rankings, certificates, and points), and messages of congratulations when a task was completed [, , , , - , - , ].
Users wanted reminders to schedule a task, such as taking medicine, making and keeping clinic appointments, and continuing a health plan [, , , , , , ]; to return to their mobile device [ , , , ]; and to motivate them with information and advice [ , , , , , ]. The preferred forms of reminders included email messages, text messages, words of the day, and pop-ups [ , , , , ].
Communication is a function that prompts users to consult and communicate with other people regarding their health problems via a mobile device. (1) Peer-to-peer communication and (2) access to professionals were 2 aspects of communication reported in 20 of the 35 studies (57%).
The users expressed interest in communicating with other people with similar experiences through online forums, communities, by instant messages within an mHealth tool, or by connecting via other types of social media; they wanted to be able to post information, share their stories, ask and answer questions, and find mutual accountability partners [, , , - , - , , , , , ].
Access to Professionals
Users wanted to be able to directly contact a health care provider via email, text message, or live chat to ask them questions or obtain advice based on their health data [, , , - , - , , , , ].
The presentation of information is an important factor that impacts user engagement, and information that is presented well is readily accepted by users. (1) Language, (2) tone of voice, and (3) presentation design are points of consideration when seeking to improve message presentation, according to the results of 16 of the 35 studies (46%).
The message needs to be clearly presented in the language used by the audience. The users recommended using simple nontechnical language that is straightforward and concise; they were tired of patronizing and technical language [, , , , , ]. When providing an actionable message, users preferred a specific description that made it clear what they were supposed to do [ , , ].
Tone of Voice
Users recommended using language that framed the information positively rather than negatively, as the latter made them feel discouraged and made them want to turn off the device [, , , , , , ]. To increase user acceptance, it was essential to make the user feel supported and relaxed by using a nonauthoritarian, friendly, and nonjudgmental tone of voice [ , , , ].
The users were quickly bored with text-heavy presentations of information; they wanted multimedia messages, for example, text combined with relevant pictures or video [, , , ]. Knowledge quizzes and games were also recommended as ways to deliver information that prompted the user to engage and learn the information [ , ]. The use of various font styles, sizes, and colors to highlight key information was suggested, as it allowed the user to skim quickly when they lacked the patience to read the entire message [ , ]. Editing the text to make it as concise as possible was also suggested by users, who were not inclined to read lengthy messages [ , , ].
Credibility is an important feature that guarantees the level of user comfort, enabling them to engage with the mobile technology without experiencing concerns. (1) Trustworthiness and (2) confidentiality were 2 descriptive themes related to credibility derived from 16 of the 35 studies (46%).
Users trusted mHealth interventions from authoritative and familiar organizations or developers that were free from advertisements [, , , , , , , ]. Users emphasized the fact that the information provided needed to be evidence-based and from credible sources to gain their trust [ , , , , , ].
Checklist of Design Features to Enhance User Engagement
Based on these themes, we produced a checklist that considers 7 aspects of design and the corresponding implementations based on an exhaustive analysis of the 35 studies. In total, there were 29 items reported that enhance user engagement, and we provide here the descriptions and examples as a reference for future studies ().
|1||The screen shows a graphic presentation rather than too much information|
|2||Pleasing color scheme with bright colors (eg, light green, white)|
|3||Simple screen presentation that is not overcrowded|
|4||Coherent scheme of colors, pictures, and themes throughout the intervention|
|5||Minimum user input needed; efficient access to the information provided, such as in a simple menu; and few buttons on the screen|
|6||Guidance provided that explains how the mHealth intervention works|
|7||Search bar or menu bar provided to accelerate the process of finding certain information|
|8||Assessment of the preferences, sociodemographic characteristics and health status of the user|
|9||Continuous monitoring of health and behavior changes or adherence to an intervention|
|10||Provision of a diary or note-taking function|
|11||Provision of personalized information matched to the user\'s characteristics|
|12||Provision of feedback on the continuously monitored data|
|13||Visual presentation of feedback, such as in graphs and tables|
|14||Provision of autonomy to customize the intervention, for example, allowing the users to choose when and how they receive reminders, to set a goal about their future use of the intervention tool, and to select their preferred styles, such as their preferred colors and fonts|
|15||Provision of material incentives (eg, cash or gifts), intangible rewards (eg, virtual badges, rankings, certificates, and points), or messages of congratulations when a task is completed|
|16||Sending of reminders to facilitate the scheduling of tasks and to ensure continuous use|
|17||Provision of access to other people with similar experiences through an online forum, community, or instant messages within the mobile tool or by connection with other forms of social media|
|18||Provision of access to a health care provider through email, text message, or live chat|
|19||Use of simple nontechnical language that can be readily understood|
|20||Use of specific descriptions when providing actionable message|
|21||Use of a positive, nonauthoritarian, and nonjudgmental tone of voice|
|22||Provision of multimedia messages, for example, text combined with relevant pictures or videos|
|23||Presentation of information in the form of knowledge quizzes and games, if possible|
|24||Use of various font styles, sizes, and colors to highlight information|
|25||Editing of the text to make it as concise as possible|
|26||Absence of advertisements|
|27||Provision of evidence-based information from credible sources|
|29||Enabling users to set a password or code to protection their data|
We used thematic synthesis to identify design features that increased user engagement with mHealth interventions based on user feedback. For each design feature, specific implementations in mobile tools were also analyzed. This study presented 7 analytical themes and 16 descriptive themes pertaining to design features that can improve user engagement with mHealth interventions. From most to least commonly mentioned in the studies, the analytical themes were personalization (29/35, 83%), reinforcement (23/35, 66%), communication (20/35, 57%), navigation (17/35, 49%), credibility (16/35, 46%), message presentation (16/35, 46%), and interface aesthetic (7/35, 20%); each analytical theme involves several descriptive subthemes that explain how to implement them when designing mHealth interventions. Overall, the 7 analytical themes were applicable to different mobile technologies and health topics, indicating that the design features identified by this study are universal across mobile apps, website platforms, text messages, and different health themes.
To promote better application of the results of this study to future mHealth intervention development, we developed a checklist of the design features that enhance user engagement; this tool has 29 evidence-based items that are clearly described to make them easy to use by developers of mHealth interventions.
Personalization, reinforcement, and communication were the design features that were mentioned the most often. Compared to other analytical themes, these 3 design features focused on the interactivity of mHealth interventions, including user-to-technology interactions and user-to-user interactions. User-to-technology communication refers to having the user input information about themselves to which the tool provides a tailored response . Two analytical themes, personalization and reinforcement, pertain to the interaction between users and technology. User-to-user interactions are represented by the theme of communication in this article, including peer-to-peer contact and consultation with professionals. Interactive features give users a sense of ownership [ ] and promote their participation in the mHealth intervention. Compared with traditional smoking cessation methods, a major value of mobile health interventions is that they can provide better and faster interactions [ ] to meet the needs of users.
Personalization, reinforcement, and communication are design features of mobile health interventions and behavior change techniques  that can improve the effectiveness of the interventions. Morrison et al [ ] also showed that personalization and communication are related to effective intervention outcomes. Therefore, these interactive design features can improve user participation and promote the effectiveness of interventions, and special attention should be given to them during the design of mobile health interventions.
In terms of presentation and navigation, users preferred a user friendly design, specifically, one that was easy to use and understand and was aesthetically pleasing, which was mainly addressed by the design of the interface aesthetic, navigation, and message presentation. A user friendly design is easy to use and understand, with features such as simple and convenient navigation, easy to understand language, and a supportive tone. Users often experience difficulties when using new technology , and an easy-to-use design can reduce that burden [ , ]. A study [ ] showed the importance of a simple design when engaging users who were resistant to change. Compared to users who intend to change, those who were resistant to change were harder to engage in an intervention; however, this issue was alleviated with the help of a simple design [ ].
Aesthetics also increases the friendliness of the design. A lack of aesthetics and text-heavy presentations made users feel bored, and users liked information presented with pictures and short texts. Additionally, a beautiful interface more easily attracts the user's attention. Studies have shown that a friendly mobile health intervention design can win the trust of users  and that credibility is a major concern for potential users [ ]. Credibility needs to be established for the user to trust and use a mobile tool [ ]. Tools that are developed in the future should provide evidence-based information, privacy policies, and password protection; they should also remove advertisements.
The 7 analytical themes from this study all appeared in different health topics, indicating that the design features proposed in this study were universal across the different health topics. Mobile apps and the website platforms shared 7 analytical themes, but text messages lacked reinforcement and interface aesthetics. The reason for this may be that the small number of articles involving text messages and the features designed to improve user engagement were not comprehensive and that text messages can be regarded as simple mobile apps, which cannot realize all of the functions of mobile apps; for example, the aesthetic interfaces design feature does not apply to text messages.
Mobile phone apps and website platforms are currently the most advanced mHealth technologies . From this perspective, the design features of this study are also universal across different mobile technologies. Other simpler technologies can select a part of the design features in the checklist as a reference according to their own functions. Different mHealth technologies have different capabilities for presentation, navigation, and interaction, which may lead to different potential for improving user engagement. In the future, the relationship between different mHealth technologies and user engagement needs to be studied.
One study  about the design features of 100 smoking cessation apps on iTunes in 2016 showed that the existing smoking cessation apps performed well in terms of language but performed poorly in terms of presentation, navigation, and interaction. Our design feature checklist can help solve this problem. Researchers can use the checklist to guide the design process of mHealth interventions. The checklist can also be used to evaluate mHealth interventions that have been developed.
There is no unified terminology for design features. The same design feature has multiple different names in different studies. For example, communication can also be called social support and social network. This study summarized and translated the existing descriptions and described each design feature in a unified term to promote a standardized description of the design features.
This study has the following limitations. First, we only selected studies that explained how a design feature is implemented in technology because some articles lacked detailed descriptions, making them impossible to analyze. However, this may mean that the design features we extracted do not comprehensively represent all research that has been performed. Second, most included articles were qualitative and mixed methods studies, which have considerable advantages given the exploratory nature of this research; however, compared with quantitative research, these methods provide less convincing evidence of the relationship between design features and user engagement. There may be sufficient studies available to explore design features that can improve engagement, and a more robust quantitative study design is needed to verify the association. Third, the mobile technology used in the studies in this review were mobile phone apps, website platforms, and text messages, which do not represent all technologies used in mHealth interventions. Other mobile tools, such as remote measurement technology and wearable devices, do not have as many functions as the aforementioned technologies included in this study. Subsets of the design features proposed in our research can also be applied to other forms of mobile technology, and relevant studies, especially regarding remote and wearable devices, are needed, considering their differences compared with the mobile phones, tablets and laptops used in this research. Fourth, the MMAT is currently the most applicable appraisal tool for a systematic mixed studies review, and it provides a detailed assessment of the quality of the included studies. The quality of the included studies varied, and 14 articles met 2 or 3 criteria of all 5 items, which means bias is present. However, the 2018 version MMAT does not provide quantification of a study’s quality, and due to the heterogeneity of the included studies and the qualitative design of this systematic review, it is difficult to assess the weight of the different included studies or to obtain a more accurate understanding of the overall risk of bias.
This study summarized research results obtained in the past ten years to identify design features in mHealth interventions that improve user participation. We made a checklist that divided the design features of mHealth interventions into 7 different aspects with associated, clearly described implementations, which can not only be used as a reference during the mHealth development process but also as an evaluation tool for the design features of newly developed mHealth interventions. This checklist can be applied to mobile apps, website platforms, and text messages and can be applied to health topics such as unhealthy lifestyles, chronic diseases, and mental health problems. The study of the relationships between these design features and user engagement is in the exploratory stage but has great potential. We synthesized the results of currently available studies to promote better application of their results and to lay a foundation for subsequent confirmatory research.
This study was supported by the National Natural Science Foundation of China (number 71573047).
Conflicts of Interest
Characteristics of the original studies.DOC File , 22 KB
Quality assessment methodology.DOC File , 27 KB
- Källander K, Tibenderana JK, Akpogheneta OJ, Strachan DL, Hill Z, ten AAHA, et al. Mobile health (mHealth) approaches and lessons for increased performance and retention of community health workers in low- and middle-income countries: a review. J Med Internet Res 2013;15(1):e17 [FREE Full text] [CrossRef] [Medline]
- Moessner M, Bauer S, Ozer F, Wolf M, Zimmer B, Kordy H. Cost-effectiveness of an internet-based aftercare intervention after inpatient treatment in a psychosomatic hospital. Psychother Res 2014;24(4):496-503. [CrossRef] [Medline]
- Noar SM, Black HG, Pierce LB. Efficacy of computer technology-based HIV prevention interventions: a meta-analysis. AIDS 2009 Jan 2;23(1):107-115. [CrossRef] [Medline]
- Bidargaddi N, Almirall D, Murphy S, Nahum-Shani I, Kovalcik M, Pituch T, et al. To Prompt or Not to Prompt? A Microrandomized Trial of Time-Varying Push Notifications to Increase Proximal Engagement With a Mobile Health App. JMIR Mhealth Uhealth 2018 Nov 29;6(11):e10123 [FREE Full text] [CrossRef] [Medline]
- Ben-Zeev D, Scherer EA, Gottlieb JD, Rotondi AJ, Brunette MF, Achtyes ED, et al. mHealth for Schizophrenia: Patient Engagement With a Mobile Phone Intervention Following Hospital Discharge. JMIR Ment Health 2016;3(3):e34 [FREE Full text] [CrossRef] [Medline]
- Muñoz RF, Aguilera A, Schueller SM, Leykin Y, Pérez-Stable EJ. From online randomized controlled trials to participant preference studies: morphing the San Francisco Stop Smoking site into a worldwide smoking cessation resource. J Med Internet Res 2012 Jun 27;14(3):e64 [FREE Full text] [CrossRef] [Medline]
- Newman MW, Lauterbach D, Munson SA, Resnick P, Morris ME. “It's not that I don't have problems, I'm just not putting them on Facebook”: Challenges and Opportunities in Using Online Social Networks for Health. 2011 Presented at: Proceedings of the 2011 ACM Conference on Computer Supported Cooperative Work; March 19-23; Hangzhou, China. [CrossRef]
- Baltierra NB, Muessig KE, Pike EC, LeGrand S, Bull SS, Hightow-Weidman LB. More than just tracking time: Complex measures of user engagement with an internet-based health promotion intervention. J Biomed Inform 2016 Feb;59:299-307. [CrossRef] [Medline]
- Lustria MLA, Cortese J, Noar SM, Glueckauf RL. Computer-tailored health interventions delivered over the Web: review and analysis of key components. Patient Educ Couns 2009 Feb;74(2):156-173. [CrossRef] [Medline]
- Mouttapa M, Robertson TP, McEligot AJ, Weiss JW, Hoolihan L, Ora A, et al. The Personal Nutrition Planner: a 5-week, computer-tailored intervention for women. J Nutr Educ Behav 2011;43(3):165-172. [CrossRef] [Medline]
- BinDhim NF, McGeechan K, Trevena L. Who Uses Smoking Cessation Apps? A Feasibility Study Across Three Countries via Smartphones. JMIR Mhealth Uhealth 2014 Feb 06;2(1):e4. [CrossRef]
- Partridge SR, McGeechan K, Hebden L, Balestracci K, Wong AT, Denney-Wilson E, et al. Effectiveness of a mHealth Lifestyle Program With Telephone Support (TXT2BFiT) to Prevent Unhealthy Weight Gain in Young Adults: Randomized Controlled Trial. JMIR Mhealth Uhealth 2015;3(2):e66 [FREE Full text] [CrossRef] [Medline]
- Recio-Rodriguez JI, Agudo Conde C, Calvo-Aponte MJ, Gonzalez-Viejo N, Fernandez-Alonso C, Mendizabal-Gallastegui N, EVIDENT Investigators. The Effectiveness of a Smartphone Application on Modifying the Intakes of Macro and Micronutrients in Primary Care: A Randomized Controlled Trial. The EVIDENT II Study. Nutrients 2018 Oct 10;10(10):1473 [FREE Full text] [CrossRef] [Medline]
- Nguyen Thanh V, Guignard R, Lancrenon S, Bertrand C, Delva C, Berlin I, et al. Effectiveness of a Fully Automated Internet-Based Smoking Cessation Program: A Randomized Controlled Trial (STAMP). Nicotine Tob Res 2019 Jan 04;21(2):163-172. [CrossRef] [Medline]
- Free C, Phillips G, Galli L, Watson L, Felix L, Edwards P, et al. The effectiveness of mobile-health technology-based health behaviour change or disease management interventions for health care consumers: a systematic review. PLoS Med 2013;10(1):e1001362 [FREE Full text] [CrossRef] [Medline]
- Story A, Aldridge RW, Smith CM, Garber E, Hall J, Ferenando G, et al. Smartphone-enabled video-observed versus directly observed treatment for tuberculosis: a multicentre, analyst-blinded, randomised, controlled superiority trial. Lancet 2019 Mar 23;393(10177):1216-1224 [FREE Full text] [CrossRef] [Medline]
- Dillingham R, Ingersoll K, Flickinger TE, Waldman AL, Grabowski M, Laurence C, et al. PositiveLinks: A Mobile Health Intervention for Retention in HIV Care and Clinical Outcomes with 12-Month Follow-Up. AIDS Patient Care STDS 2018 Jun;32(6):241-250 [FREE Full text] [CrossRef] [Medline]
- Bakker D, Rickard N. Engagement in mobile phone app for self-monitoring of emotional wellbeing predicts changes in mental health: MoodPrism. J Affect Disord 2018 Dec;227:432-442. [CrossRef] [Medline]
- Torous J, Nicholas J, Larsen ME, Firth J, Christensen H. Clinical review of user engagement with mental health smartphone apps: evidence, theory and improvements. Evid Based Ment Health 2018 Jun 05:116-119. [CrossRef] [Medline]
- Kelders SM, Van GJEWC, Werkman A, Nijland N, Seydel ER. Effectiveness of a Web-based intervention aimed at healthy dietary and physical activity behavior: a randomized controlled trial about users and usage. J Med Internet Res 2011;13(2):e32 [FREE Full text] [CrossRef] [Medline]
- Augustson E, Cole-Lewis H, Sanders A, Schwarz M, Geng Y, Coa K, et al. Analysing user-reported data for enhancement of SmokefreeTXT: a national text message smoking cessation intervention. Tob Control 2017 Nov;26(6):683-689. [CrossRef] [Medline]
- Davies CA, Spence JC, Vandelanotte C, Caperchione CM, Mummery WK. Meta-analysis of internet-delivered interventions to increase physical activity levels. Int J Behav Nutr Phys Act 2012;9:52 [FREE Full text] [CrossRef] [Medline]
- Enwald HPK, Huotari MA. Preventing the obesity epidemic by second generation tailored health communication: an interdisciplinary review. J Med Internet Res 2010;12(2):e24 [FREE Full text] [CrossRef] [Medline]
- Guertler D, Vandelanotte C, Kirwan M, Duncan MJ. Engagement and Nonusage Attrition With a Free Physical Activity Promotion Program: The Case of 10,000 Steps Australia. J Med Internet Res 2015;17(7):e176 [FREE Full text] [CrossRef] [Medline]
- Michie S, Richardson M, Johnston M, Abraham C, Francis J, Hardeman W, et al. The behavior change technique taxonomy (v1) of 93 hierarchically clustered techniques: building an international consensus for the reporting of behavior change interventions. Ann Behav Med 2013 Aug;46(1):81-95. [CrossRef] [Medline]
- Schwarzer R. Modeling Health Behavior Change: How to Predict and Modify the Adoption and Maintenance of Health Behaviors. Applied Psychology 2008 Jan;57(1):1-29. [CrossRef]
- Kok G, Schaalma H, Ruiter RAC, van EP, Brug J. Intervention mapping: protocol for applying health psychology theory to prevention programmes. J Health Psychol 2004 Jan;9(1):85-98. [CrossRef] [Medline]
- Morrison LG, Yardley L, Powell J, Michie S. What design features are used in effective e-health interventions? A review using techniques from Critical Interpretive Synthesis. Telemed J E Health 2012 Mar;18(2):137-144. [CrossRef] [Medline]
- Ting Y. The Pitfalls of Mobile Devices in Learning: A Different View and Implications for Pedagogical Design. Journal of Educational Computing Research 2012 Jun 08;46(2):119-134. [CrossRef]
- Lee S, Ha S, Widdows R. Consumer responses to high-technology products: Product attributes, cognition, and emotions. Journal of Business Research 2011 Nov;64(11):1195-1200. [CrossRef]
- Yeager CM, Benight CC. If we build it, will they come? Issues of engagement with digital health interventions for trauma recovery. Mhealth 2018;4:37 [FREE Full text] [CrossRef] [Medline]
- Walther JB, Pingree S, Hawkins RP, Buller DB. Attributes of interactive online health information systems. J Med Internet Res 2005 Jul 1;7(3):e33 [FREE Full text] [CrossRef] [Medline]
- Ritterband LM, Thorndike FP, Cox DJ, Kovatchev BP, Gonder-Frederick LA. A behavior change model for internet interventions. Ann Behav Med 2009 Aug;38(1):18-27 [FREE Full text] [CrossRef] [Medline]
- Crutzen R, de NJ, Brouwer W, Oenema A, Brug J, de Vries NK. A conceptual framework for understanding and improving adolescents' exposure to Internet-delivered interventions. Health Promot Int 2009 Sep;24(3):277-284. [CrossRef] [Medline]
- Broderick J, Devine T, Langhans E, Lemerise AJ, Lier S, Harris L. Designing Health Literate Mobile Apps. In: NAM Perspectives. Washington, DC: Institute of Medicine of the National Academies; Jan 28, 2014.
- Webb TL, Joseph J, Yardley L, Michie S. Using the internet to promote health behavior change: a systematic review and meta-analysis of the impact of theoretical basis, use of behavior change techniques, and mode of delivery on efficacy. J Med Internet Res 2010;12(1):e4 [FREE Full text] [CrossRef] [Medline]
- Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann Intern Med 2009 Aug 18;151(4):264-9, W64 [FREE Full text] [CrossRef] [Medline]
- Thomas J, Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Med Res Methodol 2008;8:45 [FREE Full text] [CrossRef] [Medline]
- Hong Q, Pluye P, Fàbregues S, Bartlett G, Boardman F, Cargo M. Mixed methods appraisal tool (MMAT), version 2018. Registration of Copyright 2018:1148552.
- Su M, Chen W, Liu C, Jou H, Hsiao Y, Tsao L. [The Design Requirements for an E-Health Management Platform: Addressing the Needs of Adolescent Girls at High Risk of Metabolic Syndrome]. Hu Li Za Zhi 2015 Oct;62(5):51-60. [CrossRef] [Medline]
- Crane D, Garnett C, Brown J, West R, Michie S. Factors Influencing Usability of a Smartphone App to Reduce Excessive Alcohol Consumption: Think Aloud and Interview Studies. Front Public Health 2017;5:39 [FREE Full text] [CrossRef] [Medline]
- Peters D, Deady M, Glozier N, Harvey S, Calvo RA. Worker Preferences for a Mental Health App Within Male-Dominated Industries: Participatory Study. JMIR Ment Health 2018 Apr 25;5(2):e30 [FREE Full text] [CrossRef] [Medline]
- Perski O, Blandford A, Ubhi HK, West R, Michie S. Smokers' and drinkers' choice of smartphone applications and expectations of engagement: a think aloud and interview study. BMC Med Inform Decis Mak 2017 Dec 28;17(1):25 [FREE Full text] [CrossRef] [Medline]
- Lazard AJ, Pikowski J, Horrell L, Ross JC, Noar SM, Sutfin EL. Adolescents' and Young Adults' Aesthetics and Functionality Preferences for Online Tobacco Education. J Cancer Educ 2020 Apr;35(2):373-379. [CrossRef] [Medline]
- Ledel Solem IK, Varsi C, Eide H, Kristjansdottir OB, Mirkovic J, Børøsund E, et al. Patients' Needs and Requirements for eHealth Pain Management Interventions: Qualitative Study. J Med Internet Res 2019 Apr 01;21(4):e13205 [FREE Full text] [CrossRef] [Medline]
- Milward J, Deluca P, Drummond C, Watson R, Dunne J, Kimergård A. Usability Testing of the BRANCH Smartphone App Designed to Reduce Harmful Drinking in Young Adults. JMIR Mhealth Uhealth 2017 Aug 08;5(8):e109 [FREE Full text] [CrossRef] [Medline]
- Rabin C, Bock B. Desired features of smartphone applications promoting physical activity. Telemed J E Health 2011 Dec;17(10):801-803. [CrossRef] [Medline]
- Coyne I, Prizeman G, Sheehan A, Malone H, While AE. An e-health intervention to support the transition of young people with long-term illnesses to adult healthcare services: Design and early use. Patient Educ Couns 2016 Dec;99(9):1496-1504. [CrossRef] [Medline]
- Peng W, Yuan S, Holtz BE. Exploring the Challenges and Opportunities of Health Mobile Apps for Individuals with Type 2 Diabetes Living in Rural Communities. Telemed J E Health 2016 Dec;22(9):733-738. [CrossRef] [Medline]
- Gkatzidou V, Hone K, Sutcliffe L, Gibbs J, Sadiq ST, Szczepura A, et al. User interface design for mobile-based sexual health interventions for young people: design recommendations from a qualitative study on an online Chlamydia clinical care pathway. BMC Med Inform Decis Mak 2015 Aug 26;15:72 [FREE Full text] [CrossRef] [Medline]
- Lyzwinski LN, Caffery L, Bambling M, Edirippulige S. University Students’ Perspectives on Mindfulness and mHealth: A Qualitative Exploratory Study. American Journal of Health Education 2018 Sep 10;49(6):341-353. [CrossRef]
- Phillips SM, Courneya KS, Welch WA, Gavin KL, Cottrell A, Nielsen A, et al. Breast cancer survivors' preferences for mHealth physical activity interventions: findings from a mixed methods study. J Cancer Surviv 2019 Apr;13(2):292-305. [CrossRef] [Medline]
- Herbeć A, Perski O, Shahab L, West R. Smokers' Views on Personal Carbon Monoxide Monitors, Associated Apps, and Their Use: An Interview and Think-Aloud Study. Int J Environ Res Public Health 2018 Feb 07;15(2):288 [FREE Full text] [CrossRef] [Medline]
- Perski O, Baretta D, Blandford A, West R, Michie S. Engagement features judged by excessive drinkers as most important to include in smartphone applications for alcohol reduction: A mixed-methods study. Digit Health 2018;4:2055207618785841 [FREE Full text] [CrossRef] [Medline]
- Utrankar A, Mayo-Gamble TL, Allen W, Novak L, Kassim AA, Bonnet K, et al. Technology use and preferences to support clinical practice guideline awareness and adherence in individuals with sickle cell disease. J Am Med Inform Assoc 2018 May 07:976-988. [CrossRef] [Medline]
- Grunberg PH, Dennis C, Da Costa D, Zelkowitz P. Infertility patients' need and preferences for online peer support. Reprod Biomed Soc Online 2018 Aug;6:80-89 [FREE Full text] [CrossRef] [Medline]
- Willoughby JF, Muldrow A. SMS for sexual health: A comparison of service types and recommendations for sexual health text message service providers. Health Education Journal 2016 Sep 27;76(2):231-243. [CrossRef]
- Hilliard ME, Hahn A, Ridge AK, Eakin MN, Riekert KA. User Preferences and Design Recommendations for an mHealth App to Promote Cystic Fibrosis Self-Management. JMIR Mhealth Uhealth 2014;2(4):e44 [FREE Full text] [CrossRef] [Medline]
- Hartzler AL, BlueSpruce J, Catz SL, McClure JB. Prioritizing the mHealth Design Space: A Mixed-Methods Analysis of Smokers' Perspectives. JMIR Mhealth Uhealth 2016 Aug 05;4(3):e95 [FREE Full text] [CrossRef] [Medline]
- Peters D, Davis S, Calvo RA, Sawyer SM, Smith L, Foster JM. Young People's Preferences for an Asthma Self-Management App Highlight Psychological Needs: A Participatory Study. J Med Internet Res 2017 Apr 11;19(4):e113 [FREE Full text] [CrossRef] [Medline]
- McClure JB, Heffner J, Hohl S, Klasnja P, Catz SL. Design Considerations for mHealth Programs Targeting Smokers Not Yet Ready to Quit: Results of a Sequential Mixed-Methods Study. JMIR Mhealth Uhealth 2017 Mar 10;5(3):e31 [FREE Full text] [CrossRef] [Medline]
- Evans C, Turner K, Suggs LS, Occa A, Juma A, Blake H. Developing a mHealth intervention to promote uptake of HIV testing among African communities in the UK: a qualitative study. BMC Public Health 2016 Dec 28;16:656 [FREE Full text] [CrossRef] [Medline]
- Peng W, Kanthawala S, Yuan S, Hussain SA. A qualitative study of user perceptions of mobile health apps. BMC Public Health 2016 Nov 14;16(1):1158 [FREE Full text] [CrossRef] [Medline]
- Carolan S, de Visser RO. Employees' Perspectives on the Facilitators and Barriers to Engaging With Digital Mental Health Interventions in the Workplace: Qualitative Study. JMIR Ment Health 2018 Jan 19;5(1):e8 [FREE Full text] [CrossRef] [Medline]
- Thornton LK, Kay-Lambkin FJ. Specific features of current and emerging mobile health apps: user views among people with and without mental health problems. Mhealth 2018;4:56 [FREE Full text] [CrossRef] [Medline]
- Saberi P, Siedle-Khan R, Sheon N, Lightfoot M. The Use of Mobile Health Applications Among Youth and Young Adults Living with HIV: Focus Group Findings. AIDS Patient Care STDS 2016 Jun;30(6):254-260 [FREE Full text] [CrossRef] [Medline]
- Aji M, Gordon C, Peters D, Bartlett D, Calvo RA, Naqshbandi K, et al. Exploring User Needs and Preferences for Mobile Apps for Sleep Disturbance: Mixed Methods Study. JMIR Ment Health 2019 May 24;6(5):e13895 [FREE Full text] [CrossRef] [Medline]
- Geuens J, Geurts L, Swinnen TW, Westhovens R, Vanden Abeele V. Mobile Health Features Supporting Self-Management Behavior in Patients With Chronic Arthritis: Mixed-Methods Approach on Patient Preferences. JMIR Mhealth Uhealth 2019 Mar 25;7(3):e12535 [FREE Full text] [CrossRef] [Medline]
- Goldenberg T, McDougal SJ, Sullivan PS, Stekler JD, Stephenson R. Preferences for a Mobile HIV Prevention App for Men Who Have Sex With Men. JMIR Mhealth Uhealth 2014;2(4):e47 [FREE Full text] [CrossRef] [Medline]
- McClure JB, Shortreed SM, Bogart A, Derry H, Riggs K, St John J, et al. The Effect of Program Design on Engagement With an Internet-Based Smoking Intervention: Randomized Factorial Trial. J Med Internet Res 2013 Mar 25;15(3):e69. [CrossRef]
- Wright CJC, Dietze PM, Lim MSC. Beyond Basic Feedback in Mobile Brief Interventions: Designing SMS Message Content for Delivery to Young Adults During Risky Drinking Events. JMIR Mhealth Uhealth 2017 Jun 20;5(6):e79 [FREE Full text] [CrossRef] [Medline]
- Fylan F, Caveney L, Cartwright A, Fylan B. Making it work for me: beliefs about making a personal health record relevant and useable. BMC Health Serv Res 2018 Jun 14;18(1):445 [FREE Full text] [CrossRef] [Medline]
- Gatwood J, Shuvo S, Ross A, Riordan C, Smith P, Gutierrez ML, et al. The Management of Diabetes in Everyday Life (MODEL) program: development of a tailored text message intervention to improve diabetes self-care activities among underserved African-American adults. Transl Behav Med 2020 Feb 03;10(1):204-212. [CrossRef] [Medline]
- Zhao Y, Zhu X, Pérez AE, Zhang W, Shi A, Zhang Z, et al. MHealth approach to promote Oral HIV self-testing among men who have sex with men in China: a qualitative description. BMC Public Health 2018 Sep 27;18(1):1146. [CrossRef]
- Tîrnăucă C, Duque R, Montaña JL. User Interaction Modeling and Profile Extraction in Interactive Systems: A Groupware Application Case Study. Sensors (Basel) 2017 Jul 20;17(7):1669 [FREE Full text] [CrossRef] [Medline]
- Appleton KM, Bray J, Price S, Liebchen G, Jiang N, Mavridis I, et al. A Mobile Phone App for the Provision of Personalized Food-Based Information in an Eating-Out Situation: Development and Initial Evaluation. JMIR Form Res 2019 Nov 04;3(4):e12966 [FREE Full text] [CrossRef] [Medline]
- Digital Health Center of Excellence. US Food and Drug Administration. URL: https://www.fda.gov/medical-devices/digital-health [accessed 2020-03-26]
- Lyles CR, Harris LT, Le T, Flowers J, Tufano J, Britt D, et al. Qualitative evaluation of a mobile phone and web-based collaborative care intervention for patients with type 2 diabetes. Diabetes Technol Ther 2011 May;13(5):563-569. [CrossRef] [Medline]
- Krebs P, Duncan DT. Health App Use Among US Mobile Phone Owners: A National Survey. JMIR Mhealth Uhealth 2015;3(4):e101 [FREE Full text] [CrossRef] [Medline]
- Krug S. Don't Make Me Think!: A Common Sense Approach to Web Usability. Indianapolis, United States: Que Corp; 2000.
- Lambert MJ, Barley DE. Research summary on the therapeutic relationship and psychotherapy outcome. Psychotherapy: Theory, Research, Practice, Training 2001;38(4):357-361. [CrossRef]
- Lowry PB, Wilson DW, Haig WL. A Picture is Worth a Thousand Words: Source Credibility Theory Applied to Logo and Website Design for Heightened Credibility and Consumer Trust. International Journal of Human-Computer Interaction 2013 Nov 27;30(1):63-93. [CrossRef]
- Lorimer K, McDaid L. Young men's views toward the barriers and facilitators of Internet-based Chlamydia trachomatis screening: qualitative study. J Med Internet Res 2013;15(12):e265 [FREE Full text] [CrossRef] [Medline]
- Fogg BJ. Prominence-interpretation theory: Explaining how people assess credibility online. CHI'03 extended abstracts on human factors in computing systems 2003:722-723. [CrossRef]
- Bhavnani SP, Narula J, Sengupta PP. Mobile technology and the digitization of healthcare. Eur Heart J 2016 Feb 11;37(18):1428-1438. [CrossRef]
- Paige SR, Alber JM, Stellefson ML, Krieger JL. Missing the mark for patient engagement: mHealth literacy strategies and behavior change processes in smoking cessation apps. Patient Educ Couns 2018 May;101(5):951-955. [CrossRef] [Medline]
|mHealth: mobile health|
|MMAT: Mixed Methods Appraisal Tool|
Edited by G Eysenbach; submitted 22.06.20; peer-reviewed by S Six, M Rauws; comments to author 10.07.20; revised version received 30.09.20; accepted 24.10.20; published 09.12.20
©Yanxia Wei, Pinpin Zheng, Hui Deng, Xihui Wang, Xiaomei Li, Hua Fu. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 09.12.2020.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.