Published on 12.12.17 in Vol 19, No 12 (2017): December
End User and Implementer Experiences of mHealth Technologies for Noncommunicable Chronic Disease Management in Young Adults: Systematic Review
Background: Chronic noncommunicable diseases (NCDs) such as asthma, diabetes, cancer, and persistent musculoskeletal pain impose an escalating and unsustainable burden on young people, their families, and society. Exploring how mobile health (mHealth) technologies can support management for young people with NCDs is imperative.
Objective: The aim of this study was to identify, appraise, and synthesize available qualitative evidence on users’ experiences of mHealth technologies for NCD management in young people. We explored the perspectives of both end users (young people) and implementers (health policy makers, clinicians, and researchers).
Methods: A systematic review and meta-synthesis of qualitative studies. Eligibility criteria included full reports published in peer-reviewed journals from January 2007 to December 2016, searched across databases including EMBASE, MEDLINE (PubMed), Scopus, and PsycINFO. All qualitative studies that evaluated the use of mHealth technologies to support young people (in the age range of 15-24 years) in managing their chronic NCDs were considered. Two independent reviewers identified eligible reports and conducted critical appraisal (based on the Joanna Briggs Institute Qualitative Assessment and Review Instrument: JBI-QARI). Three reviewers independently, then collaboratively, synthesized and interpreted data through an inductive and iterative process to derive emergent themes across the included data. External validity checking was undertaken by an expert clinical researcher and for relevant content, a health policy expert. Themes were subsequently subjected to a meta-synthesis, with findings compared and contrasted between user groups and policy and practice recommendations derived.
Results: Twelve studies met our inclusion criteria. Among studies of end users (N=7), mHealth technologies supported the management of young people with diabetes, cancer, and asthma. Implementer studies (N=5) covered the management of cognitive and communicative disabilities, asthma, chronic self-harm, and attention deficit hyperactivity disorder. Quality ratings were higher for implementer compared with end user studies. Both complementary and unique user themes emerged. Themes derived for end users of mHealth included (1) Experiences of functionality that supported self-management, (2) Acceptance (technical usability and feasibility), (3) Importance of codesign, and (4) Perceptions of benefit (self-efficacy and empowerment). For implementers, derived themes included (1) Characteristics that supported self-management (functional, technical, and behavior change); (2) Implementation challenges (systems level, service delivery level, and clinical level); (3) Adoption considerations for specific populations (training end users; specific design requirements); and (4) Codesign and tailoring to facilitate uptake and person-centered care.
Conclusions: Synthesizing available data revealed both complementary and unique user perspectives on enablers and barriers to designing, developing, and implementing mHealth technologies to support young people’s management of their chronic NCDs.
Trial Registration: PROSPERO CRD42017056317; http://www.crd.york.ac.uk/PROSPERO/display_record.asp?ID=CRD 42017056317 (Archived by WebCite at http://www.webcitation.org/6vZ5UkKLp)
J Med Internet Res 2017;19(12):e406
Young people are digital natives, and the portability and capabilities of digital technologies can act as a lever to connect them to health systems. This capability to connect is especially important for young people with chronic noncommunicable diseases (NCDs) during the critical transition from childhood to young adulthood [, ].
Young People’s Use of Mobile Technologies to Support Self-Management of Chronic NCDs
We have previously identified how mobile health (mHealth) technologies could support self-management of young people with persistent musculoskeletal pain who are making this transition [, ] and how to specifically address their self-management needs by improving access to disease information, strategies to manage symptoms, and social support [ ]. Self-management is well recognized as a fundamental component of chronic NCD care, denoting the active participation of people in their care with the aim of minimizing the impact of chronic disease on physical health status and functioning and enabling people to cope with the psychological effects of illness [ ]. Core self-management skills include problem solving, decision making, resource utilization, forming patient-health professional relationships, taking action, and self-tailoring, all skills that can be feasibly supported by appropriate mHealth technologies as highlighted in findings from a recent systematic review on this issue [ ]. Furthermore, the use of mHealth technologies as an enabler to self-management is an intuitive choice for young people, given the high rates of Internet usage globally, with rates nearing 100% for the millennial generation in many of the world’s largest economies [ ]. Young people are also more likely than older generations to own a mobile phone in virtually every country [ ]. Digital technologies can also provide a potential mechanism to help mitigate care disparity [ ], reaching across high, middle, and low-income economies [ ] to enable the delivery of integrated, holistic information about chronic NCD management [ ].
Evidence-Practice and Policy-Practice Gaps for the Use of Mobile Health Technologies to Support Self-Management of Chronic NCDs
Although the use of mHealth technologies, including mobile apps, to support self-management of NCDs has also grown substantially , the evaluation of their quality, safety, and outcomes indicate that significant evidence-practice and policy-practice gaps remain [ , , ]. In particular, there is a dearth of high-quality evidence on the use of mHealth technologies to support young people’s self-management of their persistent musculoskeletal pain conditions [ , ]. Recent efforts address some of these gaps, providing evidence for how mHealth apps can improve the access of young people with chronic pain to disease information, facilitate symptom management and social support [ ], and support their self-management of cancer pain [ , ]. In the context of young people’s use of mHealth to support their management of other chronic NCDs (asthma, diabetes, and cancer), findings from a recent systematic review indicate the need for more high-quality studies targeting the development, evaluation, use, and effectiveness of mobile apps [ ]. One significant issue common to mHealth interventions is that they fail to be fully embedded into real-world settings and scaled up, with many studies being conducted as pilots or feasibility trials [ , ]. Another key finding from this same review emphasized the critical role of codesign of mobile apps. This means bringing together both end users (here, young people) and implementers (policy makers or health professionals tasked with implementation) to ensure meaningful design and to facilitate strong engagement, adoption, and sustained uptake [ ]. Codesign includes consideration of factors such as feasibility, engagement, ease of use, ease of navigation, ease of understanding, satisfaction, acceptability, reliability, functionality, aesthetics, information quality, and subjective quality [ , , , , ].
Why This Study?
The primary motivation for this systematic review was to inform appropriate mHealth resource design, evaluation, and implementation specifically targeted for young people with chronic NCDs including persistent musculoskeletal pain. The experiences of young people with chronic NCDs diseases were considered more broadly, as the self-management of chronic conditions frequently overlaps and is associated with comorbidities and multi-morbidities [, ] requiring similar core self-management skills [ ]. To optimally inform implementation approaches, a comprehensive understanding of users’ experiences and perceptions is essential. Qualitative (including mixed methods) studies are likely to provide the richest insights, and such perspectives and insights are recognized as a critical component of implementation approaches related to interventions and system-wide models of care [ , ]. Additionally, as the implementation of new interventions is recommended to be a partnered process between end users and implementers, identifying unique and overlapping user perspectives could lead to better shared decision making and care integration [ ].
This systematic review therefore had two key aims: (1) to identify users’ (end user and implementers) experiences with mHealth technologies to support the self-management of young people with chronic NCDs, and (2) to identify what factors these users (end user and implementers) perceived or experienced as facilitators or barriers to the uptake and implementation of mHealth technologies for young people with chronic NCDs.
Conduct of Systematic Review
This systematic review followed an a priori published protocol with detailed methods . Our review is reported in accordance the Preferred Reporting Items for Systematic Review and Meta-Analysis (PRISMA) statement checklist [ ] and Enhancing Transparency in Reporting the Synthesis of Qualitative Research (ENTREQ) checklist [ ] ( and ). This systematic review followed an a priori published protocol with detailed methods [ ] and can be found at: http://www.crd.york.ac.uk/PROSPERO/ display_record.asp?ID=CRD42017056317.
Types of Participants
This review considered all qualitative studies on young people (in the age range of 15-24 years) with chronic NCDs (end users), which included technologies intended for use by patients . Studies were included where ≥50% of the cohort met the age criteria or where the mean age range (rounded) of participants fell within the 15 to 24 year age range. Additionally, the experiences and perspectives of “Implementers” (defined as including health service delivery providers, administrators, researchers, clinicians, and policy makers) supporting young people with chronic NCDs were included and considered separately.
Chronic NCDs were defined as conditions of long duration and generally slow progression, lasting 3 months or more and included, but were not limited to, musculoskeletal conditions, diabetes, respiratory conditions (such as asthma), cardiovascular diseases, mental health disorders, and cancer .
Phenomena of Interest
This review considered studies that evaluated the use of mHealth technologies to support young people manage their chronic NCDs . To be included, studies needed to have evaluated users’ (implementers and end users) (1) perspectives or experiences (ie, perceptions of feasibility, engagement, ease of use, ease of navigation, ease of understanding, satisfaction, acceptability, reliability, functionality, aesthetics, information quality, and subjective quality) of using mHealth technologies to support the management of chronic NCDs and (2) factors that users (end user and implementers) perceived or experienced as facilitators or barriers to the uptake and/or implementation of mHealth technologies for young people with chronic NCDs [ ]. In this review, mHealth included any mobile device or service, such as mobile phones, short message service (SMS), smartphones, personal digital assistants, and devices that work on wireless technology or Bluetooth-compatible devices [ ]. Interventions delivered using a Web-based platform were included only if it was specified that the patient accessed the service via a mobile phone or other mobile device.
Studies carried out in any setting were considered. The rationale included the portable and accessible nature of mHealth technologies, which enables varied use not just within different care settings by different patients but extending across different contexts by the same patient (ie, continuing to access and utilize the same mobile phone app in the community [locally and remotely] in primary care and tertiary care settings).
Types of Studies
This review considered primary research studies that used qualitative methods to collect and analyze data, including but not limited to phenomenology, grounded theory, ethnography, critical enquiry, participatory action research, and descriptive qualitative studies. The qualitative components of mixed-methods studies were also included.
A three-step search strategy was utilized in this review . An initial limited search of MEDLINE (PubMed) and CINAHL and PsycINFO was to be undertaken, followed by analysis of the text words contained in the title and abstract and the index terms used to describe an article. A second search using all identified keywords and index terms was then undertaken across all databases including EMBASE, MEDLINE (PubMed), Scopus, and PsycINFO. Two independent academic research librarians were consulted to provide feedback on the final search strategy. The search for gray literature included ProQuest Dissertations and Theses, KT, Epistemonikos, as well as health policy and nongovernmental organization literature based on the research team’s knowledge. Third, the reference list of all included reports and articles were hand searched for additional studies. Studies published in English were considered for inclusion in this review. The search was carried out in December 2016 by a senior review methodologist (JC). Studies from 2007 were included to align with global access to 147 Wideband Code-Division Multiple Access; the standard found in third generation mobile telecommunications and available globally [ ].
Initial keywords used were chronic, long term, persistent, noncommunicable, disease, respiratory, asthma, cystic fibrosis, lung disease, diabetes, cancer, heart disease, cardiovascular disease, pain, muscular disease, joint diseases, musculoskeletal, kidney disease, young, adolescent, adolescence, eHealth, mHealth, mobile application, mobile health app, mobile health application, smartphone application, digital technologies, intervention, qualitative, experience, phenomenology, grounded theory, action research, implementation, implementer, and end user. The full search strategies are included in.
Screening and Selection
Search results were collated in a reference database (Endnote X7 version 3.1, Thomson Reuters, New York), duplicates were deleted, and initial screening of titles and abstracts was conducted by one reviewer (JC), followed by the retrieval of full texts. Full texts were then reviewed against the inclusion criteria by two independent reviewers (HS and JC) to confirm eligibility. Disagreements were resolved through discussion.
Assessment of Methodological Quality
Papers selected for retrieval were assessed by two independent reviewers (JC and HS) for methodological quality before inclusion using the standardized critical appraisal instrument for qualitative research from the Joanna Briggs Institute, JBI-QARI . Studies were not excluded on the basis of quality ratings. Any disagreements were resolved through discussion until consensus was reached.
Data were extracted by one reviewer (JC) from papers included in the review using the standardized extraction tool from JBI-QARI . A second reviewer (HS) also completed data extraction for 30% of articles to confirm congruence. The primary focus of data extraction was the identification of specific qualitative findings—reported themes, subthemes, and metaphors—related to the phenomena of interest, which were subsequently synthesized as described below. Additionally, descriptive data, including details about the mHealth apps, study methods, country of development, and age range of participants were extracted.
The credibility of findings was assessed based on how they were supported in the text , as follows:
- Unequivocal: findings accompanied by an illustration that is beyond reasonable doubt and therefore not open to challenge.
- Credible: findings accompanied by an illustration lacking clear association with it and therefore open to challenge.
- Unsupported: findings not supported by data.
A meta-synthesis approach was used to organize and interpret pooled data . Initially, three reviewers (JC, AMB, and HS) familiarized themselves with the extracted data and independently developed preliminary categorizations. At a subsequent 3-day workshop, these independently and deductively derived categories were presented, discussed, and iteratively and inductively organized into consensus-based descriptive themes from which we derived new, higher-order themes that extended beyond the findings of primary studies. Findings were linked back to the research questions to ensure relevance and appropriate contextualization. Themes were then subjected to a meta-synthesis to inform declarative statements that could be applied as an evidence-base to our research aims. Four members of the team (AMB, JC, MB, and HS) participated in the meta-synthesis. Findings based on the experiences of end users and implementers were meta-synthesized separately and compared and contrasted.
On the basis of consensus, a reporting framework was developed to reflect these synthesized findings. The reporting framework was populated with derived themes and supporting evidence from primary study findings. To ensure external validity, one member of the team (JS) with substantial clinical and research expertise in the development and implementation of digital technologies for young people with chronic conditions provided independent feedback over the meta-synthesis process. Where relevant, findings and supporting evidence were adjusted to reflect a consensus decision, and the reporting framework was refined. Finally, a systems and health policy expert (MB) was engaged to assist with final policy and practice recommendations, with a final round of independent review (JS) conducted as outlined previously.
Identification and Selection
The initial search identified 4046 potential studies from which 1193 studies were excluded as duplicates and 2815 were excluded based on the review on their titles or abstracts ().
Overall, 38 studies were identified as potentially meeting the inclusion criteria based on the review of their titles and abstracts. From these, 12 studies were ultimately included [- ]. Reasons for exclusion included not being a research paper [ ], not being qualitative or having a qualitative component [ - ], investigating the wrong phenomena of interest [ - ], not meeting the definition of mHealth [ , ], the population being outside the target age band [ , - ], and the population being affected by a condition not considered to be a chronic NCD (eg, mHealth promotion interventions with no specific chronic NCD or lifestyle behaviors) [ - ]. Seven studies contributed findings on end users [ , , - , , ], whereas 5 studies [ , , , , ] reported on implementers.
Included Study Characteristics
Characteristics of included studies are described in(end user studies) and (implementer studies). Among end users, mHealth technologies were applied to aid in managing diabetes [ , , ], cancer (chemotherapy symptom management) [ , ], and asthma [ , ]. Implementers included occupational therapists [ ], speech language pathologists [ ], nurses [ ], physicians [ , ], as well as medical [ , ] and nonmedical [ , ] health care professionals assisting in the management of cognitive and communicative disabilities [ ], asthma [ , ], chronic self-harm [ ], and attention deficit hyperactivity disorder (ADHD) [ ]. Studies on end users were carried out in the United Kingdom [ , ], United States [ , , ], and Norway [ , ], whereas studies on implementers were conducted in the United Kingdom [ , ], United States [ , ], and Sweden [ ].
End user studies
Phenomena of interest
|mHealtha technology used||Method or designb; AAc|
|Ashurst et al 2014 ||Use of an app to help prepare for clinical appointments||Young people with type 1 diabetes; aged 16 to 25 years; mean age 20.3 years||Apps developed by young people with diabetes to facilitate agenda setting in clinic consultations, data logging and insulin dose calculation||Open-ended questions (email and web-based); |
AA: Inductive conventional content analysis; summative content analysis
|Community; United Kingdom|
|Baggott et al 2012 ||Perceptions about using mobile oncology symptom tracker (mOST) and any technical difficulties they experienced||Adolescents and young adults with cancer; 13 to 21 years; receiving chemotherapy; mean age 18.2 years||A mobile phone–based electronic symptom diary (mOST)||Interviews and questionnaire; |
AA: Not specified
|Pediatric hospitals; inpatient or clinic settings; United States|
|Carpenter et al 2016 ||How app features promote self-observation, self-judgment and foster positive self-reflection; app features work synchronously to increase adolescents’ asthma self-management and improve outcomes||Convenience sample of 20 adolescents with asthma; 12 to 17 years; mean 14.7 years; >50% over 15 years||Two asthma self-management apps (one targeted to adults and one to children)||20 to 30 min telephone interview with verbatim transcription; |
AA: Framework synthesis based on a framework analysis (self-regulation theory)
|Pediatric practice located in an urban area; United States|
|Froisland and Arsand 2015 ||To evaluate the effect of the designed tool with regard to empowerment, self-efficacy, and self-treatment||Adolescents with type 1 diabetes; 13 to 19 years; mean age 16.2 years; >50% over 15 years||Mobile phone–based tool designed to capture and visualize adolescent food intake to affect understanding of calorie counting and help doctor-adolescent communication||Semistructured interview; |
AA: Deductive approach based on empowerment theory
|Pediatric clinic; Norway|
|Froisland et al 2012 ||Adolescent patients’ experiences with two different mobile phone apps used for diabetes care||Adolescents with type 1 diabetes; 13 to 19 years; mean 16.2 years||App that contained a visual or picture-based diabetes diary to record physical activity, food eaten that communicated with glucometer and Web-based SMSd used to contact providers and receive educational messages||Structured interview (transcribed) with field notes; |
AA: Inductive qualitative description influenced by phenomenology and hermeneutics
|Pediatric clinics; Norway|
|Gibson et al 2010 ||Key benefits of the Advanced Symptom Management System (ASyMS-YG)||Young people; inpatient intravenous chemotherapy; 13 to 18 years; median age 15 years; >50% over 15 years)||ASyMS: through which patients can report chemotherapy-related symptoms through mobile||Questionnaires and semistructured interviews; |
AA: Thematic analysis
|Cancer units; United Kingdom|
|Rhee et al 2014 ||Feasibility and user acceptability of mobile phone–based asthma self-management aid for adolescents (mASMAA)||Adolescents with asthmas; |
Adolescent-parent dyads; 13 to 17 years; mean 15.1 years; >50% over 15 years
|mASMAA which facilitates symptom monitoring, treatment adherence, and adolescent patent partnership||Focus groups; semistructured questions (recorded and transcribed); |
AA: Content analysis
|Clinical setting (emergency department and primary care clinics in a university medical center); United States|
amHealth: mobile health.
bQualitative design or study type is specified where explicitly stated within studies, otherwise descriptive detail is provided.
cAA: analytic approach.
dSMS: short message service.
Phenomena of interest
|mHealtha technology used||Method or designb; AAc|
|Buchholz et al 2013 ||Professionals’ views of satisfaction, participation, and involvement in daily life of adolescents and adults with communicative disabilities who tried texting with picture symbols and speech synthesis through mobile phones||Four occupational therapists and three speech language pathologists who had worked with end users (adolescents and adults with cognitive and communicative disabilities using the intervention)||Texting with picture symbols and speech synthesis in mobile phones||Semistructured interview with independent transcription; |
AA: Retrospective qualitative analysis theory influenced by directed content analysis
|Community setting; Sweden|
|Geryk et al 2016 ||The use of attitudes and preferences for asthma mHealth app features among parents and clinicians||20 caregivers and 6 clinicians involved in the care of adolescents with asthma||Two asthma self-management apps (one targeted at adults and one at children)||Questionnaires and interviews; |
AA: Thematic analysis
|Pediatric practices; United States|
|Owens and Charles 2016 ||Barriers to recruitment and implementation of a texting intervention for adolescents who self-harm||Clinicians and service managers working in child and adolescent mental health services (CAHMS) with adolescents who self-harm||An SMS text messaging (short message service), (TeenTEXT) that delivered, scheduled, or prompted personalized messages||Field notes and focus groups; |
AA: Inductive thematic analysis
|CAHMS; United Kingdom|
|Schneider et al 2014 ||Physicians’ views on patient-provider communication with their adolescent asthma patients, mechanisms for relating better with patients, their use of mobile technologies, and willingness to integrate technology in patient care||Residents and attending physicians about mHealth use for adolescents’ management of asthma||Mobile technology for patient care (no one specific tool or technology)||Interviews (with recording and transcription); |
AA: Constant comparative method using a priori codes
|One pediatric group in an urban academic medical center; United States|
|Simons et al 2016 ||To explore patients’ and health care professionals’ views regarding the use of remote monitoring technology (RMT) during medication titration for attention deficit hyperactivity disorder (ADHD)||Health care professionals working with people with ADHD||RMT for people undergoing ADHD medication titration which sent automated text messages (linking to questionnaires)||Exploratory cross-sectional focus group; |
AA: Thematic analysis and charting were used to search for data patterns within and across participant groups
|Four National Health Service mental health providers; United Kingdom|
amHealth: mobile health.
bQualitative design or study type is specified where explicitly stated within studies, otherwise descriptive detail is provided.
cAA: analytic approach.
Methodological Quality Assessment
shows the findings of the critical appraisal for studies of end users (n=7) and implementers (n=5), respectively. Studies on implementers were scored as higher quality than those on end users.
This was particularly true for question 8 on the representation of participant voices, which were adequately represented for all 5 studies on implementers but only for 4 of the 7 studies on end users. Researchers’ cultural or theoretical backgrounds were inconsistently reported (question 6), whereas the impact of the researcher on the research was rarely addressed (question 7).
Data Analysis and Meta-Synthesis
Results of the meta-synthesis are presented below. Data are presented as a synthesized finding with supporting themes and component subthemes (for a summary of themes or subthemes, see). Results are reported separately for end users and implementers. Examples of supporting evidence are provided in Textboxes along with statements about level of credibility. Data were subsequently examined for complementarity, indicating both common and unique user themes, which subsequently informed recommendations for policy and practice. Full supporting data and original findings are presented in and .
End Users’ Experiences and Perspectives
Theme 1. Functionality of mHealth Technology
End users perceived the functionality of mHealth technologies as important; specifically, subthemes related to (1) functionality as an important enabler to supporting self-management and (2) person-centered clinical encounters ().
|Ashurst et al 2014 ||Yk||Y||Y||Y||Y||Nl||N||N||Y||Y|
|Baggott et al 2012 ||Um||U||U||N||U||N||N||N||Y||U|
|Carpenter et al 2016 ||Y||Y||Y||Y||Y||N||N||Y||Y||Y|
|Froisland and Arsand 2015 ||Y||Y||Y||Y||Y||Y||N||N||Y||Y|
|Froisland et al 2012 ||Y||Y||Y||Y||Y||Y||N||Y||Y||Y|
|Gibson et al 2010 ||Y||Y||Y||Y||N||Y||N||Y||Y||Y|
|Rhee et al 2014 ||U||Y||Y||Y||Y||U||N||Y||Y||Y|
|Buchholz et al 2013 ||Y||Y||Y||Y||Y||N||N||Y||Y||Y|
|Geryk et al 2016 ||Y||Y||Y||Y||Y||Y||N||Y||Y||Y|
|Owens and Charles 2016 ||Y||Y||Y||Y||Y||Y||N||Y||Y||Y|
|Schneider et al 2014 ||U||Y||Y||Y||Y||N||N||Y||Y||Y|
|Simons et al 2016 ||Y||Y||Y||Y||Y||Y||Y||Y||Y||Y|
aQ1: Is there congruity between the stated philosophical perspective and the research methodology?
bQ2: Is there congruity between the research methodology and the research question or objectives?
cQ3: Is there congruity between the research methodology and the methods used to collect data?
dQ4: Is there congruity between the research methodology and the representation and analysis of data?
eQ5: Is there congruity between the research methodology and the interpretation of results?
fQ6: Is there a statement locating the researcher culturally or theoretically?
gQ7: Is the influence of the researcher on the research, and vice-versa, addressed?
hQ8: Are participants, and their voices, adequately represented?
iQ9: Is the research ethical according to current criteria or for recent studies, and is there evidence of ethical approval by an appropriate body?
jQ10: Do the conclusions drawn in the research report flow from the analysis, or interpretation, of the data? .
|End users||Functionality of mHealtha technology||mHealth functionality to support self-management|
|mHealth functionality to support young person-centered clinical encounters|
|Acceptance of mHealth technologies||Perceptions of technical usability|
|Perceptions and experiences around acceptability and feasibility|
|The importance of codesign||Intrapersonal factors|
|Perceptions of benefit||Self-efficacy|
|Implementers||mHealth characteristics that support young people’s management of noncommunicable diseases||Functional aspects of design that support end users’ management|
|Technical characteristics can help their delivery of clinical care|
|mHealth can support positive health behavior change|
|Implementation challenges||Micro level factors|
|Meso level factors|
|Macro level factors|
|Adoption of mHealth technologies in a specific young population||The need for training of end users|
|The need for design to facilitate uptake and match social context or peer expectations|
|Codesign and tailoring||Importance of codesign|
|Tailoring to end user needs|
amHealth: mobile health.
mHealth Functionality to Support Self-Management
The functionality of mHealth technologies was perceived as supporting young people’s self-management of a range of NCDs including asthma, diabetes, and cancer. Specifically, the functionality offered by mHealth technologies assisted young people in managing their conditions in a number of different ways. This included the following:
- monitoring their health status and symptom triggers via graphical charting [ ] and sign or symptom awareness using self-checks [ , , ]
- improving their comprehension and understanding of their health condition [ ]
- providing reminders about medication adherence [ ]
- providing ready access to automated tailoring of personal health information related to the management of their condition(s) [ ]
- providing relevant information, support, and reassurance about planning for emergencies and safety issues through prompting timely communication with health professionals [ , , ]
mHealth Functionality to Support Young Person-Centered Clinical Encounters
The functionality of the mHealth technologies supported a young person-centered clinical encounter by enabling accurate and immediately available clinically relevant personal data at a consultation , providing a record of clinical health information to treating practitioners (portability and accuracy of data over a cumulative period of time) [ ], and enabling end users to direct the focus of the clinical encounter [ ].
Theme 2. Acceptance of mHealth Technologies
End users’ acceptance of mHealth technologies was related to two subthemes: (1) technical capability (usability; how it’s working now and how they perceived optimization) and (2) acceptability and feasibility ().
Perceptions of Technical Usability
Users identified technical aspects of the mHealth technologies that affected usability and made suggestions for optimization or improvement as it related to implementation at scale.
Whereas mHealth technologies were perceived as useful to supporting their health needs [, ], especially for tracking functions such as data logging, dose calculation (insulin), and for agenda setting (identifying and remembering what to discuss at appointment in the context of diabetes) [ ], participants also identified the need for specific technical adjustments to better support management of their condition(s) [ , ]. This included bypassing the need for accessing SMS text messaging via an Internet browser on the mobile phone; however, end users preferred a capability to use direct SMS text messaging. Furthermore, end users also reported a preference for having a download availability of the software for use directly on their own mobile phones [ ].
Perceptions and Experiences Around Acceptability and Feasibility
Users identified characteristics of mHealth technologies that aligned with their preferences for disease management support, specifically apps that were intuitive (self-explanatory and simple to understand) and provided practical self-management information that was immediately usable [, , ].
Whereas some features were reported as not relevant or acceptable (eg, a requirement to record peak flow for asthma management) , the use of mHealth technologies was still considered useful and feasible as end users were able to adapt to and accommodate mHealth technology into their routines [ ].
Theme 3. The Importance of Codesign
End users identified the critical importance of codesign of mHealth technologies, which included subthemes based on intra and extra-personal factors considered important to end users  ( ).
Competing time demands and inadequate knowledge of condition-specific triggers and value judgments (such as a perception of already adequate self-management) [, ] were cited as factors that needed to be considered in mHealth technology codesign.
Capacity for tailoring design and making technology more broadly acceptable for end users were important considerations. Understanding disease-specific requirements and young people’s needs around the use of technology for self-management  were deemed important, including design considered within the context of their specific peer or social setting [ ].
Theme 4. Perceptions of Benefit
End users perceived benefits in the use of mHealth technology that included the subthemes of self-efficacy and empowerment ().
End users indicated that mHealth technologies were beneficial and positively influenced their internal sense of control, consistent with improved self-efficacy [, , ].
mHealth technologies were perceived by end users as empowering their NCD self-management skills and knowledge. This was perceived as resulting in increased confidence and more positive perceptions about their ability to better manage their lives [, ] through improving their knowledge and accessibility to health providers [ ].
Implementers’ Experiences and Perspectives
Theme 1. mHealth Characteristics That Support Young People’s Management of NCDs
Implementers identified multiple components of young people’s NCD management that can be supported by mHealth technologies (). Three subthemes emerged: functional aspects of design that support end users’ management, technical characteristics that support clinicians’ delivery of clinical care for young people, and how mHealth can support positive health behavior change.
Functional Aspects of Design That Support End Users’ Management
Implementers identified a range of design features that were perceived to support end users’ management of their conditions. These included the following:
- tracking side effects and symptoms for clinical management [ , , ]
- focusing the agenda for clinical appointments [ , , ]
- reminders for medication adherence and to overcome supply problems [ , ]
- enabling bilateral communication between end users and clinicians [ , , ]
- overcoming communication deficiencies [ ]
- habituation of components of self-management (medication management and adherence [ ])
- providing alerts for end users and their clinicians about deteriorating health conditions [ ]
- remote technology enabling social connectedness and access to health support (motivation, coaching, and providing information to their treating physician) [ , ].
Technical Characteristics Can Help Their Delivery of Clinical Care
Implementers identified several technical features that they believed would assist their delivery of clinical care and optimize their engagement with end users, such as communication reminders (use of medicines and low peak flows) and focusing clinical encounters through more efficient preparation [, ].
mHealth Can Support Positive Health Behavior Change
Implementers perceived mHealth technologies to positively influence end users to independently manage their condition and to facilitate positive health behavior change [, , ] through independent communication [ ], age-related appeal [ ], and providing positive feedback to end users (eg, improved asthma tracking, reminders for medication use and refills, peak flow assessment, and communication to health professionals) [ ].
Theme 2. Implementation Challenges
Important challenges to implementation of mHealth technologies were experienced or perceived by implementers as extending across multiple levels of the health care system. This aligned with three subthemes: challenges at the clinical level (micro), challenges at the service delivery level (meso), and challenges at a systems level (macro;).
Micro Level Factors
Factors identified as barriers to implementation at the clinical level included accuracy of health indicator monitoring  and a limitation of task-specific capability for specific health conditions [ ].
Meso Level Factors
At the organizational level, key factors identified as barriers included the internal regulatory environment of organizations , resource allocation (remuneration and funding) [ ], issues with integration into the current work flow [ , ], organizational climate and readiness for change [ ], and interoperability with existing information and technology infrastructures [ ].
Macro Level Factors
At the systems level, health information security and national or jurisdictional electronic health (eHealth) regulatory frameworks were highlighted as key challenges to implementation of mHealth technologies [, ].
Theme 3. Adoption of mHealth Technologies in a Specific Young Population
Implementers perceived the need for mHealth to be adaptable or tailored for vulnerable populations, referring specifically to young people with cognitive and communicative disability. Two subthemes emerged: (1) the need for training of end users and (2) the need for design to facilitate uptake and match social context or peer expectations ().
The Need for Training of End Users
In a single study, Bucholtz et al  identified that specific training of end users is required to facilitate better uptake or adoption of mHealth technologies in this specific population.
The Need for Design to Facilitate Uptake and Match Social Context or Peer Expectations
Design to facilitate adoption included a focus on mHealth technology supporting end users “blending in” and a capacity to streamline function with their existing technology (eg, software installed on end users’ own mobile phones). Additional considerations were devices that were physically easy to handle, hardware designed to meet specific end user needs (eg, texting with symbols and speech synthesis), and devices that fit well into end users’ daily routines.
Theme 4. Codesign and Tailoring
Implementers perceived specific characteristics of mHealth technologies that they considered important to support end users’ management of NCDs. Two subthemes emerged: (1) the importance of codesign and (2) tailoring to end user needs ().
Importance of Codesign
Implementers identified the importance of working collaboratively with end users to optimize functionality requirements as part of the early phase of development of mHealth technologies [, , ].
Tailoring to End User Needs
Implementers identified the need for the design of mHealth technologies to be adaptable to end users, providing for tailored age-relevant design, content, and functionality [, ], as well as meeting condition-specific requirements [ ].
Policy and Practice Recommendations and Implications
On the basis of our evidence meta-synthesis, we derived five key recommendations and described the associated policy and practice implications (- ). The use of mHealth in management of young people with chronic NCDs can support self-management and drive meaningful change in contemporary health ecosystems. However, identifying and resolving implementation challenges is critical to enabling sustainable scaling-up of mHealth solutions. These recommendations should help to inform appropriate resource design, evaluation, and implementation in a way which all users will find acceptable and which health systems will find sustainable.
This systematic review extends our understanding of users’ experiences and perspectives of mHealth for chronic NCDs management in young people and highlights the specific enablers and barriers to implementation. The clear evidence of benefit for the use of mHealth technologies by young people for education, monitoring, and the self-management of their chronic NCDs often fails to sustainably translate into real-world settings, consistent with reports that “... benefits can only spring from effective implementation that credits interaction with human and organizational factors ” . Our evidence synthesis provides novel insights to inform and guide actionable policy and practice recommendations on “how” we can implement mHealth technologies to better support young people’s management of their chronic NCDs. The key findings from this evidence synthesis also show both complementary and unique perspectives on the use of mHealth for chronic NCD management in young people. Collectively, mHealth technologies were perceived by users as supporting young people’s self-management across a range of chronic NCDs including diabetes [ , , ], cancer (chemotherapy symptom management) [ , ], asthma [ , , ], cognitive and communicative disabilities [ ], chronic self-harm [ ], and ADHD [ ]. No studies were identified that specifically examined persistent musculoskeletal pain.
Complementary perspectives on the use of mHealth technologies to enable young people’s management of NCDs were evident for a number of themes and subthemes. These included codesign of mHealth technologies; functional and technical aspects of mHealth technologies that were person-centered and which aligned with young people’s current technology use (habits, routines, and preferences); and which supported the delivery of clinical care and positive behavior change. The benefits of mHealth use were uniquely perceived by end users (young people) as empowering them to more independently manage their chronic health conditions.
Implementers (specifically clinicians) perceived a great benefit in mHealth affording access to clinical data during consultations and as an enabler to support person-centered clinical encounters. Barriers to the uptake or adoption of mHealth technologies were uniquely identified by implementers as representing “whole of system” (multi-level) factors, including at the clinical level (micro factors), at the organizational level (meso factors), and at the systems level (macro factors). Implementers also identified the need for specific design considerations for mHealth apps for a vulnerable population.
These complementary and unique perspectives highlight both the interdependencies and complexities encountered by different users interacting with a rapidly evolving digital health ecosystem. To interpret our findings and make meaningful recommendations for policy and practice, the use of a design and implementation framework that is plural and pragmatic helps to address such complex interdependencies between human characteristics (users), digital technologies, and health systems.shows the application of such a framework to our synthesized findings (darker shading indicates themes and lighter shading, subthemes) [ ]. This Holistic Framework developed by van Gemert-Pijnen and colleagues [ ] has been widely used to guide the design and implementation of eHealth technologies in chronic care management [ ]. The framework allows for an inherently fluid, iterative, and cyclical nature of design, implementation, and evaluation of digital technologies. We focused on key domains relevant to our findings (contextual enquiry and value specification, design and implementation [operationalization]) [ ]. Given the significant overlay between contextual enquiry and value specification in our data, these were collapsed into a single domain.
Complementary Users’ Perspectives on the Importance of Codesign
Codesign emerged for all users as a fundamental design principle and enabler to the uptake of mHealth technologies. The triangulation between user group perspectives is reflected in the mirroring of themes on codesign, as shown in. These complementary perspectives related to (1) the “contextual enquiry and value specification” domain and (2) the “design” domain. For this reason, codesign is shown in as overlapping both these domains. A formative evaluation loop guides iterations to mHealth technologies during this developmental phase; a step also identified in the primary studies as an important component of mHealth development. Involving end users and other stakeholder user-groups was perceived as critical to ensuring a clear understanding of (1) what the end user wants and needs to best support their self-management (user-friendly, acceptable, meaningful, and safe) and (2) how mHealth technologies could be optimized to meet person-centered needs and support behavior change. Using participatory models of codesign to jointly develop digital technologies that is meaningful to end users, aligns with current recommendations for development and implementation of digital technologies [ , , ]. In a recent study published outside of our search dates, user-centered codesign principles were effectively applied to improve usability (easy to use, easy to understand, efficient to complete, and acceptable) of a real-time mHealth app for adolescents self-managing cancer pain [ ].
Clarity was also deemed important by users around identifying who the required stakeholders would be, what specific roles they would undertake, and at what stages they would be needed. These findings are consistent with recommendations from a recent systematic review of mHealth for NCD management indicating a need for explicit identification of relevant stakeholders as a mechanism to help make sense of eHealth systems for users, to specify mHealth purposes and benefits, and to establish their value, including identifying factors promoting or inhibiting engagement and participation .
Contextual enquiry allows for identification of factors relevant to guiding mHealth design that is acceptable and feasible for end users; a theme that emerged from users reported in the primary studies in our review and more widely reported by others as a critical design factor [, , ]. Contextual factors from our review included value specifications such as the intended use of technology (self-management), the nature of the condition (eg, NCDs, disease status, and level of impairment), the target population (young people), functional requirements (eg, monitoring, medication titration, tracking, decision-support, goal-setting, and cocare), and the care setting (eg, home, school, work, and hospital). Similar factors have been identified in recent systematic reviews of mHealth technology use in NCDs [ , , ]. Implementers’ values were further reflected in their perspectives on the importance of the tailoring capabilities of mHealth to meet end users’ specific condition needs. Organizational needs did not emerge in this review as a key codesign value specification, although contemporary guidance on mHealth technology would suggest this is a critical preimplementation factor [ , ].
Users’ Perspectives on the Importance of mHealth Design Characteristics
Emerging evidence supports use of mHealth for self-management to facilitate clinical interactions and to encourage positive health behaviors . To promote use and adherence, mHealth design needs to reflect meaningful functionality for end users [ , , , ] and to make sense within the context of their daily lives [ ]. Our findings support these recommendations with mHealth functionality identified as a critical design factor by both user groups ( ).
End users’ perceived functional characteristics of mHealth technologies as helping their self-management adherence, including self-tracking, condition self-monitoring (condition status and medication), self-observations providing for early warning of condition flare-ups, self-reflection, improving their understanding of their condition, and providing reassurance by facilitating contact with their health professionals. Implementers’ perspectives similarly recognized meaningful functionalities could assist adherence by leveraging off young people’s habitual use of mHealth technologies. Functionality that extended reach to young people in remote settings, or to those with low accessibility was also perceived by implementers as important; an issue highlighted by us in a study of the gaps and needs of young people with persistent musculoskeletal pain  and consistent with health policy in nations with large care disparity gaps created by geography, such as Australia and Canada.
Functionality characteristics that enabled person-centered care was identified by both user groups as important, including features that focused end users on their condition status and helped them prepare for clinical encounters. From the implementer perspective, technical capabilities were perceived as enablers to supporting their delivery of clinical care. While protecting patient privacy, similar technical capabilities that supported person-centered care by facilitating bilateral communication and which helped the end user focus on the purpose of the clinical encounter were perceived as important. Consistent with these findings, systematic review-level evidence indicates that person-centered care is a key enabler to adoption and adherence of mHealth technologies for self-management [, ]. This person-centered focus is also central to recommendations from contemporary health policy across all settings and economies [ , ].
Implementers described mHealth technologies as helpful in supporting behavior change for young people with NCDs. For example, through sustained engagement of young people by monitoring of their health condition and by providing positive feedback as reinforcement for behavior change. Here, mHealth technologies may be utilized as a catalytic tool for driving sustainable management of NCDs [, ]. However, perceptions and actual outcomes around behavioral change do not necessarily align. More effort and focus is required to understand how mHealth technologies can be used to effect meaningful, sustained behavior change [ , ]. This emerging area requires more than pilot or feasibility studies, arguing for more appropriately designed trials, longer term evaluation, and real-world, population-based health monitoring [ , , ].
Users’ Perspectives on mHealth Technology Implementation Challenges and Solutions
Technical issues associated with real-world use of mHealth technologies impact usability and wider acceptance (end users), scaling-up, and sustainable implementation (implementers;). The need to address recognized technical issues and to optimize mHealth technologies in the “readiness” phase of implementation highlights the critical role for rapid, continuous cycles of evaluation (formative and summative evaluation). Linking design refinements to improve end user experience and to help drive adoption and uptake (ie, implementation “success”) emerged as important for both user groups in our review. Judging “readiness“ and “success” can help mitigate against implementation challenges, and we have derived such a system-level framework that is described comprehensively elsewhere [ ].
From the end user perspective, mHealth apps that are readily accessible and downloadable onto young people’s current mobile devices is an example of one such “readiness” lever [, ], especially if apps align with end users’ habitual routines [ ]. Implementers also highlighted the need for accurate disease monitoring and task-specific capabilities to support young people with unique NCD requirements. These perspectives again emphasize the importance of upstream “readiness” contextual enquiry and value specification as integral to effective codesign and to supporting successful downstream implementation efforts [ ].
Although contemporary health policy reform agendas articulate the need for innovative use of mHealth for NCD management [, , ], currently, very limited processes and frameworks exist to guide the development and implementation [ , , , ]. This challenge resonates with the findings of our review. Many studies consisted of pilot projects or small-scale implementations with evidence of feasibility and acceptability (as per their study aims), however, without extensive consideration of the implementation frameworks needed for building scale. Even with the application of theoretical frameworks to mHealth technologies to gauge scalability (eg, the use of normalization process theory; person-centered design and participatory methods of intervention development), significant barriers to implementation can still stymie uptake [ ]. These same mHealth technology implementation challenges are articulated in reviews of older populations with NCDs [ , ]. In the latter review by Matthew-Maich and colleagues [ ], successful implementation of mHealth required addressing factors across the whole of health systems. Our review found similar “whole of system” factors, including at the micro level (technical factors); at the meso level (organizational, culture, climate, environment, health workforce needs, work flow disruption, technophobia, natural fit for population and health condition, and funding models); and at the macro level (regulatory frameworks, governance, and flexibility; ). These multilevel barriers emphasize the critical importance of taking a system-wide approach to supporting implementation (for comprehensive reviews on implementation, see Briggs et al) [ , ]. Such an approach involves the systematic identification of “readiness” for implementation, as well as postimplementation evaluation of “success” [ ]. This approach aligns well with the Holistic Framework we have adopted here for the specific embedding of mHealth technologies within complex health ecosystems [ ].
Moving mHealth From Promise Into Policy and Practice
It is hard to see a future without mHealth technologies as a complement to a rapidly evolving health care ecosystem. Digital disruption is here. Rather than focusing on barriers and challenges, perhaps we need to seek opportunities for embedding of mHealth within existing health systems where evidence for effectiveness is already well established (eg, self-management) . Further value may be derived from identifying where in health systems, health services, and clinical populations or interfaces potential synergies can be identified that provide a natural “fit” for implementing and building scale in mHealth use [ ]. Here, mHealth can be viewed as a catalytic tool implemented to strengthen health systems [ , ]. In lower and middle-income countries, factors such as a lack of infrastructure, health workforces, resources, and regulatory frameworks have already driven innovative mHealth solutions; for example, using partnerships arrangements and modifications of existing mHealth technologies that can be readily and sustainably implemented [ ]. Implementation guidance and enabling strategies to support mHealth initiatives more broadly is available, for example, in the mHealth assessment and planning for scale toolkit [ ].
Beyond implementation, ongoing evaluation and monitoring of mobile and other digital health interventions is deemed critical to inform health policy and practice [, ]. The World Health Organization provides guidance in this regard from the collective learning of 5 years of engagement with various international lead agencies working to strengthen their digital health deployments, develop robust evaluations, and scale up their activities nationally and regionally [ ].
Strengths and Limitations
The Holistic Framework adopted to underpin the interpretation of our review findings is based on extensive research on the uptake and impact of eHealth technologies and on models for development, implementation, and evaluation . The Framework also provides a level of construct validity to our findings. Whereas consideration was given to alternate implementation frameworks [ ] such as the Consolidated Framework for Implementation Research [ ], technology acceptance model [ ], and normalization process theory [ ], none of these frameworks better satisfied the need for both an integrated whole of system approach and one specifically validated for eHealth applications.
The number of studies in this review provided sufficient data to interrogate our review questions and represented both end users and implementers. The yield was not sufficient, however, to enable meaningful sensitivity analyses to be undertaken based on criteria such as study quality, diseases, settings, or credibility of findings. Most studies used mHealth apps to support self-management and comanagement of young people with NCDs. End users included young people in our age range of interest; however, most were focused at the younger end of this range. Generalizability to other cultures and contexts was limited by the small samples and by cultural and socioeconomic specificity. Our results may not be transferable to low and middle-income economies despite almost ubiquitous use of mobile phones. This represents a critical area of research need given the widespread use of mobile technologies in such global settings and the urgent need to address NCDs through health information and health connectivity at scale [, ]. Implementers were broadly representative of the whole of system; however, health policy makers were not explicitly identified. Although we did not include parents as implementers specifically in our search, for two [ , ] of three possible studies that included parents, their perspectives were captured within pooled implementer data. Explicit parent perspectives may provide important additional insights especially for the younger end of our age range of interest. Data on experiences and perspectives about actual or potential risk and harm associated with use of mHealth technologies were limited, although these are very important factors to consider [ ].
Most studies were of short duration, posing challenges for exploring implementation effectiveness and limiting long-term evaluation of outcomes. The quality of studies was variable, and the use of reporting standards for qualitative research (such as the Consolidated Criteria for Reporting Qualitative Research)  was inconsistent, possibly suggesting a high risk of bias. This raises issues of confidence about internal validity and trustworthiness, making the data extraction, interpretation, and the confidence in evidence more complex. The confidence of reported findings could be readily addressed with the use of a reporting system such as Confidence in the Evidence from Reviews of Qualitative Research [ ]. Another quality indicator that was insufficiently met for most studies was the positioning of the researcher within the research, arguing again for improved reporting against standards. Some studies also provided secondary data interpretation without explicit quotations to support their interpretation, suggesting potential researcher bias. Study designs that better align with the rapid evolution of mHealth technologies are required as randomized trials are expensive, slow, and do not accommodate the dynamic nature of digital technologies, issues also highlighted by others [ , ].
Our evidence meta-synthesis revealed both complementary and unique user perspectives on enablers and barriers to designing, developing, and implementing mHealth technologies to support young people’s management of chronic NCDs. mHealth technologies should be considered as a tool to enable self-management, to improve clinical encounters, and to encourage positive health behaviors. Developing mHealth technologies should involve a genuinely collaborative codesign process between end users and implementers, with the capacity to tailor and adapt technologies to meet person-centered needs. This approach will help to ensure meaningful mHealth solutions for young people, while also supporting implementation efforts. Whole-of-system readiness to adopt mHealth technologies must be considered if implementation initiatives are to be successful and sustainable. Continuous cycles of improvement are needed to maintain technical and functional optimization, ensuring that mHealth solutions remain relevant to young people. The use of contemporary frameworks that support digital health monitoring and provide evaluation guidance is advisable.
This research was supported by grant cofunding awarded from MOVE: muscle, bone and joint health and Arthritis and Osteoporosis Western Australia, with in kind support from the School of Physiotherapy and Exercise Science, Curtin University. The authors wish to thank the staff at Joanna Briggs Institute for their assistance with study design (Micah Peters, Edoardo Aromataris, and Craig Lockwood) and Diana Blackwood and Jayanthi Joseph (Senior Academic Librarians, Curtin University) for assistance with validation of the search strategy. AMB is supported by an NHMRC TRIP Fellowship (#1132548). Joanna Briggs Institute received grant funding to support developing and undertaking the search, screening abstracts, quality appraisal, and data extraction.
HS, JC, JS, and AMB devised the review. JC and HS screened the papers for inclusion. JC extracted data, and HS confirmed congruence. JC and HS appraised the quality of the papers. JC, HS, and AMB developed categorical themes through an inductive analysis. JC, HS, MB, and AMB reflected on and interpreted the categorical themes to develop new themes and apply a meta-synthesis to inform declarative statements that could be applied as an evidence-base. JS provided external validation of the reporting framework. All the authors (HS, JC, JS, MB, and AMB) provided input to policy and practice recommendations and contributed to drafting, revisions, and final manuscript development.
Conflicts of Interest
Multimedia Appendix 1
Preferred reporting items for systematic reviews and meta-analyses (PRISMA) checklist.PDF File (Adobe PDF File), 66KB
Multimedia Appendix 2
Enhancing transparency in reporting the synthesis of qualitative research (ENTREQ) checklist.PDF File (Adobe PDF File), 45KB
Multimedia Appendix 3
Search strategy.PDF File (Adobe PDF File), 54KB
Multimedia Appendix 4
Themed categories for end users’ experiences of mHealth technologies.PDF File (Adobe PDF File), 67KB
Multimedia Appendix 5
Themed categories for implementers’ experiences of mHealth technologies.PDF File (Adobe PDF File), 67KB
- Majeed-Ariss R, Baildam E, Campbell M, Chieng A, Fallon D, Hall A, et al. Apps and adolescents: a systematic review of adolescents' use of mobile phone and tablet apps that support personal management of their chronic or long-term physical conditions. J Med Internet Res 2015 Dec 23;17(12):e287 [FREE Full text] [CrossRef] [Medline]
- Slater H, Jordan J, Chua J, Schütze R, Wark JD, Briggs AM. Young people's experiences of persistent musculoskeletal pain, needs, gaps and perceptions about the role of digital technologies to support their co-care: a qualitative study. BMJ Open 2016;6(12):e014007 [FREE Full text] [CrossRef]
- Slater H, Jordan J, Chua J, Schütze R, Briggs A. MOVE. Melbourne: MOVE muscle bone and joint health; 2016. Young people's experiences of living with persistent pain, their interactions with health services and their needs and preferences for pain management including digital technologies URL: https://www.move.org.au/page/funded-research-completed-projects-young-people-pe [accessed 2017-11-25] [WebCite Cache]
- Stinson JN, Lalloo C, Harris L, Isaac L, Campbell F, Brown S, et al. iCanCope with pain: user-centred design of a web- and mobile-based self-management program for youth with chronic pain based on identified health care needs. Pain Res Manag 2014;19(5):257-265 [FREE Full text] [Medline]
- Lorig KR, Holman H. Self-management education: history, definition, outcomes, and mechanisms. Ann Behav Med 2003 Aug;26(1):1-7. [Medline]
- Pew Research Center. Pew Global. 2016. Smartphone Ownership and Internet Usage Continues to Climb in Emerging Economies: but advanced economies still have higher rates of technology use URL: http://www.pewglobal.org/2016/02/22/smartphone-ownership-and-internet-usage-continues-to-climb-in-emerging-economies/ [accessed 2017-08-31] [WebCite Cache]
- World Health Organization. IRIS. Geneva; 2011. mHealth: New horizons for health through mobile technologies: second global survey on eHealth URL: http://apps.who.int/iris/handle/10665/44607 [accessed 2017-08-30] [WebCite Cache]
- Slater H, Dear BF, Merolli MA, Li LC, Briggs AM. Use of eHealth technologies to enable the implementation of musculoskeletal Models of Care: evidence and practice. Best Pract Res Clin Rheumatol 2016 Jun;30(3):483-502 [FREE Full text] [CrossRef] [Medline]
- Keogh E, Rosser BA, Eccleston C. e-Health and chronic pain management: current status and developments. Pain 2010 Oct;151(1):18-21. [CrossRef] [Medline]
- Riley WT, Rivera DE, Atienza AA, Nilsen W, Allison SM, Mermelstein R. Health behavior models in the age of mobile interventions: are our theories up to the task? Transl Behav Med 2011 Mar;1(1):53-71 [FREE Full text] [CrossRef] [Medline]
- Stoyanov SR, Hides L, Kavanagh DJ, Zelenko O, Tjondronegoro D, Mani M. Mobile app rating scale: a new tool for assessing the quality of health mobile apps. JMIR Mhealth Uhealth 2015;3(1):e27 [FREE Full text] [CrossRef] [Medline]
- Black AD, Car J, Pagliari C, Anandan C, Cresswell K, Bokun T, et al. The impact of eHealth on the quality and safety of health care: a systematic overview. PLoS Med 2011 Jan;8(1):e1000387 [FREE Full text] [CrossRef] [Medline]
- Slater H, Briggs A, Stinson J, Campbell JM. End user and implementer experiences of mHealth technologies for noncommunicable chronic disease management in young adults: a qualitative systematic review protocol. JBI Database System Rev Implement Rep 2017 Aug;15(8):2047-2054. [CrossRef] [Medline]
- Jibb LA, Cafazzo JA, Nathan PC, Seto E, Stevens BJ, Nguyen C, et al. Development of a mHealth real-time pain self-management app for adolescents with cancer: an iterative usability testing study [formula: see text]. J Pediatr Oncol Nurs 2017;34(4):283-294. [CrossRef] [Medline]
- Jibb LA, Stevens BJ, Nathan PC, Seto E, Cafazzo JA, Johnston DL, et al. Implementation and preliminary effectiveness of a real-time pain management smartphone app for adolescents with cancer: a multicenter pilot clinical study. Pediatr Blood Cancer 2017 Oct;64(10):e26554. [CrossRef] [Medline]
- Ossebaard HC, Van Gemert-Pijnen L. eHealth and quality in health care: implementation time. Int J Qual Health Care 2016 Jun;28(3):415-419. [CrossRef] [Medline]
- Heffernan KJ, Chang S, Maclean ST, Callegari ET, Garland SM, Reavley NJ, et al. Guidelines and recommendations for developing interactive eHealth apps for complex messaging in health promotion. JMIR Mhealth Uhealth 2016;4(1):e14 [FREE Full text] [CrossRef] [Medline]
- Matthew-Maich N, Harris L, Ploeg J, Markle-Reid M, Valaitis R, Ibrahim S, et al. Designing, implementing, and evaluating mobile health technologies for managing chronic conditions in older adults: a scoping review. JMIR Mhealth Uhealth 2016 Jun 09;4(2):e29 [FREE Full text] [CrossRef] [Medline]
- Stinson JN, Jibb LA, Nguyen C, Nathan PC, Maloney AM, Dupuis LL, et al. Development and testing of a multidimensional iPhone pain assessment application for adolescents with cancer. J Med Internet Res 2013;15(3):e51 [FREE Full text] [CrossRef] [Medline]
- Dominick CH, Blyth FM, Nicholas MK. Unpacking the burden: understanding the relationships between chronic pain and comorbidity in the general population. Pain 2012 Feb;153(2):293-304. [CrossRef] [Medline]
- Tegethoff M, Belardi A, Stalujanis E, Meinlschmidt G. Comorbidity of mental disorders and chronic pain: chronology of onset in adolescents of a national representative cohort. J Pain 2015 Oct;16(10):1054-1064. [CrossRef] [Medline]
- Nilsen P. Making sense of implementation theories, models and frameworks. Implement Sci 2015;10:53 [FREE Full text] [CrossRef] [Medline]
- Briggs AM, Jordan JE, Jennings M, Speerin R, Bragge P, Chua J, et al. Supporting the evaluation and implementation of musculoskeletal models of care: a globally informed framework for judging readiness and success. Arthritis Care Res (Hoboken) 2017 Apr;69(4):567-577. [CrossRef] [Medline]
- Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Int J Surg 2010;8(5):336-341 [FREE Full text] [CrossRef] [Medline]
- Tong A, Flemming K, McInnes E, Oliver S, Craig J. Enhancing transparency in reporting the synthesis of qualitative research: ENTREQ. BMC Med Res Methodol 2012;12:181 [FREE Full text] [CrossRef] [Medline]
- World Health Organisation. IRIS. Geneva Global status report on non communicable diseases URL: http://www.who.int/nmh/publications/ncd-status-report-2014/en/ [accessed 2017-08-31] [WebCite Cache]
- Hamine S, Gerth-Guyette E, Faulx D, Green BB, Ginsburg AS. Impact of mHealth chronic disease management on treatment adherence and patient outcomes: a systematic review. J Med Internet Res 2015;17(2):e52 [FREE Full text] [CrossRef] [Medline]
- TMCnet. 2007. GSA announces 100 commercial HSDPA networks worldwide URL: http://www.tmcnet.com/usubmit/2007/03/19/2431009.htm [accessed 2017-08-30] [WebCite Cache]
- Joanna Briggs Institute. Adelaide: Joanna Briggs Institute, University of Adelaide; 2014. Joanna Briggs Institute Reviewers’ Manual: 2014 edition URL: https://joannabriggs.org/assets/docs/sumari/ReviewersManual-2014.pdf [accessed 2017-08-30] [WebCite Cache]
- Ashurst EJ, Jones RB, Abraham C, Jenner M, Boddy K, Besser RE, et al. The diabetes app challenge: user-led development and piloting of internet applications enabling young people with diabetes to set the focus for their diabetes consultations. Med 2 0 2014;3(2):e5 [FREE Full text] [CrossRef] [Medline]
- Baggott C, Gibson F, Coll B, Kletter R, Zeltzer P, Miaskowski C. Initial evaluation of an electronic symptom diary for adolescents with cancer. JMIR Res Protoc 2012 Dec 11;1(2):e23 [FREE Full text] [CrossRef] [Medline]
- Buchholz M, Müller I, Ferm U. Text messaging with pictures and speech synthesis for adolescents and adults with cognitive and communicative disabilities - professionals' views about user satisfaction and participation. Technol Disabil 2013;25(2):87-98. [CrossRef]
- Carpenter DM, Geryk LL, Sage A, Arrindell C, Sleath BL. Exploring the theoretical pathways through which asthma app features can promote adolescent self-management. Transl Behav Med 2016 Dec;6(4):509-518. [CrossRef] [Medline]
- Frøisland DH, Årsand E. Integrating visual dietary documentation in mobile-phone-based self-management application for adolescents with type 1 diabetes. J Diabetes Sci Technol 2015 May;9(3):541-548 [FREE Full text] [CrossRef] [Medline]
- Frøisland DH, Arsand E, Skårderud F. Improving diabetes care for young people with type 1 diabetes through visual learning on mobile phones: mixed-methods study. J Med Internet Res 2012;14(4):e111 [FREE Full text] [CrossRef] [Medline]
- Geryk LL, Roberts CA, Sage AJ, Coyne-Beasley T, Sleath BL, Carpenter DM. Parent and clinician preferences for an asthma app to promote adolescent self-management: a formative study. JMIR Res Protoc 2016 Dec 06;5(4):e229 [FREE Full text] [CrossRef] [Medline]
- Gibson F, Aldiss S, Taylor RM, Maguire R, McCann L, Sage M, et al. Utilization of the Medical Research Council evaluation framework in the development of technology for symptom management: the ASyMS-YG Study. Cancer Nurs 2010;33(5):343-352. [CrossRef] [Medline]
- Owens C, Charles N. Implementation of a text-messaging intervention for adolescents who self-harm (TeenTEXT): a feasibility study using normalisation process theory. Child Adolesc Psychiatry Ment Health 2016;10:14 [FREE Full text] [CrossRef] [Medline]
- Rhee H, Allen J, Mammen J, Swift M. Mobile phone-based asthma self-management aid for adolescents (mASMAA): a feasibility study. Patient Prefer Adherence 2014;8:63-72 [FREE Full text] [CrossRef] [Medline]
- Schneider T, Panzera AD, Martinasek M, McDermott R, Couluris M, Lindenberger J, et al. Physicians' perceptions of mobile technology for enhancing asthma care for youth. J Child Health Care 2014 Nov 26;20(2):153-163. [CrossRef] [Medline]
- Simons L, Valentine A, Falconer C, Groom M, Daley D, Craven M. Developing mHealth remote monitoring technology for attention deficit hyperactivity disorder: a qualitative study eliciting user priorities and needs. JMIR Mhealth Uhealth Mar 23 2016;4(1):e31. [CrossRef] [Medline]
- Killackey E, Anda AL, Gibbs M, Alvarez-Jimenez M, Thompson A, Sun P, et al. Using internet enabled mobile devices and social networking technologies to promote exercise as an intervention for young first episode psychosis patients. BMC Psychiatry 2011 May 12;11:80 [FREE Full text] [CrossRef] [Medline]
- Baggott C, Miaskowski C. Feasibility and usability testing of an electronic symptom diary for teens with cancer. In: Commun Nurs Res Spring.: Western Institute of Nursing; 2012 Presented at: 2013 Western Institute of Nursing Annual Communicating Nursing Research Conference; April 2013; Anaheim p. 135 URL: https://www.researchgate.net/publication/268146242_Feasibility_And_Usability_Testing_Of_An_Electronic_Symptom_Diary_For_Teens_With_Cancer
- Ammerlaan J, van Os-Medendorp H, Scholtus L, de Vos A, Zwier M, Bijlsma H, et al. Feasibility of an online and a face-to-face version of a self-management program for young adults with a rheumatic disease: experiences of young adults and peer leaders. Pediatr Rheumatol Online J 2014;12:10 [FREE Full text] [CrossRef] [Medline]
- Campbell JE, Morgan M, Barnett V, Spreat S. Handheld devices and video modeling to enhance the learning of self-help skills in adolescents with autism spectrum disorder. OTJR (Thorofare N J) 2015 Apr;35(2):95-100. [CrossRef] [Medline]
- Carroll AE, Marrero DG, Downs SM. The HealthPia GlucoPack Diabetes phone: a usability study. Diabetes Technol Ther 2007 Apr;9(2):158-164. [CrossRef] [Medline]
- Franklin VL, Greene A, Waller A, Greene SA, Pagliari C. Patients' engagement with “Sweet Talk” - a text messaging support system for young people with diabetes. J Med Internet Res 2008;10(2):e20 [FREE Full text] [CrossRef] [Medline]
- Prakasam G, Rees C, Lyden M, Parkin CG. Use of a novel smartphone-based diabetes management system improved feelings of confidence and safety and reduced hypoglycemia fear among parents/caregivers of children/adolescents with type 1 diabetes. J Diabetes Sci Technol 2017 Jan;11(1):182-183 [FREE Full text] [CrossRef] [Medline]
- Wakefield CE, Sansom-Daly UM, McGill BC, Ellis SJ, Doolan EL, Robertson EG, et al. Acceptability and feasibility of an e-mental health intervention for parents of childhood cancer survivors: “Cascade”. Support Care Cancer 2016 Dec;24(6):2685-2694. [CrossRef] [Medline]
- Zhao Y, Calvo R, Pardo A, Gunn H, Steinbeck K. What we learned from TransitionMate: A mobile app designed to support young people with chronic illness. In: Proceedings of the Annual Meeting of the Australian Special Interest Group for Computer Human Interaction. 2015 Presented at: OzCHI 2015; December 07 - 10, 2015; Parkville, VIC, Australia p. 162-166 URL: http://dl.acm.org/citation.cfm?id=2838739 [CrossRef]
- Cafazzo JA, Casselman M, Hamming N, Katzman DK, Palmert MR. Design of an mHealth app for the self-management of adolescent type 1 diabetes: a pilot study. J Med Internet Res 2012;14(3):e70 [FREE Full text] [CrossRef] [Medline]
- Gibson F, Aldiss S, Taylor R, Maguire R, Kearney N. Involving health professionals in the development of an advanced symptom management system for young people: the ASyMS-YG study. Eur J Oncol Nurs 2009 Jul;13(3):187-192. [CrossRef] [Medline]
- Ranney ML, Choo EK, Cunningham RM, Spirito A, Thorsen M, Mello MJ, et al. Acceptability, language, and structure of text message-based behavioral interventions for high-risk adolescent females: a qualitative study. J Adolesc Health 2014 Jul;55(1):33-40 [FREE Full text] [CrossRef] [Medline]
- Ammerlaan JJ, Scholtus LW, Drossaert CH, van Os-Medendorp H, Prakken B, Kruize AA, et al. Feasibility of a website and a hospital-based online portal for young adults with juvenile idiopathic arthritis: views and experiences of patients. JMIR Res Protoc 2015;4(3):e102 [FREE Full text] [CrossRef] [Medline]
- Gibson F, Miller M, Kearney N. Technology into practice: young people's, parents' and nurses' perceptions of WISECARE+. Pediatr Nurs 2007;19(10):31-34. [Medline]
- Aldiss S, Taylor R, Soanes L, Maguire R, Sage M, Kearney N, et al. Working in collaboration with young people and health professionals. a staged approach to the implementation of a randomised controlled trial. J Res Nurs 2011 Nov;16(6):561-576. [CrossRef]
- Anderson K, Burford O, Emmerton L. Mobile health apps to facilitate self-care: a qualitative study of user experiences. PLoS One 2016;11(5):e0156164 [FREE Full text] [CrossRef] [Medline]
- Brodin J. Can ICT give children with disabilities equal opportunities in school? Improv Sch 2010 Apr 22;13(1):99-112. [CrossRef]
- Cushing A, Manice MP, Ting A, Parides MK. Feasibility of a novel mHealth management system to capture and improve medication adherence among adolescents with asthma. Patient Prefer Adherence 2016;10:2271-2275 [FREE Full text] [CrossRef] [Medline]
- Mayberry LS, Berg CA, Harper KJ, Osborn CY. The design, usability, and feasibility of a family-focused diabetes self-care support mHealth intervention for diverse, low-income adults with type 2 diabetes. J Diabetes Res 2016;2016:7586385. [CrossRef] [Medline]
- Schneider T, Panzera AD, Couluris M, Lindenberger J, McDermott R, Bryant CA. Engaging teens with asthma in designing a patient-centered mobile app to aid disease self-management. Telemed J E Health 2015 Aug 10:170-176 Epub ahead of print. [CrossRef] [Medline]
- Smith KL, Kerr DA, Fenner AA, Straker LM. Adolescents just do not know what they want: a qualitative study to describe obese adolescents' experiences of text messaging to support behavior change maintenance post intervention. J Med Internet Res 2014;16(4):e103 [FREE Full text] [CrossRef] [Medline]
- Bohleber L, Crameri A, Eich-Stierli B, Telesko R, von WA. Can we foster a culture of peer support and promote mental health in adolescence using a web-based app? A control group study. JMIR Ment Health 2016 Sep 23;3(3):e45 [FREE Full text] [CrossRef] [Medline]
- Carey T, Haviland J, Tai S, Vanags T, Mansell W. MindSurf: a pilot study to assess the usability and acceptability of a smartphone app designed to promote contentment, wellbeing, and goal achievement. BMC Psychiatry 2016 Dec 12;16(1):442 [FREE Full text] [CrossRef] [Medline]
- Ybarra ML, Holtrop JS, Prescott TL, Strong D. Process evaluation of a mHealth program: lessons learned from Stop My Smoking USA, a text messaging-based smoking cessation program for young adults. Patient Educ Couns 2014 Nov;97(2):239-243. [CrossRef] [Medline]
- O'Brien C. Marquette University ePublications. Ann Arbor: Marquette University; 2013. mPeer: A Mobile Health Approach to Monitoring PTSD in Veterans URL: http://epublications.marquette.edu/cgi/viewcontent.cgi?article=1230&context=theses_open [WebCite Cache]
- World Health Organization. 2015. The MAPS Toolkit: mHealth Assessment and Planning for Scale URL: http://who.int/life-course/publications/mhealth-toolkit/en/ [accessed 2017-08-31] [WebCite Cache]
- World Health Organization. Geneva: World Health Organization; 2016. Monitoring and evaluating digital health interventions: a practical guide to conducting research and assessment URL: http://apps.who.int/iris/bitstream/10665/252183/1/9789241511766-eng.pdf [accessed 2017-08-30] [WebCite Cache]
- van Gemert-Pijnen JE, Nijland N, van LM, Ossebaard HC, Kelders SM, Eysenbach G, et al. A holistic framework to improve the uptake and impact of eHealth technologies. J Med Internet Res 2011;13(4):e111 [FREE Full text] [CrossRef] [Medline]
- Mair FS, May C, O'Donnell C, Finch T, Sullivan F, Murray E. Factors that promote or inhibit the implementation of e-health systems: an explanatory systematic review. Bull World Health Organ 2012 May 1;90(5):357-364 [FREE Full text] [CrossRef] [Medline]
- Wang Y, Xue H, Huang Y, Huang L, Zhang D. A systematic review of application and effectiveness of mHealth interventions for obesity and diabetes treatment and self-management. Adv Nutr 2017 May;8(3):449-462. [CrossRef] [Medline]
- Hyder AA, Wosu AC, Gibson DG, Labrique AB, Ali J, Pariyo GW. Noncommunicable disease risk factors and mobile phones: a proposed research agenda. J Med Internet Res 2017 May 05;19(5):e133 [FREE Full text] [CrossRef] [Medline]
- Pham Q, Wiljer D, Cafazzo JA. Beyond the randomized controlled trial: a review of alternatives in mHealth clinical trial methods. JMIR Mhealth Uhealth 2016 Sep 09;4(3):e107 [FREE Full text] [CrossRef] [Medline]
- Australian Health Ministers' Advisory Council. Health.gov. Canberra: Australian Government; 2017. National Strategic Framework for Chronic Conditions URL: http://www.health.gov.au/internet/main/publishing.nsf/content/A0F1B6D61796CF3DCA257E4D001AD4C4/$File/National%20Strategic%20Framework%20for%20Chronic%20Conditions.pdf [accessed 2017-08-30] [WebCite Cache]
- Agarwal S, LeFevre AE, Lee J, L'Engle K, Mehl G, Sinha C, et al. Guidelines for reporting of health interventions using mobile phones: mobile health (mHealth) evidence reporting and assessment (mERA) checklist. Br Med J 2016;352:i1174. [Medline]
- Möhler R, Köpke S, Meyer G. Criteria for reporting the development and evaluation of complex interventions in healthcare: revised guideline (CReDECI 2). Trials 2015 May 03;16:204 [FREE Full text] [CrossRef] [Medline]
- Briggs AM, Chan M, Slater H. Models of care for musculoskeletal health: moving towards meaningful implementation and evaluation across conditions and care settings. Best Pract Res Clin Rheumatol 2016 Jun;30(3):359-374. [CrossRef] [Medline]
- Briggs AM, Chan M, Slater H. Extending evidence to practice: implementation of Models of Care for musculoskeletal health conditions across settings. Best Pract Res Clin Rheumatol 2016 Jun;30(3):357-358. [CrossRef] [Medline]
- Labrique A, Blynn E, Ahmed S, Gibson D, Pariyo G, Hyder AA. Health surveys using mobile phones in developing countries: automated active strata monitoring and other statistical considerations for improving precision and reducing biases. J Med Internet Res 2017 May 05;19(5):e121 [FREE Full text] [CrossRef] [Medline]
- Gehring ND, McGrath P, Wozney L, Soleimani A, Bennett K, Hartling L, et al. Pediatric eMental healthcare technologies: a systematic review of implementation foci in research studies, and government and organizational documents. Implement Sci 2017 Jun 21;12(1):76 [FREE Full text] [CrossRef] [Medline]
- Michie S, Yardley L, West R, Patrick K, Greaves F. Developing and evaluating digital interventions to promote behavior change in health and health care: recommendations resulting from an international workshop. J Med Internet Res 2017 Jun 29;19(6):e232 [FREE Full text] [CrossRef] [Medline]
- Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci 2009;4:50 [FREE Full text] [CrossRef] [Medline]
- Holden RJ, Karsh B. The technology acceptance model: its past and its future in health care. J Biomed Inform 2010 Feb;43(1):159-172 [FREE Full text] [CrossRef] [Medline]
- Lim KK, Chan M, Navarra S, Haq SA, Lau CS. Development and implementation of Models of Care for musculoskeletal conditions in middle-income and low-income Asian countries. Best Pract Res Clin Rheumatol 2016 Jun;30(3):398-419. [CrossRef] [Medline]
- Atun R, Jaffar S, Nishtar S, Knaul FM, Barreto ML, Nyirenda M, et al. Improving responsiveness of health systems to non-communicable diseases. Lancet 2013 Feb 23;381(9867):690-697. [CrossRef] [Medline]
- Suris JC, Akre C, Piguet C, Ambresin AE, Zimmermann G, Berchtold A. Is internet use unhealthy? A cross-sectional study of adolescent Internet overuse. Swiss Med Wkly 2014;144:w14061 [FREE Full text] [CrossRef] [Medline]
- Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care 2007 Dec;19(6):349-357 [FREE Full text] [CrossRef] [Medline]
- Lewin S, Glenton C, Munthe-Kaas H, Carlsen B, Colvin CJ, Gülmezoglu M, et al. Using qualitative evidence in decision making for health and social interventions: an approach to assess confidence in findings from qualitative evidence syntheses (GRADE-CERQual). PLoS Med 2015 Oct;12(10):e1001895 [FREE Full text] [CrossRef] [Medline]
|ADHD: attention deficit hyperactivity disorder|
|eHealth: electronic health|
|JBI-QARI: Joanna Briggs Institute, Meta-Analysis of Statistics Assessment and Review Instrument|
|mHealth: mobile health|
|NCD: noncommunicable disease|
|SMS: short message service|
Edited by G Eysenbach; submitted 02.09.17; peer-reviewed by J McDonagh, J Ploeg; comments to author 25.10.17; revised version received 02.11.17; accepted 02.11.17; published 12.12.17
©Helen Slater, Jared M Campbell, Jennifer N Stinson, Megan M Burley, Andrew M Briggs. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 12.12.2017.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.