Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Monday, March 11, 2019 at 4:00 PM to 4:30 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?


Journal Description

The Journal of Medical Internet Research (JMIR), now in its 20th year, is the pioneer open access eHealth journal and is the flagship journal of JMIR Publications. It is the leading digital health journal globally in terms of quality/visibility (Impact Factor 2018: 4.945, ranked #1 out of 26 journals in the medical informatics category) and in terms of size (number of papers published). The journal focuses on emerging technologies, medical devices, apps, engineering, and informatics applications for patient education, prevention, population health and clinical care. As a leading high-impact journal in its disciplines (health informatics and health services research), it is selective, but it is now complemented by almost 30 specialty JMIR sister journals, which have a broader scope. Peer-review reports are portable across JMIR journals and papers can be transferred, so authors save time by not having to resubmit a paper to different journals. 

As an open access journal, we are read by clinicians, allied health professionals, informal caregivers, and patients alike, and have (as with all JMIR journals) a focus on readable and applied science reporting the design and evaluation of health innovations and emerging technologies. We publish original research, viewpoints, and reviews (both literature reviews and medical device/technology/app reviews).

We are also a leader in participatory and open science approaches, and offer the option to publish new submissions immediately as preprints, which receive DOIs for immediate citation (eg, in grant proposals), and for open peer-review purposes. We also invite patients to participate (eg, as peer-reviewers) and have patient representatives on editorial boards.

Be a widely cited leader in the digitial health revolution and submit your paper today!


Recent Articles:

  • Source: Image created by the Authors; Copyright: Jens N Nygren; URL:; License: Creative Commons Attribution (CC-BY).

    Barriers and Enablers Affecting Successful Implementation of the Electronic Health Service Sisom: Multicenter Study of Child Participation in Pediatric Care


    Background: Children’s participation in health care is one of the most important components in the management of their disease. Electronic health (eHealth) services that are adapted to the needs of children have the potential for restructuring how children and professionals work together. Therefore, a digital interactive assessment and communication tool, Sisom, was developed to give children aged between 6 and 12 years a voice in their own health care. However, the implementation of eHealth services such as Sisom in daily practice in pediatric health care is rarely investigated. Objective: The aim of this study was to explore the process of implementing Sisom for children in pediatric care in Sweden. More specifically, the study aimed to (1) evaluate whether the implementation strategy was conducted as planned, (2) understand the barriers and facilitators of the implementation strategy in pediatric care settings, (3) gain insight into how professionals work with the specific intervention, and (4) gain insight into the usefulness and effects of the intervention from the professionals’ perspectives. Methods: A process evaluation design was used to study the implementation of Sisom at 4 pediatric care centers in Sweden. An extensive amount of qualitative and quantitative data was collected before, during, and after the intervention through self-report checklists, memos, and interviews with professionals. In total, 46 children, aged between 6 and 13 years, participated. The children used Sisom on two occasions during 6 months. When they used Sisom, a printed report formed the basis for a forthcoming dialogue between professionals, children, and their parents. Results: To our knowledge, this is the first implementation study of an eHealth communication tool aimed at strengthening children’s participation in pediatric health care. Key factors for successful implementation were alignment of the solution with the values and goals of the organization, health care professionals’ beliefs in the usefulness and usability of the solution, and health care professionals’ willingness to change their professional roles guided by the solution. Conclusions: The results from the study show that it is possible to restructure health care delivery toward a child-centered approach, if there is a willingness and preparedness in the organization to implement an eHealth solution with the aim of restructuring the way of working with children’s participation.

  • Telemedicine. Source: Freepik (pressfoto) / Placeit; Copyright: JMIR Publications; URL:; License: Creative Commons Attribution (CC-BY).

    On-Demand Telemedicine as a Disruptive Health Technology: Qualitative Study Exploring Emerging Business Models and Strategies Among Early Adopter...


    Background: On-demand telemedicine is increasingly adopted by health organizations to meet patient demand for convenient, accessible, and affordable services. Little guidance is currently available to new entrant organizations as they consider viable business models and strategies to harness the disruptive potential of on-demand telemedicine services (in particular, virtual urgent care clinics [VCCs] as a predominant and catalyst form of on-demand telemedicine). Objective: We recognized on-demand telemedicine as a disruptive technology to explore the experiences of early adopter organizations as they launch on-demand telemedicine services and deploy business models and strategies. Focusing on VCC service lines, this study addressed the following research questions: (1) what is the emerging business model being deployed for on-demand telemedicine?; (2) what are the core components of the emerging business model for on-demand telemedicine?; and (3) what are the disruptive business strategies employed by early adopter organizations as they launch on-demand telemedicine services? Methods: This qualitative study gathered data from 32 semistructured phone interviews with key informants from 19 VCC early adopter organizations across the United States. Interview protocols were developed based on noted dissemination and implementation science frameworks. We used the constant comparison method to transform study data into stable dimensions that revealed emerging business models, core business model components (value proposition, key resources, key processes, and profit formula), and accompanying business strategies. Results: Early adopters are deploying business models that most closely align with a value-adding process model archetype. By and large, we found that this general model appropriately matches resources, processes, and profit formulas to support the disruptive potential of on-demand telemedicine. In total, 4 business strategy areas were discovered to particularly contribute to business model success for on-demand disruption among early adopters: fundamental disruptions to the model of care delivery; outsourcing support for on-demand services; disruptive market strategies to target potential users; and new and unexpected organizational partnerships to increase return on investment. Conclusions: On-demand telemedicine is a potentially disruptive innovation currently in the early adopter stage of technology adoption and diffusion. On-demand telemedicine must cross into the early majority stage to truly be a positive disruption that will increase accessibility and affordability for health care consumers. Our findings provide guidance for adopter organizations as they seek to deploy viable business models and successful strategies to smooth the transition to early majority status. We present important insights for both early adopters and potential early majority organizations to better harness the disruptive potential of on-demand telemedicine.

  • Source:; Copyright: Porapak Apichodilok; URL:; License: Licensed by JMIR.

    A Scale to Assess the Methodological Quality of Studies Assessing Usability of Electronic Health Products and Services: Delphi Study Followed by Validity and...


    Background: The usability of electronic health (eHealth) and mobile health apps is of paramount importance as it impacts the quality of care. Methodological quality assessment is a common practice in the field of health for different designs and types of studies. However, we were unable to find a scale to assess the methodological quality of studies on the usability of eHealth products or services. Objective: This study aimed to develop a scale to assess the methodological quality of studies assessing usability of mobile apps and to perform a preliminary analysis of of the scale’s feasibility, reliability, and construct validity on studies assessing usability of mobile apps, measuring aspects of physical activity. Methods: A 3-round Delphi panel was used to generate a pool of items considered important when assessing the quality of studies on the usability of mobile apps. These items were used to write the scale and the guide to assist its use. The scale was then used to assess the quality of studies on usability of mobile apps for physical activity, and it assessed in terms of feasibility, interrater reliability, and construct validity. Results: A total of 25 experts participated in the Delphi panel, and a 15-item scale was developed. This scale was shown to be feasible (time of application mean 13.10 [SD 2.59] min), reliable (intraclass correlation coefficient=0.81; 95% CI 0.55-0.93), and able to discriminate between low- and high-quality studies (high quality: mean 9.22 [SD 0.36]; low quality: mean 6.86 [SD 0.80]; P=.01). Conclusions: The scale that was developed can be used both to assess the methodological quality of usability studies and to inform its planning.

  • Source: flickr; Copyright: Marco Verch; URL:; License: Creative Commons Attribution (CC-BY).

    Reducing Potentially Inappropriate Prescriptions for Older Patients Using Computerized Decision Support Tools: Systematic Review


    Background: Older adults are more vulnerable to polypharmacy and prescriptions of potentially inappropriate medications. There are several ways to address polypharmacy to prevent its occurrence. We focused on computerized decision support tools. Objective: The available literature was reviewed to understand whether computerized decision support tools reduce potentially inappropriate prescriptions or potentially inappropriate medications in older adult patients and affect health outcomes. Methods: Our systematic review was conducted by searching the literature in the MEDLINE, CENTRAL, EMBASE, and Web of Science databases for interventional studies published through February 2018 to assess the impact of computerized decision support tools on potentially inappropriate medications and potentially inappropriate prescriptions in people aged 65 years and older. Results: A total of 3756 articles were identified, and 16 were included. More than half (n=10) of the studies were randomized controlled trials, one was a crossover study, and five were pre-post intervention studies. A total of 266,562 participants were included; of those, 233,144 participants were included and assessed in randomized controlled trials. Intervention designs had several different features. Computerized decision support tools consistently reduced the number of potentially inappropriate prescriptions started and mean number of potentially inappropriate prescriptions per patient. Computerized decision support tools also increased potentially inappropriate prescriptions discontinuation and drug appropriateness. However, in several studies, statistical significance was not achieved. A meta-analysis was not possible due to the significant heterogeneity among the systems used and the definitions of outcomes. Conclusions: Computerized decision support tools may reduce potentially inappropriate prescriptions and potentially inappropriate medications. More randomized controlled trials assessing the impact of computerized decision support tools that could be used both in primary and secondary health care are needed to evaluate the use of medication targets defined by the Beers or STOPP (Screening Tool of Older People’s Prescriptions) criteria, adverse drug reactions, quality of life measurements, patient satisfaction, and professional satisfaction with a reasonable follow-up, which could clarify the clinical usefulness of these tools. Clinical Trial: International Prospective Register of Systematic Reviews (PROSPERO) CRD42017067021;

  • Source: Image created by the Authors; Copyright: The Authors; URL:; License: Licensed by the authors.

    Associations Between Characteristics of Web-Based Diabetes News and Readers’ Sentiments: Observational Study in the Netherlands


    Background: Although experts agree that Web-based health information often contains exaggeration and misrepresentation of science, it is not yet known how this information affects the readers’ sentiments. Objective: This study aimed to investigate whether specific aspects of Web-based diabetes research news are associated with positive or negative sentiments in readers. Methods: A retrospective observational study of the comments on diabetes research news posted on Facebook pages was conducted as a function of the innovations’ developmental phase, the intended treatment effect, and the use of strong language to intensify the news messages (superlatives). Data for the investigation were drawn from the diabetes research news posted between January 2014 and January 2018 on the two largest Dutch Facebook pages on diabetes and the corresponding reader comments. By manually coding these Facebook user comments, three binary outcome variables were created, reflecting the presence of a positive sentiment, the presence of a negative sentiment, and the presence of a statement expressing hopefulness. Results: Facebook users made a total of 3710 comments on 173 diabetes research news posts that were eligible for further analysis. Facebook user comments on posts about diabetes prevention (odds ratio [OR] 0.55, 95% CI 0.37-0.84), improved blood glucose regulation (OR 0.68, 95% CI 0.56-0.84), and symptom relief (OR 0.31, 95% CI 0.21-0.44) were associated with less positive sentiments as compared with potential diabetes cures. Furthermore, comments on innovations supported by preclinical evidence in animals were associated with more positive sentiments (OR 1.46, 95% CI 1.07-1.99) and statements expressing hope (OR 1.47, 95% CI 1.01-2.14), when compared with innovations that have evidence from large human trials. This study found no evidence for the associations between language intensification of the news posts and the readers’ sentiments. Conclusions: Our finding that the attitudes toward diabetes research news on Facebook are most positive when clinical efficacy is not (or not yet) proven in large patient trials suggests that news authors and editors, as well as medical professionals, must exercise caution when acting as a conduit for diabetes research news.

  • Leveraging Interdisciplinary Education Toward Securing the Future of Connected Health Research in Europe: Qualitative Study


    Background: Connected health (CH) technologies have resulted in a paradigm shift, moving health care steadily toward a more patient-centered delivery approach. CH requires a broad range of disciplinary expertise from across the spectrum to work in a cohesive and productive way. Building this interdisciplinary relationship at an earlier stage of career development may nurture and accelerate the CH developments and innovations required for future health care. Objective: This study aimed to explore the perceptions of interdisciplinary CH researchers regarding the design and delivery of an interdisciplinary education (IDE) module for disciplines currently engaged in CH research (engineers, computer scientists, health care practitioners, and policy makers). This study also investigated whether this module should be delivered as a taught component of an undergraduate, master’s, or doctoral program to facilitate the development of interdisciplinary learning. Methods: A qualitative, cross-institutional, multistage research approach was adopted, which involved a background study of fundamental concepts, individual interviews with CH researchers in Greece (n=9), and two structured group feedback sessions with CH researchers in Ireland (n=10/16). Thematic analysis was used to identify the themes emerging from the interviews and structured group feedback sessions. Results: A total of two sets of findings emerged from the data. In the first instance, challenges to interdisciplinary work were identified, including communication challenges, divergent awareness of state-of-the-art CH technologies across disciplines, and cultural resistance to interdisciplinarity. The second set of findings were related to the design for interdisciplinarity. In this regard, the need to link research and education with real-world practice emerged as a key design concern. Positioning within the program context was also considered to be important with a need to balance early intervention to embed integration with later repeat interventions that maximize opportunities to share skills and experiences. Conclusions: The authors raise and address challenges to interdisciplinary program design for CH based on an abductive approach combining interdisciplinary and interprofessional education literature and the collection of qualitative data. This recipe approach for interdisciplinary design offers guidelines for policy makers, educators, and innovators in the CH space. Gaining insight from CH researchers regarding the development of an IDE module has offered the designers a novel insight regarding the curriculum, timing, delivery, and potential challenges that may be encountered.

  • Source: Pexels; Copyright: Tracy Le Blanc; URL:; License: Licensed by the authors.

    Health Researchers’ Use of Social Media: Scoping Review


    Background: Health researchers are increasingly using social media in a professional capacity, and the applications of social media for health researchers are vast. However, there is currently no published evidence synthesis of the ways in which health researchers use social media professionally, and uncertainty remains as to how best to harness its potential. Objective: This scoping review aimed to explore how social media is used by health researchers professionally, as reported in the literature. Methods: The scoping review methodology guided by Arksey and O’Malley and Levac et al was used. Comprehensive searches based on the concepts of health research and social media were conducted in MEDLINE, EMBASE, CINAHL, PsycINFO, ERIC, and Web of Science databases, with no limitations applied. Articles were screened at the title and abstract level and at full text by two reviewers. One reviewer extracted data that were analyzed descriptively to map the available evidence. Results: A total of 8359 articles were screened at the title and abstract level, of which 719 were also assessed at full text for eligibility. The 414 articles identified for inclusion were published in 278 different journals. Studies originated from 31 different countries, with the most prevalent being the United States (52.7% [218/414]). The health discipline of the first authors varied, with medicine (33.3% [138/414]) being the most common. A third of the articles covered health generally, with 61 health-specific topics. Papers used a range of social media platforms (mean 1.33 [SD 0.7]). A quarter of the articles screened reported on social media use for participant recruitment (25.1% [104/414]), followed by practical ways to use social media (15.5% [64/414]), and use of social media for content analysis research (13.3% [55/414]). Articles were categorized as celebratory (ie, opportunities for engagement, 72.2% [299/414]), contingent (ie, opportunities and possible limitations, 22.7% [94/414]) and concerned (ie, potentially harmful, 5.1% [21/414]). Conclusions: Health researchers are increasingly publishing on their use of social media for a range of professional purposes. Although most of the sentiment around the use of social media in health research was celebratory, the uses of social media varied widely. Future research is needed to support health researchers to optimize their social media use.

  • Source: Pexels; Copyright: Pixabay; URL:; License: Licensed by the authors.

    Smartphone Apps to Support Coordinated Specialty Care for Prodromal and Early Course Schizophrenia Disorders: Systematic Review


    Background: Demand for mental health services, especially for clinical high-risk and early psychosis, has increased, creating a need for new solutions to increase access to and quality of care. Smartphones and mobile technology are potential tools to support coordinated specialty care for early psychosis, given their potential to augment the six core roles of care: case management and team leadership, recovery-oriented psychotherapy, medication management, support for employment and education, coordination with primary care services, and family education and support. However, the services smartphones are actually offering specifically for coordinated specialty care and the level of evidence are unknown. Objective: This study aimed to review the published literature on smartphone technology to enhance care for patients with prodromal and early course psychosis and schizophrenia and to analyze studies by type, aligned with coordinated specialty care domains. Methods: A systematic literature search was conducted on August 16 and 17, 2019, using the PubMed, EMBASE, Web of Sciences, and PsycINFO electronic databases. The eligible studies were reviewed and screened based on inclusion and exclusion criteria. Results: The search uncovered 388 unique results, of which 32 articles met the initial inclusion criteria; 21 eligible studies on 16 unique app platforms were identified. Feasibility studies showed a high user engagement and interest among patients, monitoring studies demonstrated a correlation between app assessments and clinical outcomes, and intervention studies indicated that these apps have the potential to advance care. Eighteen studies reported on app use for the case management roles of coordinated specialty care. No app studies focused on employment and education, coordination with primary care services, and family education and support. Conclusions: Although the published literature on smartphone apps for prodromal and first-episode psychosis is small, it is growing exponentially and holds promise to augment both monitoring and interventions. Although the research results and protocols for app studies are not well aligned with all coordinated specialty care roles today, high rates of adoption and feasibility suggest the potential for future efforts. These results will be used to develop coordinated specialty care–specific app evaluation scales and toolkits.

  • Patient and pediatrician discussing Health-Related Quality of Life questionnaires. Source: Image created by the Authors; Copyright: The Authors; URL:; License: Creative Commons Attribution + Noncommercial (CC-BY-NC).

    Measurement Properties of the Online EuroQol-5D-Youth Instrument in Children and Adolescents With Type 1 Diabetes Mellitus: Questionnaire Study


    Background: The lack of continuity between health-related quality of life (HRQoL) instruments designed for children and adults hinders change analysis with a life course approach. To resolve this gap, EuroQol (EQ) developed the EQ-5D-Youth (EQ-5D-Y), derived from the EQ-5D for adults. Few studies have assessed the metric properties of EQ-5D-Y in children with specific chronic conditions, and none have done so for children with type I diabetes mellitus (T1DM). Objective: This study aimed to evaluate the acceptability, validity, reliability, and responsiveness of the EQ-5D-Y in children and adolescents with T1DM, when administered online. Methods: Participants with T1DM were consecutively recruited from July to December 2014, from a list of potential candidates aged 8-19 years, who attended outpatient pediatric endocrinology units. Before every quarterly routine visit, participants received an email/telephone reminder to complete the online version of two generic HRQoL questionnaires: EQ-5D-Y and KIDSCREEN-27. The EQ-5D-Y measures five dimensions, from which an equally weighted summary score was constructed (range: 0-100). Completion rate and distribution statistics were calculated. Construct validity was evaluated through known group comparisons based on general health, acute diabetic decompensations, mental health, family function, and a multitrait, multimethod matrix between EQ-5D-Y and KIDSCREEN by using Spearman correlations. Construct validity hypotheses were stated a priori. Reliability was assessed with the intraclass correlation coefficient and responsiveness by testing changes over time and calculating the effect size. Reliability and responsiveness were tested among the stable and improved subsamples defined by a KIDSCREEN-10 index change of <4.5 points or ≥4.5 points, respectively, from the first to the fourth visit. Results: Of the 136 participants, 119 (87.5%) responded to the EQ-5D-Y at the last visit. The dimensions that showed higher percentages of participants with problems were “having pain/discomfort” (34.6%) and “worried/sad/unhappy” (28.7%). The mean (SD) of the EQ-5D-Y summary score was 8.5 (10.9), with ceiling and floor effects of 50.7% and 0%, respectively. Statistically significant HRQoL differences between groups defined by their general health (excellent/very good and good/regular/bad) and mental health (Strengths and Difficulties Questionnaire score ≤15 and >16, respectively) were found in three EQ-5D-Y dimensions (“doing usual activities,” “having pain/discomfort,” and “feeling worried/sad/unhappy”), summary score (effect size for general health and mental health groups=0.7 and 1.5, respectively), and KIDSCREEN-10 index (effect size for general health and mental health groups=0.6 and 0.9, respectively). Significant differences in the EQ-5D-Y dimensions were also found according to acute diabetic decompensations in “looking after myself” (P=.005) and according to family function in “having pain/discomfort” (P=.03). Results of the multitrait, multimethod matrix confirmed three of the four relationships hypothesized as substantial (0.21, 0.58, 0.50, and 0.46). The EQ-5D-Y summary score presented an intraclass correlation coefficient of 0.83. Statistically significant change between visits was observed in the improved subsample, with an effect size of 0.7 (P<.001). Conclusions: These results support the use of the EQ-5D-Y administered online as an acceptable, valid, reliable, and responsive instrument for evaluating HRQoL in children and adolescents with T1DM.

  • Source: Shutterstock Inc; Copyright: xtock; URL:; License: Licensed by the authors.

    Cost Per Participant Recruited From Rural and Remote Areas Into a Smoking Cessation Trial Via Online or Traditional Strategies: Observational Study


    Background: Rural and remote residents are more likely to smoke than those who live in major cities; however, recruitment of research participants from rural and remote areas can be challenging. The cost per participant recruited from rural and remote areas via online (eg, social media) and traditional strategies (eg, print) has implications for researchers on how to allocate resources to maximize the number of participants recruited. Participant characteristics such as demographics, financial stress, mental health, and smoking-related factors may be associated with recruitment method (ie, online vs traditional), and so it is important to understand whether certain subgroups are more likely to be recruited via a particular strategy. Objective: This study aimed to determine the cost per participant recruited and examine whether characteristics such as demographics, financial stress, mental health, and smoking-related factors may be associated with the recruitment method (ie, online vs traditional). Methods: Participants were recruited into a randomized trial that provided smoking cessation support. Eligible participants were aged 18 years or older; used tobacco daily; had access to video communication software, internet, and telephone; had an email address; and lived in a rural or remote area of New South Wales, Australia. This study describes the natural (observed) experience of recruiting participants via online and traditional methods into a smoking cessation trial. Results: Over 17 months, 655 participants were recruited into the smoking cessation trial. A total of 88.7% (581/655) of the participants were recruited via online methods. Moreover, 1.8% (12/655) of the participants were recruited from remote locations and none from very remote areas. The cost per participant recruited by the various online strategies ranged from Aus $7.29 (US $4.96, £4.09, and €4.43) for Gumtree, a local online classified website, to Aus $128.67 (US $87.63, £72.20, and €78.28) for email. The cost per participant recruited using traditional strategies ranged from Aus $0 (US $0, £0, and €0) for word of mouth to Aus $3990.84 (US $2757.67, £2227.85, and €2477.11) for telephone. Women had greater odds of being recruited via online methods than men (odds ratio 2.50, 95% CI 1.42-4.40). No other characteristics were associated with the recruitment method. Conclusions: The cost per participant recruited via online and traditional strategies varied, with the range being smaller for online than traditional recruitment strategies. Women have greater odds of being recruited via online strategies into rural smoking cessation trials. Clinical Trial: Australian New Zealand Clinical Trials Registry ACTRN12617000514303;

  • Source: Creative Commons (Flickr); Copyright: Ministerio TIC Colombia; URL:; License: Creative Commons Attribution (CC-BY).

    Qualitative Synthesis of Young People’s Experiences With Technology-Assisted Cognitive Behavioral Therapy: Systematic Review


    Background: Cognitive behavioral therapy (CBT) for young people is increasingly being provided using technology-assisted formats. Although there is increasing evidence regarding the efficacy of such approaches, as illustrated by quantitative systematic reviews, the literature has also highlighted challenges with implementation factors, including high attrition rates and variable user engagement. Qualitative review methods can help to address the factors that impact young peoples’ experience of technology-assisted cognitive behavioral therapy (tech-assisted CBT) and, thus, enable us to better understand such implementation factors. To date, no such qualitative synthesis exists. Objective: The primary aim of this review was to systematically identify and synthesize the qualitative literature concerning the experiences of young people who have used tech-assisted CBT. Methods: This systematic review applied Thomas and Harden’s 2008 qualitative thematic synthesis approach. This involved line-by-line coding of the results sections of included studies and an inductive analysis on identified themes, followed by the generation of analytical themes through a process of iteration and interpretation of the descriptive themes. PsycINFO, ACM Digital Library, PubMed, EMBASE, and JMIR Publications databases were searched. The inclusion criteria were (1) studies involving school-aged young people over preschool age (6 years) but under the age of 18 years, (2) use of any form of tech-assisted CBT for any time period, (3) a stated focus of qualitative data to document the experiences of participants, and (4) studies published in English. The exclusion criteria were (1) interventions only provided face-to-face with no technological component, (2) only focused on the performance of the technology rather than participant experience, and (3) numerical data that sought to represent qualitative data. Results: A total of 14 studies were included in this review. Overall, these studies represented interventions for low mood and anxiety (n=10), trauma or self-harm (n=2), and physical difficulties (n=2). Overall, 5 analytical themes emerged on young people’s experiences with tech-assisted CBT: (1) helpfulness, (2) therapeutic process, (3) transferability, (4) gameplay experience, and (5) limitations. In addition, these analytical themes contained the following subthemes: positive experiences, tech-assisted CBT versus face-to-face CBT, understanding of a CBT model, process of change, skills development, application to everyday life settings, parental involvement, character relatedness, playability, negative experiences, and broad content. Conclusions: Overall, young people’s experiences with tech-assisted CBT were mostly positive. The use of gaming environments, relatable characters, concrete metaphors, and age-appropriate narratives contributed to these positive experiences. Evidence suggests that technology can help to mediate face-to-face relationships with therapists and help young people to understand the CBT model. Clear barriers also emerged, including over-reliance on reading and writing skills and dissatisfaction with overly generalized content and comparison with commercial technologies.

  • Source: Image created by the Authors; Copyright: The Authors; URL:; License: Creative Commons Attribution (CC-BY).

    Investigating Software Requirements for Systems Supporting Task-Shifted Interventions: Usability Study


    Background: There is a considerable shortfall in specialized health care professionals worldwide to deliver health services, and this shortfall is especially pronounced in low-middle-income countries. This has led to the implementation of task-shifted interventions, in which specific tasks are moved away from highly qualified health workers to health workers with less training. The World Health Organization (WHO) has published recommendations for such interventions, but guidelines for software and systems supporting such interventions are not included. Objective: The objective of this study was to formulate a number of software requirements for computer systems supporting task-shifted interventions. As the treatment of mental health problems is generally considered to be a task for highly trained health care professionals, it poses interesting case studies for task-shifted interventions. Therefore, we illustrated the use of the identified software requirements in a mobile system created for a task-shifted depression intervention to be provided to older adults in deprived areas of São Paulo, Brazil. Methods: Using a set of recommendations based on the WHO’s guidance documentation for task-shifted interventions, we identified 9 software requirements that aim to support health workers in management and supervision, training, good relationship with other health workers, and community embeddedness of the intervention. These 9 software requirements were used to implement a system for the provision of a psychosocial depression intervention with mobile Android interfaces to structure interventions and collect data, and Web interfaces for supervision and support of the health care workers delivering the intervention. The system was tested in a 2-arm pilot study with 33 patients and 11 health workers. In all, 8 of these 11 health workers participated in a usability study subsequent to the pilot. Results: The qualitative and quantitative feedback obtained with the System Usability Scale suggest that the system was deemed to have a usability of between OK and Good. Nevertheless, some participants’ responses indicated that they felt they needed technical assistance to use the system. This was reinforced by answers obtained with perceived usefulness and ease of use questionnaires, which indicated some users felt that they had issues around correct use of the system and perceived ability to become skillful at using the system. Conclusions: Overall, these high-level requirements adequately captured the functionality required to enable the health workers to provide the intervention successfully. Nevertheless, the analysis of results indicated that some improvements were required for the system to be useable in a task-shifted intervention. The most important of these were better access to a training environment, access for supervisors to metadata such as duration of sessions or exercises to identify issues, and a more robust and human-error–proof approach to the availability of patient data on the mobile devices used during the intervention.

Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Latest Submissions Open for Peer-Review:

View All Open Peer Review Articles
  • Hackathons as a stepping stone in health care innovation: a case study with systematic recommendations

    Date Submitted: Nov 14, 2019

    Open Peer Review Period: Nov 14, 2019 - Jan 9, 2020

    Background: Up until recently, developing health technologies is time consuming and expensive, and often involves patients, doctors, and other health care professionals (HCPs) only as passive recipien...

    Background: Up until recently, developing health technologies is time consuming and expensive, and often involves patients, doctors, and other health care professionals (HCPs) only as passive recipients of the end-product. So far, users have been poorly involved in ideation and creation stages in digital health. In order to best address their unmet needs, a transdisciplinary and user-led approach, involving co-creation and direct user feedback, is required. In this context, hackathon events have become increasingly popular to generate enthusiasm for user-centered innovation. Objective: This case study describes preparatory steps and performance of a health hackathon directly involving patients and HCPs at all stages. Feasibility and outcomes were assessed, leading to the development of systematic recommendations for future hackathons as a vehicle for bottom-up innovation in health care. Methods: A 2-day hackathon was conducted in February 2017 in Berlin, Germany. Data was collected through a field study. Collected field notes were subsequently discussed in 15 informal meetings within the research team. Experiences of conducting two further hackathons in December 2017 and November 2018 were included. Results: In total, 30 participants took part, with 63% (n=19) aged between 25 and 34 years, 30% (n=9) between 35 and 44 years, and 7% (n=2) being younger than 24 years. 43% (n=13) were female. Participation rate of medical experts, including patients and HCPs, was 30% (n=9). Five multidisciplinary teams were formed, and each tackled a specific health care problem. All presented projects were apps: a chatbot for skin cancer recognition, an augmented reality exposure-based therapy (e.g., for arachnophobia), an app for medical neighborhood connectivity, a doctor appointment platform and a self-care app for people suffering from depression. Patients and HCPs initiated all of the projects. Performing the hackathon resulted in a significant growth of the digital health community of Berlin, and was followed up by larger hackathons. Systematic recommendation for conducting cost-efficient hackathons (n≤30) were developed, including aspects of community building, stakeholder engagement, mentoring, themes, announcements, follow-up, and timing for each step. Conclusions: This study shows that hackathons are effective in bringing innovation to health care, are more cost- and time efficient and potentially more sustainable than traditional medical device and digital product development. Our systematic recommendations can be useful to other individuals and organizations that want to establish user-led innovation in academic hospitals by conducting transdisciplinary hackathons. Clinical Trial: n/a

  • Design of a conceptual model for knowledge commons in the patient world arena: A case study approach

    Date Submitted: Nov 12, 2019

    Open Peer Review Period: Nov 12, 2019 - Jan 7, 2020

    Background: The rapidly developing opportunities for better individual health in long term conditions continue to overwhelm the delivery shortcomings of the current health care system. Objective: To p...

    Background: The rapidly developing opportunities for better individual health in long term conditions continue to overwhelm the delivery shortcomings of the current health care system. Objective: To present a conceptual model by which novel health technologies could be introduced and evaluated through knowledge commons, with special attention to protect personal real-life data and information flows between the stakeholders involved. Methods: The model is developed from a real-life case by applying the theory of knowledge commons on existing examples successfully improving health outcomes from new health technologies. Results: The presented conceptual model demonstrates how real time use of data obtained from patients and professional providers can be enabled and how this improves health outcomes in interaction with health technologies, i.e. pharmaceuticals, devices, procedures and organizational systems. Analyses of these data can inform additional stakeholders to optimize individual and group health, to improve care and the effectiveness of systems and to generate new knowledge, future opportunities and improved implementation on a population-based level. Conclusions: We propose the patient as the core agent when introducing and evaluating novel health technologies and for enabling collaborative knowledge-based co-production of health among involved stakeholders.

  • Mobile Health for Perinatal Depression and Anxiety: A Scoping Review

    Date Submitted: Nov 12, 2019

    Open Peer Review Period: Nov 11, 2019 - Jan 6, 2020

    Background: The perinatal period is a vulnerable time during which depression and anxiety commonly occur. Left untreated or under-treated, there may be significant adverse effects, therefore, access t...

    Background: The perinatal period is a vulnerable time during which depression and anxiety commonly occur. Left untreated or under-treated, there may be significant adverse effects, therefore, access to rapid, effective treatment is essential. Treatments for mild-to-moderate symptoms according to a stepped care approach involve psychoeducation, peer support, and psychological therapy, all of which have been shown to be efficaciously delivered through digital means. Women experience significant barriers to care due to system and individual level factors, such as cost, accessibility, and availability of childcare. The use of mobile phones is widespread in this population, and delivery of mental health services via mobile phones has been suggested as a means of reducing barriers. Objective: To understand the extent, range, and nature of mobile health (mHealth) tools for prevention, screening, and treatment of perinatal depression and anxiety in order to identify gaps and inform opportunities for future work. Methods: Using a scoping review framework, 4 databases were searched for terms related to mobile phones, perinatal period, and depression/anxiety. 477 unique records were retrieved, 81 of which were reviewed by full text. Peer-reviewed publications were included if they described the population as women pregnant or up to 1 year postpartum, and a tool for preventing, screening, or treating depression/anxiety explicitly delivered via mobile phone. Studies published in 2007 or earlier, not in English, or as case reports were excluded. Results: 26 publications describing 22 unique studies were included (77.3% published after 2017). mHealth applications (apps) were slightly more common than texting-based interventions (n=12, 54.5% vs n=10, 45.5%). Most tools were for depression only (n=12, 54.5%) or anxiety and depression (n=9, 41.0%); one was for anxiety only (4.5%). Interventions were rarely for use starting in pregnancy and continuing into postpartum (n=2, 9%). Tools were for prevention (n=10, 45.5%), screening (n=6, 27.3%), and treatment (n=6, 27.3%). Interventions delivered included psychoeducation (n=16, 72.7%), peer support (n=4, 18.2%), and psychological therapy (n=4, 18.2%). Cost was measured in 3 (14%) studies. Conclusions: Future work in this growing area should incorporate active psychological treatment, address continuity of care across the perinatal period, and consider clinical sustainability to realize mHealth’s potential.

  • Impact Of Electronic Health Record Interface Design On Unsafe Prescribing Of Ciclosporin, Tacrolimus and Diltiazem: A Cohort Study In English NHS Primary Care

    Date Submitted: Nov 11, 2019

    Open Peer Review Period: Nov 11, 2019 - Jan 6, 2020

    Background: In England, national safety guidance recommends that ciclosporin, tacrolimus and diltiazem are prescribed by brand name due to their narrow therapeutic windows and, in the case of tacrolim...

    Background: In England, national safety guidance recommends that ciclosporin, tacrolimus and diltiazem are prescribed by brand name due to their narrow therapeutic windows and, in the case of tacrolimus, to reduce the chance of organ transplantation rejection. Various small studies have shown that changes to electronic health records (EHR) interface can affect prescribing choices. Objective: Our objectives were to assess variation by EHR system in breaches of safety guidance around prescribing of ciclosporin, tacrolimus and diltiazem; and to conduct user-interface research into the causes of such breaches. Methods: We carried out a retrospective cohort study using prescribing data in English primary care. Participants were English general practices and their respective electronic health records. The main outcome measures were (1) variation in ratio of breaching / adherent prescribing all practices (2) description of observations of EHR usage. Results: A total of 2,575,411 prescriptions were issued in 2018 for ciclosporin, tacrolimus and diltiazem (over 60mg); of these, 316,119 prescriptions breached NHS guidance (12.3%). Breaches were most common amongst users of the EMIS EHR (in 23.2% of ciclosporin & tacrolimus prescriptions, and 22.7% of diltiazem prescriptions); but breaches were observed in all EHRs. Conclusions: Design choices in EHR strongly influence safe prescribing of ciclosporin, tacrolimus and diltiazem; and breaches are prevalent in general practices in England. We recommend that all EHR vendors review their systems to increase safe prescribing of these medicines in line with national guidance. Almost all clinical practice is now mediated through an EHR system: further quantitative research into the effect of EHR design on clinical practice is long overdue.

  • Communicating Uncertainty from Limitations in Quality of Evidence to the Public in Written Consumer Health Information: a Parallel-group, Web-based Randomized Controlled Trial

    Date Submitted: Nov 8, 2019

    Open Peer Review Period: Nov 8, 2019 - Jan 3, 2020

    Background: Uncertainty is integral to evidence-informed decision making and is of particular importance for preference-sensitive decisions. Communicating uncertainty to patients and the public has lo...

    Background: Uncertainty is integral to evidence-informed decision making and is of particular importance for preference-sensitive decisions. Communicating uncertainty to patients and the public has long been identified as a goal in the informed and shared decision-making movement. Despite this, there is little quantitative research on how uncertainty in health information is perceived by readers. Objective: The aim of this study was to examine the impact of different uncertainty descriptions regarding the evidence for a treatment effect in a written research summary for the public. Methods: We developed 8 versions of a research summary on a fictitious tinnitus drug with varying degrees (Q1), sources (Q2) and magnitudes of uncertainty (Q3). We recruited 2099 members of the German public from a web-based research panel. Of these, 1727 fulfilled the inclusion criteria and were randomly presented with one of these research summaries. Randomization was conducted by a centralized computer using a random number generator. Web-based recruitment and data collection were fully automated. Participants were not aware of the purpose of the study and alternative presentations. We measured the following outcomes: perception of the treatment effectiveness (primary); certainty in the judgement of treatment effectiveness; perception of the body of evidence; text quality; intended decision. Outcomes were self-assessed. Results: We did not find a global effect for Q1 and Q2 (p=.25 and p=.73), but for Q3 (p=.048). Pairwise comparisons showed a weaker perception of the treatment effectiveness for the research summary with 3 sources of uncertainty compared to a version with 2 sources of uncertainty (p=.037). Specifically, 9% less participants perceived the tinnitus drug as possibly beneficial, while 8% more considered it to be of unclear benefit in the group with 3 sources of uncertainty. There was no difference compared to a version with 1 source of uncertainty (p=.31), however. We did not find any meaningful differences between the research summaries for the secondary outcomes. Conclusions: Communicating even a large magnitude of uncertainty for a treatment effect had little impact on perceived effectiveness. Efforts to improve public understanding of research are needed to improve understanding of evidence based health information. Clinical Trial: German Clinical Trials Register DRKS00015911,

  • Adopting patient portals in hospitals: a qualitative study

    Date Submitted: Nov 5, 2019

    Open Peer Review Period: Nov 5, 2019 - Dec 31, 2019

    Background: Theoretical models help to explain or predict the adoption of eHealth technology and illustrate the complexity of the adoption process. Various models provide insights into general factors...

    Background: Theoretical models help to explain or predict the adoption of eHealth technology and illustrate the complexity of the adoption process. Various models provide insights into general factors that influence the use of eHealth technology. However, they do not give hospitals much actionable knowledge on how to facilitate the adoption process. Objective: Our study aims to provide insights into patient portal adoption processes among patients and hospital staff (including healthcare professionals, managers and administrative clerks). Studying the experiences and views of stakeholders answers this question: How can hospitals encourage patients and healthcare professionals to adopt a patient portal? Methods: 22 semi-structured (group) interviews (N=69) in 12 hospitals and four focus groups with members of (semi)national organizations and patient portal suppliers (N=53). Results: The effort hospitals put into adopting patient portals can be split into three themes. First, informing patients and healthcare professionals about the portal. The communication strategy has four objectives: 1) knowing about the portal, 2) knowing how it works, 3) encouraging visits to the portal and 4) knowing where to get support. Second, embedding the patient portal in the daily routine of healthcare professionals and management with three forms of support: 1) hospital policy, 2) management by numbers, and 3) a structured implementation strategy that includes all staff of one department. This embedding requires changing work processes and routines. Third, adjusting the portal to meet patients’ needs in the effort to optimize user-friendliness in two ways: 1) using patient feedback, and 2) focusing on optimizing the portal for patients with special needs (e.g. low literacy,, low digital skills). Interestingly,hospitals are reluctant to involve patients in the continuous development of patient portals, because they have experienced that nothing can be done with the feedback received (e.g. technologically impossible or too expensive). Conclusions: Asking stakeholders what they have learned from their attempts to stimulate patient portal use in hospitals elicited rich insights into the adoption process. This practical knowledge helps to translate the relatively abstract success factors one finds in scientific adoption models to the everyday pragmatics of eHealth projects in hospitals.