Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Monday, March 11, 2019 at 4:00 PM to 4:30 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?


Journal Description

The Journal of Medical Internet Research (JMIR), now in its 21st year, is the pioneer open access eHealth journal and is the flagship journal of JMIR Publications. It is the leading digital health journal globally in terms of quality/visibility (Impact Factor 2018: 4.945, ranked #1 out of 26 journals in the medical informatics category) and in terms of size (number of papers published). The journal focuses on emerging technologies, medical devices, apps, engineering, and informatics applications for patient education, prevention, population health and clinical care. As a leading high-impact journal in its disciplines (health informatics and health services research), it is selective, but it is now complemented by almost 30 specialty JMIR sister journals, which have a broader scope. Peer-review reports are portable across JMIR journals and papers can be transferred, so authors save time by not having to resubmit a paper to different journals. 

As an open access journal, we are read by clinicians, allied health professionals, informal caregivers, and patients alike, and have (as with all JMIR journals) a focus on readable and applied science reporting the design and evaluation of health innovations and emerging technologies. We publish original research, viewpoints, and reviews (both literature reviews and medical device/technology/app reviews).

We are also a leader in participatory and open science approaches, and offer the option to publish new submissions immediately as preprints, which receive DOIs for immediate citation (eg, in grant proposals), and for open peer-review purposes. We also invite patients to participate (eg, as peer-reviewers) and have patient representatives on editorial boards.

Be a widely cited leader in the digitial health revolution and submit your paper today!


Recent Articles:

  • Source: Image created by the Authors; Copyright: Nastasia Griffioen; URL:; License: Creative Commons Attribution (CC-BY).

    A Stimulated Recall Method for the Improved Assessment of Quantity and Quality of Social Media Use


    Background: Social media are as popular as ever, and concerns regarding the effects of social media use on adolescent well-being and mental health have sparked many scientific studies into use effects. Social media research is currently at an important crossroads: conflicting results on social media use’s effects on well-being are abundant, and recent work in the field suggests that a new approach is required. The field is in need of an approach involving objective data regarding use where necessary and attention to different kinds of detail such as the why and how of social media use. Objective: We present a novel paradigm implementing a principle from educational sciences called stimulated recall and demonstrate how it can be applied to social media use research. Our stimulated recall paradigm implements a number of elements that can fill the gaps currently present in social media and well-being research. Methods: Objective data are collected regarding users’ social media behaviors through video footage and in-phone data and used for a structured stimulated recall interview to facilitate detailed and context-sensitive processing of these objective data. In this interview, objective data are reviewed with the participant in an act of co-research, in which details such as the reasons for their use (eg, boredom) and processes surrounding their use (eg, with whom) are discussed and visualized in a stimulated recall chart. Results: Our ongoing study (N=53) implementing this paradigm suggests this method is experienced as pleasant by participants in spite of its personal and intensive nature. Conclusions: The stimulated recall paradigm offers interesting and necessary avenues for approaching social media use research from new angles, addressing aspects of use that have thus far remained underexposed. The answers to questions such as “Why do adolescents use social media?” “In what ways exactly do they use social media?” and “How does social media use make them feel in the moment?” are now within reach, an important step forward in the field of social media use and well-being research.

  • This file contains the TOC image for the publication. Source: Foter; Copyright:; URL:; License: Public Domain (CC0).

    Lifestyle Disease Surveillance Using Population Search Behavior: Feasibility Study


    Background: As the process of producing official health statistics for lifestyle diseases is slow, researchers have explored using Web search data as a proxy for lifestyle disease surveillance. Existing studies, however, are prone to at least one of the following issues: ad-hoc keyword selection, overfitting, insufficient predictive evaluation, lack of generalization, and failure to compare against trivial baselines. Objective: The aims of this study were to (1) employ a corrective approach improving previous methods; (2) study the key limitations in using Google Trends for lifestyle disease surveillance; and (3) test the generalizability of our methodology to other countries beyond the United States. Methods: For each of the target variables (diabetes, obesity, and exercise), prevalence rates were collected. After a rigorous keyword selection process, data from Google Trends were collected. These data were denormalized to form spatio-temporal indices. L1-regularized regression models were trained to predict prevalence rates from denormalized Google Trends indices. Models were tested on a held-out set and compared against baselines from the literature as well as a trivial last year equals this year baseline. A similar analysis was done using a multivariate spatio-temporal model where the previous year’s prevalence was included as a covariate. This model was modified to create a time-lagged regression analysis framework. Finally, a hierarchical time-lagged multivariate spatio-temporal model was created to account for subnational trends in the data. The model trained on US data was, then, applied in a transfer learning framework to Canada. Results: In the US context, our proposed models beat the performances of the prior work, as well as the trivial baselines. In terms of the mean absolute error (MAE), the best of our proposed models yields 24% improvement (0.72-0.55; P<.001) for diabetes; 18% improvement (1.20-0.99; P=.001) for obesity, and 34% improvement (2.89-1.95; P<.001) for exercise. Our proposed across-country transfer learning framework also shows promising results with an average Spearman and Pearson correlation of 0.70 for diabetes and 0.90 and 0.91 for obesity, respectively. Conclusions: Although our proposed models beat the baselines, we find the modeling of lifestyle diseases to be a challenging problem, one that requires an abundance of data as well as creative modeling strategies. In doing so, this study shows a low-to-moderate validity of Google Trends in the context of lifestyle disease surveillance, even when applying novel corrective approaches, including a proposed denormalization scheme. We envision qualitative analyses to be a more practical use of Google Trends in the context of lifestyle disease surveillance. For the quantitative analyses, the highest utility of using Google Trends is in the context of transfer learning where low-resource countries could benefit from high-resource countries by using proxy models.

  • Source: The Authors / Placeit; Copyright: JMIR Publications; URL:; License: Creative Commons Attribution (CC-BY).

    Analysis of Collective Human Intelligence for Diagnosis of Pigmented Skin Lesions Harnessed by Gamification Via a Web-Based Training Platform: Simulation...


    Background: The diagnosis of pigmented skin lesion is error prone and requires domain-specific expertise, which is not readily available in many parts of the world. Collective intelligence could potentially decrease the error rates of nonexperts. Objective: The aim of this study was to evaluate the feasibility and impact of collective intelligence for the detection of skin cancer. Methods: We created a gamified study platform on a stack of established Web technologies and presented 4216 dermatoscopic images of the most common benign and malignant pigmented skin lesions to 1245 human raters with different levels of experience. Raters were recruited via scientific meetings, mailing lists, and social media posts. Education was self-declared, and domain-specific experience was tested by screening tests. In the target test, the readers had to assign 30 dermatoscopic images to 1 of the 7 disease categories. The readers could repeat the test with different lesions at their own discretion. Collective human intelligence was achieved by sampling answers from multiple readers. The disease category with most votes was regarded as the collective vote per image. Results: We collected 111,019 single ratings, with a mean of 25.2 (SD 18.5) ratings per image. As single raters, nonexperts achieved a lower mean accuracy (58.6%) than experts (68.4%; mean difference=−9.4%; 95% CI −10.74% to −8.1%; P<.001). Collectives of nonexperts achieved higher accuracies than single raters, and the improvement increased with the size of the collective. A collective of 4 nonexperts surpassed single nonexperts in accuracy by 6.3% (95% CI 6.1% to 6.6%; P<.001). The accuracy of a collective of 8 nonexperts was 9.7% higher (95% CI 9.5% to 10.29%; P<.001) than that of single nonexperts, an improvement similar to single experts (P=.73). The sensitivity for malignant images increased for nonexperts (66.3% to 77.6%) and experts (64.6% to 79.4%) for answers given faster than the intrarater mean. Conclusions: A high number of raters can be attracted by elements of gamification and Web-based marketing via mailing lists and social media. Nonexperts increase their accuracy to expert level when acting as a collective, and faster answers correspond to higher accuracy. This information could be useful in a teledermatology setting.

  • The use of internet-based programs at schools is growing, specifically, those concerning the prevention of alcohol abuse, providing highly personalized behavioral and motivational feedback to the individual. The internet has become a worldwide available and promising tool, accessible to large populations showing that computer tailoring is effective in supporting health-related changes for a number of different behaviors. Source: Image created by the Authors; Copyright: The Authors; URL:; License: Licensed by JMIR.

    A Web-Based, Computer-Tailored Intervention to Reduce Alcohol Consumption and Binge Drinking Among Spanish Adolescents: Cluster Randomized Controlled Trial


    Background: Alcohol consumption, including binge drinking (BD) and heavy episodic drinking (HED), is one of the leading risk factors among Spanish adolescents leading to significant social, health, and economic consequences. Reduction of BD and HED in adolescents can be achieved using Web-based, computer-tailored (CT) interventions, providing highly personalized feedback that is adapted to a person’s individual characteristics and needs. Randomized controlled trials assessing the effects of tailored BD reduction programs among Spanish adolescents are scarce. Objective: The aim of this study was to test the effectiveness of the Web-based, CT intervention Alerta Alcohol, aimed at the prevention of BD in Spanish adolescents. As a secondary outcome, effects on HED, weekly consumption, and any consumption were also assessed. The adherence and process evaluation were assessed. Methods: A cluster randomized controlled trial conducted among 15 Spanish schools was developed. Each school was randomized into either an experimental condition (EC) (N=742) or a control condition (CC) (N=505). Finally, 351 participants for the EC and 261 for the CC were included in the analysis (N=612). Baseline assessment took place in January and February 2017. Demographic variables and alcohol use were assessed at baseline. Follow-up assessment of alcohol use took place 4 months later in May and June 2017. Participants were compared according to their randomization group (EC versus CC). After the baseline assessment, participants in the EC started the intervention, which consisted of short stories about BD, in which CT feedback was based on the I-Change Model for behavior change. Participants in the CC group only received the baseline questionnaire. Effects of the intervention were assessed using a three-level mixed logistic regression analysis for BD, HED, and any consumption, and a three-level mixed linear regression analysis for weekly consumption. Results: In total, 1247 adolescents participated in the baseline assessment and 612 participated in the follow-up assessment; the attrition rate was 50.92%. The intervention was effective in reducing HED among adolescents; the odds of HED in the CC was nine times that in the experimental condition (P=.04). No effects were found for BD, weekly consumption, and any consumption. Process evaluations revealed that the adolescents were satisfied with the program (68.8%), would use the program again (52.9%), and would recommend it to someone else (62.8%). Females and non-binge drinkers showed better responses in the process evaluation. Conclusions: Our intervention was effective regarding HED but not regarding BD, weekly consumption, and any consumption. It may be that limiting alcohol consumption to prevent HED was easier in the Spanish context than it was to carry out further steps, such as reducing other patterns of alcohol consumption. Hence, additional actions are needed to accomplish these latter goals, including community approaches and policy actions aimed at denormalizing alcohol consumption among Spanish adolescents. Trial Registration: NCT03288896;

  • Source: stockvault; Copyright: frhuynh; URL:; License: Licensed by JMIR.

    Online Information on Electronic Cigarettes: Comparative Study of Relevant Websites From Baidu and Google Search Engines


    Background: Online information on electronic cigarettes (e-cigarettes) may influence people’s perception and use of e-cigarettes. Websites with information on e-cigarettes in the Chinese language have not been systematically assessed. Objective: The aim of this study was to assess and compare the types and credibility of Web-based information on e-cigarettes identified from Google (in English) and Baidu (in Chinese) search engines. Methods: We used the keywords vaping or e-cigarettes to conduct a search on Google and the equivalent Chinese characters for Baidu. The first 50 unique and relevant websites from each of the two search engines were included in this analysis. The main characteristics of the websites, credibility of the websites, and claims made on the included websites were systematically assessed and compared. Results: Compared with websites on Google, more websites on Baidu were owned by manufacturers or retailers (15/50, 30% vs 33/50, 66%; P<.001). None of the Baidu websites, compared to 24% (12/50) of Google websites, were provided by public or health professional institutions. The Baidu websites were more likely to contain e-cigarette advertising (P<.001) and less likely to provide information on health education (P<.001). The overall credibility of the included Baidu websites was lower than that of the Google websites (P<.001). An age restriction warning was shown on all advertising websites from Google (15/15) but only on 10 of the 33 (30%) advertising websites from Baidu (P<.001). Conflicting or unclear health and social claims were common on the included websites. Conclusions: Although conflicting or unclear claims on e-cigarettes were common on websites from both Baidu and Google search engines, there was a lack of online information from public health authorities in China. Unbiased information and evidence-based recommendations on e-cigarettes should be provided by public health authorities to help the public make informed decisions regarding the use of e-cigarettes.

  • Source: Image created by the Authors; Copyright: The Authors; URL:; License: Licensed by JMIR.

    Digital Competencies and Attitudes Toward Digital Adherence Solutions Among Elderly Patients Treated With Novel Anticoagulants: Qualitative Study


    Background: Nonadherence to medication is a driver of morbidity and mortality, and complex medication regimens in patients with chronic diseases foster the problem. Digital technology might help, but despite numerous solutions being developed, none are currently widely used, and acceptance rates remain low, especially among the elderly. Objective: This study aimed to better understand and operationalize how new digital solutions can be evaluated. Particularly, the goal was to identify factors that help digital approaches targeting adherence to become more widely accepted. Methods: A qualitative study using a conceptual grounded theory approach was conducted. We included patients aged 65 years and older who routinely took new oral anticoagulants. To generate theses about the digital competencies of the target group with daily medication intake, face-to-face interviews were conducted, recorded, and anonymized. After coding the interviews, categories were generated, discussed, and combined with several theses until saturation of the statements was reached. Results: The methodological approach led to the finding that after interviews in 20 of 77 potentially available patients, a saturation of statements was reached. The average patient’s age was 75 years, and 50% (10/20) of the subjects were female. The data identified five main coding categories—Diseases and medicine, Technology, Autonomy, Patient narrative, and Attitude toward technologies—each including positive and negative subcategories. Main categories and subcategories were summarized as Adherence Radar, which can be considered as a framework to assess the potential of adherence solutions in the process of prototyping and can be applied to all adherence tools in a holistic manner. Conclusions: The Adherence Radar can be used to increase the acceptance rate of digital solutions targeting adherence. For a patient-centric design, an app should be adapted to the individual patient’s needs. According to our results, this application should be based on gender and educational background as well as the individual physician-patient relationship. If used in a proper, individualized manner, digital adherence solutions could become a new cornerstone for the treatment of chronically ill individuals.

  • Source: Unsplash; Copyright: Unsplash; URL:; License: Creative Commons Attribution (CC-BY).

    Systematic Evaluation of Research Progress on Natural Language Processing in Medicine Over the Past 20 Years: Bibliometric Study on PubMed


    Background: Natural language processing (NLP) is an important traditional field in computer science, but its application in medical research has faced many challenges. With the extensive digitalization of medical information globally and increasing importance of understanding and mining big data in the medical field, NLP is becoming more crucial. Objective: The goal of the research was to perform a systematic review on the use of NLP in medical research with the aim of understanding the global progress on NLP research outcomes, content, methods, and study groups involved. Methods: A systematic review was conducted using the PubMed database as a search platform. All published studies on the application of NLP in medicine (except biomedicine) during the 20 years between 1999 and 2018 were retrieved. The data obtained from these published studies were cleaned and structured. Excel (Microsoft Corp) and VOSviewer (Nees Jan van Eck and Ludo Waltman) were used to perform bibliometric analysis of publication trends, author orders, countries, institutions, collaboration relationships, research hot spots, diseases studied, and research methods. Results: A total of 3498 articles were obtained during initial screening, and 2336 articles were found to meet the study criteria after manual screening. The number of publications increased every year, with a significant growth after 2012 (number of publications ranged from 148 to a maximum of 302 annually). The United States has occupied the leading position since the inception of the field, with the largest number of articles published. The United States contributed to 63.01% (1472/2336) of all publications, followed by France (5.44%, 127/2336) and the United Kingdom (3.51%, 82/2336). The author with the largest number of articles published was Hongfang Liu (70), while Stéphane Meystre (17) and Hua Xu (33) published the largest number of articles as the first and corresponding authors. Among the first author’s affiliation institution, Columbia University published the largest number of articles, accounting for 4.54% (106/2336) of the total. Specifically, approximately one-fifth (17.68%, 413/2336) of the articles involved research on specific diseases, and the subject areas primarily focused on mental illness (16.46%, 68/413), breast cancer (5.81%, 24/413), and pneumonia (4.12%, 17/413). Conclusions: NLP is in a period of robust development in the medical field, with an average of approximately 100 publications annually. Electronic medical records were the most used research materials, but social media such as Twitter have become important research materials since 2015. Cancer (24.94%, 103/413) was the most common subject area in NLP-assisted medical research on diseases, with breast cancers (23.30%, 24/103) and lung cancers (14.56%, 15/103) accounting for the highest proportions of studies. Columbia University and the talents trained therein were the most active and prolific research forces on NLP in the medical field.

  • Source: freepik; Copyright: peoplecreations; URL:; License: Licensed by JMIR.

    Why Employees (Still) Click on Phishing Links: Investigation in Hospitals


    Background: Hospitals have been one of the major targets for phishing attacks. Despite efforts to improve information security compliance, hospitals still significantly suffer from such attacks, impacting the quality of care and the safety of patients. Objective: This study aimed to investigate why hospital employees decide to click on phishing emails by analyzing actual clicking data. Methods: We first gauged the factors that influence clicking behavior using the theory of planned behavior (TPB) and integrating trust theories. We then conducted a survey in hospitals and used structural equation modeling to investigate the components of compliance intention. We matched employees’ survey results with their actual clicking data from phishing campaigns. Results: Our analysis (N=397) reveals that TPB factors (attitude, subjective norms, and perceived behavioral control), as well as collective felt trust and trust in information security technology, are positively related to compliance intention. However, compliance intention is not significantly related to compliance behavior. Only the level of employees’ workload is positively associated with the likelihood of employees clicking on a phishing link. Conclusions: This is one of the few studies in information security and decision making that observed compliance behavior by analyzing clicking data rather than using self-reported data. We show that, in the context of phishing emails, intention and compliance might not be as strongly linked as previously assumed; hence, hospitals must remain vigilant with vulnerabilities that cannot be easily managed. Importantly, given the significant association between workload and noncompliance behavior (ie, clicking on phishing links), hospitals should better manage employees’ workload to increase information security. Our findings can help health care organizations augment employees’ compliance with their cybersecurity policies and reduce the likelihood of clicking on phishing links.

  • Dementia Care Competency & Training Network homepage (montage). Source: The Authors / Freepik; Copyright: The Authors; URL:; License: Creative Commons Attribution (CC-BY).

    A Web-Based Dementia Education Program and its Application to an Australian Web-Based Dementia Care Competency and Training Network: Integrative Systematic...


    Background: Dementia education that meets quality and safety standards is paramount to ensure a highly skilled dementia care workforce. Web-based education provides a flexible and cost-effective medium. To be successful, Web-based education must contain features that promote learning and support knowledge translation into practice. The Dementia Care Competency and Training Network (DCC&TN) has developed an innovative Web-based program that promotes improvement of the attitudes, knowledge, skills, behavior, and practice of clinicians, regardless of their work setting, in order to improve the quality of life for people living with dementia. Objective: This review aims to (1) determine the key features that are associated with an effective and functional Web-based education program—an effective and functional Web-based program is defined as one that measures results, is accessible, is user friendly, and translates into clinical practice—and (2) determine how these features correlate with the DCC&TN. Methods: Six electronic databases—Medline, Embase, Cumulative Index to Nursing and Allied Health Literature (CINAHL), AusHealth, Nursing@Ovid, and Google Scholar—were searched for articles published between 2009 and 2018 using the following keywords: Education, Distance, Continuing, Learning, Online, Web-Based, Internet, Dementia, Program Evaluation, Validation Studies, Outcome and Process Assessment Healthcare, Nursing, Assisted Instruction, and Facilitated. The Critical Appraisal Skills Programme (CASP) and Kirkpatrick’s model for the evaluation of training were used to ensure quality and rigor of the analysis. Results: A total of 46 studies met the inclusion criteria. In total, 14 key features were associated with an effective Web-based learning environment, which enabled the environment to be as follows: self-directed, individualized, interactive, multimodal, flexible, accessible, consistent, cost-effective, measurable with respect to participant satisfaction, equitable, facilitated, nurturing of critical thinking and reflection, supportive of creating a learning community, and translated into practice. These features were further categorized into five subgroups: applicability, attractiveness, functionality, learner interaction, and implementation into practice. Literature frequently cites Kirkpatrick’s four-level model of evaluation and application in the review of education and training; however, few studies appeared to integrate all four levels of Kirkpatrick’s model. Features were then correlated against the DCC&TN, with an encouraging connection found between these features and their inclusion within the content and structure of the DCC&TN. Conclusions: A total of 14 key features were identified that support an effective and functional Web-based learning environment. Few studies incorporated Kirkpatrick’s salient elements of the model—reaction, learning, behavior, and results—in their evaluation and clinical application. It could, therefore, be considered prudent to include Kirkpatrick’s levels of training evaluation within studies of dementia training. There were few studies that evaluated Web-based dementia education programs, with even fewer reporting evidence that Web-based training could increase staff confidence, knowledge, skills, and attitudes toward people with dementia and be sustainable over time. The DCC&TN appeared to contain the majority of key features and is one of the few programs inclusive of hospital, community, and residential care settings. The 14 key features can potentially enhance and complement future development of online training programs for health sciences education and beyond. The DCC&TN model could potentially be used as a template for future developers and evaluators of Web-based dementia training.

  • Source: Image created by the Authors; Copyright: The Authors; URL:; License: Licensed by JMIR.

    Efficacy of an Electronic Health Management Program for Patients With Cardiovascular Risk: Randomized Controlled Trial


    Background: In addition to medication, health behavior management is crucial in patients with multiple risks of cardiovascular mortality. Objective: This study aimed to examine the efficacy of a 3-month Smart Management Strategy for Health–based electronic program (Smart Healthing). Methods: A 2-arm randomized controlled trial was conducted to assess the efficacy of Smart Healthing in 106 patients with at least one indicator of poor disease control and who had hypertension, diabetes, or hypercholesterolemia. The intervention group (n=53) took part in the electronic program, which was available in the form of a mobile app and a Web-based PC application. The program covered 4 areas: self-assessment, self-planning, self-learning, and self-monitoring by automatic feedback. The control group (n=53) received basic educational material concerning disease control. The primary outcome was the percentage of participants who achieved their clinical indicator goal after 12 weeks into the program: glycated hemoglobin (HbA1c) <7.0%, systolic blood pressure (SBP) <140 mmHg, or low-density lipoprotein cholesterol <130 mg/dL. Results: The intervention group showed a significantly higher success rate (in comparison with the control group) for achieving each of 3 clinical indicators at the targeted goal levels (P<.05). Only the patients with hypertension showed a significant improvement in SBP from the baseline as compared with the control group (72.7% vs 35.7%; P<.05). There was a significant reduction in HbA1c in the intervention group compared with the control group (difference=0.54%; P≤.05). In the intervention group, 20% of patients with diabetes exhibited a ≥1% decrease in HbA1c (vs 0% among controls; P≤.05). Conclusions: A short-term self-management strategy-based electronic program intervention may improve clinical outcomes among patients with cardiovascular risks. Clinical Trial: NCT03294044;

  • The EPIO app. Source: Image created by the Authors; Copyright: Lise Solberg Nes; URL:; License: Creative Commons Attribution (CC-BY).

    A User-Centered Approach to an Evidence-Based Electronic Health Pain Management Intervention for People With Chronic Pain: Design and Development of EPIO


    Background: Chronic pain conditions are complicated and challenging to live with. Electronic health (eHealth) interventions show promise in helping people cope with chronic illness, including pain. The success of these interventions depends not only on the technology and intervention content but also on the users’ acceptance and adherence. Involving all stakeholders (eg, patients, spouses, health care providers, designers, software developers, and researchers) and exploring their input and preferences in the design and development process is an important step toward developing meaningful interventions and possibly strengthening treatment outcomes. Objective: The aim of this study was to design and develop a user-centered, evidence-based eHealth self-management intervention for people with chronic pain. Methods: The study employed a multidisciplinary and user-centered design approach. Overall, 20 stakeholders from the project team (ie, 7 researchers, 5 editors, 7 software developers, and 1 user representative), together with 33 external stakeholders (ie, 12 health care providers, 1 health care manger, 1 eHealth research psychologist, and 17 patients with chronic pain and 2 of their spouses) participated in a user-centered development process that included workshops, intervention content development, and usability testing. Intervention content was developed and finalized based on existing evidence, stakeholder input, and user testing. Stakeholder input was examined through qualitative analyses with rapid and in-depth analysis approaches. Results: Analyses from stakeholder input identified themes including a need for reliable, trustworthy, and evidence-based content, personalization, options for feedback, behavioral tracking, and self-assessment/registration as factors to include in the intervention. Evidence-based intervention content development resulted in one face-to-face introduction session and 9 app-based educational and exercise-based modules. Usability testing provided further insight into how to optimize the design of the intervention to the user group, identifying accessibility and a simple design to be essential. Conclusions: The design and development process of eHealth interventions should strive to combine well-known evidence-based concepts with stakeholder input. This study, designing and developing the pain management intervention EPIO, illustrates how a stakeholder-centered design approach can provide essential input in the development of an eHealth self-management intervention for people with chronic pain. Clinical Trial: NCT03705104;

  • The Saliva Collection Kit. Source: Flickr; Copyright: Marco Verch; URL:; License: Creative Commons Attribution (CC-BY).

    Valuable Genomes: Taxonomy and Archetypes of Business Models in Direct-to-Consumer Genetic Testing


    Background: Recent progress in genome data collection and analysis technologies has led to a surge of direct-to-consumer (DTC) genetic testing services. Owing to the clinical value and sensitivity of genomic data, as well as uncertainty and hearsay surrounding business practices of DTC genetic testing service providers, DTC genetic testing has faced significant criticism by researchers and practitioners. Research in this area has centered on ethical and legal implications of providing genetic tests directly to consumers, but we still lack a more profound understanding of how businesses in the DTC genetic testing markets work and provide value to different stakeholders. Objective: The aim of this study was to address the lack of knowledge concerning business models of DTC genetic testing services by systematically identifying the salient properties of various DTC genetic testing service business models as well as discerning dominant business models in the market. Methods: We employed a 3-phased research approach. In phase 1, we set up a database of 277 DTC genetic testing services. In phase 2, we drew on these data as well as conceptual models of DTC genetic testing services and iteratively developed a taxonomy of DTC genetic testing service business models. In phase 3, we used a 2-stage clustering method to cluster the 277 services that we identified during phase 1 and derived 6 dominant archetypes of DTC genetic testing service business models. Results: The contributions of this research are 2-fold. First, we provided a first of its kind, systematically developed taxonomy of DTC genetic testing service business models consisting of 15 dimensions in 4 categories. Each dimension comprises 2 to 5 characteristics and captures relevant aspects of DTC genetic testing service business models. Second, we derived 6 archetypes of DTC genetic testing service business models named as follows: (1) low-cost DTC genomics for enthusiasts, (2) high-privacy DTC genomics for enthusiasts, (3) specific information tests, (4) simple health tests, (5) basic low-value DTC genomics, and (6) comprehensive tests and low data processing. Conclusions: Our analysis paints a much more complex business landscape in the DTC genetic testing market than previously anticipated. This calls for further research on business models and their effects that underlie DTC genetic testing services and invites specific regulatory interventions to protect consumers and level the playing field.

Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Latest Submissions Open for Peer-Review:

View All Open Peer Review Articles
  • Appalachian Informatics Platform: A Multicentric Data Warehouse with Embedded Data Analytics and Interactive Visualization for Clinical and Translational Research

    Date Submitted: Jan 24, 2020

    Open Peer Review Period: Jan 24, 2020 - Mar 20, 2020

    Background: The Appalachian population is distinct, not just culturally and geographically, but also in its health care needs, facing the most health care disparities in the United States. Thus, to me...

    Background: The Appalachian population is distinct, not just culturally and geographically, but also in its health care needs, facing the most health care disparities in the United States. Thus, to meet these unique demands, bringing in modern technologies to Appalachian medical centers and utilizing these technologies to develop solutions becomes critical. This requires the foundation of a centralized data warehouse with an arsenal of analytics and data science tools to transform healthcare data into actionable clinical interventions. It’s an especially challenging task given the fragmented state of medical data within Appalachia and the need for integration of other types of data such as environmental, social, economic, etc. with medical data. Objective: This paper presents the initial experience of developing an integrated platform at a mid-level Appalachian academic medical center along with its initial uses. Methods: The Appalachian Informatics Platform was developed by the Appalachian Clinical and Translational Science Institute’s Division of Clinical Informatics and consists of four major components: a centralized clinical data warehouse, modeling (statistical and machine learning), visualization, and model evaluation. Results: The Appalachian informatics platform is functional and has supported several research efforts since its implementation. Conclusions: The platform provides an inexpensive yet seamless way to translate clinical and translational research ideas into clinical applications for regions like Appalachia that have limited resources and a largely rural population.

  • Examining the Relationship of Portal Usage and Self-Efficacious Health Information Seeking Behaviors Using HINTS Data: A Causal Inference Framework

    Date Submitted: Jan 14, 2020

    Open Peer Review Period: Jan 14, 2020 - Mar 10, 2020

    Background: Much attention has been drawn to patient portals as an important tool for health providers to facilitate patient engagement. However, little is known about whether the use of patient porta...

    Background: Much attention has been drawn to patient portals as an important tool for health providers to facilitate patient engagement. However, little is known about whether the use of patient portals contributes to improved management of patients’ health, within the context of their confidence in acquiring health information and their ability to monitor and take good care of their own health, due to the lack of an observational or randomized dataset with measurements of self-efficacy outcomes both pre- and post-adoption of patient portals. Objective: To develop a testing framework that allows for causal inference and use it to investigate whether the exposure to patient portals, or intensively using a patient portal, will improve patients’ self-efficacy towards obtaining health information and taking care of themselves. Methods: This study was a secondary data analysis that used data from the National Cancer Institute’s HINTS 5 Cycle 1 data. Patient portal usage frequency was used to define the treatment, and survey items measuring self-efficacy were selected as the main outcomes, including patients’ confidence in getting health information and taking care of their own health. To enable establishing causality from survey data, we propose a novel testing framework that can identify the causal relationship using instrumental variables and conditional independent tests. Results: Using patient portals is shown to improve patients’ confidence of getting health information. The estimand of the weighted average causal effect from two-stage least squares regression is 0.15, with a 95% Confidence Interval (0.06, 0.23), p<0.001. It means that increasing the portal usage intensity, for instance, from 1-2 times to 3-5 times, the expected average increase in the response score (measured on a Likert-type scale) is 0.15. However, we cannot conclude a causal effect of using patient portals on patients’ confidence in self-care. Conclusions: The proposed statistical method exploits the potential of national survey data for causal inference studies and the results advocate patient portals and promote the need to provide better support and education to patients. It also urges randomized controlled studies to further investigate this identified treatment effect.

  • Artificial Intelligence and Health Technology Assessment: Anticipating a New Level of Complexity

    Date Submitted: Jan 11, 2020

    Open Peer Review Period: Jan 11, 2020 - Mar 7, 2020

    Artificial Intelligence is seen as a strategic lever to improve access, quality and efficiency of care and services, and to build learning and value-based health systems. Many studies examined the tec...

    Artificial Intelligence is seen as a strategic lever to improve access, quality and efficiency of care and services, and to build learning and value-based health systems. Many studies examined the technical performance of AI within an experimental context. These studies provide limited insights into the issues that its use in a real-world context of care and services (RWCCS) raises. To help decision-makers address these issues in a systemic and holistic way, this article relies on the "Health Technology Assessment (HTA) core model" to contrast the expectations of the health sector towards the use of AI with the risks that should be mitigated for its responsible deployment. The analysis adopts the perspective of payors because of their central role in regulating, financing and reimbursing novel technologies. This article suggests that AI-based systems should be seen as a health system transformation lever, rather than a discrete set of technological devices. Their use could bring significant changes and impacts at several levels: technological, clinical, human and cognitive (patient and clinician), professional and organizational, economic, legal and ethical. The assessment of the "AI value proposition" should thus go beyond "technical performance" and "price" logics by performing a holistic analysis of value in a RWCCS. In order to guide AI developments, generate knowledge and draw lessons that can be translated into action, the right political, regulatory, organizational, clinical and technological conditions for innovation should be created as a first step.

  • A feature-based hybrid recommender system for risk prediction : Machine learning approach

    Date Submitted: Jan 6, 2020

    Open Peer Review Period: Jan 9, 2020 - Mar 5, 2020

    Background: In the era of health informatics, exponential growth of information generated by health information systems and healthcare organizations demands expert and intelligent recommendation syste...

    Background: In the era of health informatics, exponential growth of information generated by health information systems and healthcare organizations demands expert and intelligent recommendation systems. It has become one of the most valuable tools as it reduces problems such as information overload while selecting and suggesting doctors, hospitals, medicine, diagnosis etc according to patients’ interests. Objective: Recommendation uses Hybrid Filtering as one of the most popular approaches, but the major limitations of this approach are selectivity and data integrity issues.Mostly existing recommendation systems & risk prediction algorithms focus on a single domain, on the other end cross-domain hybrid filtering is able to alleviate the degree of selectivity and data integrity problems to a better extent. Methods: We propose a novel algorithm for recommendation & predictive model using KNN algorithm with machine learning algorithms and artificial intelligence (AI). We find the factors that directly impact on diseases and propose an approach for predicting the correct diagnosis of different diseases. We have constructed a series of models with good reliability for predicting different surgery complications and identified several novel clinical associations. We proposed a novel algorithm pr-KNN to use KNN for prediction and recommendation of diseases Results: Beside that we compared the performance of our algorithm with other machine algorithms and found better performance of our algorithm, with predictive accuracy improving by +3.61%. Conclusions: The potential to directly integrate these predictive tools into EHRs may enable personalized medicine and decision-making at the point of care for patient counseling and as a teaching tool. Clinical Trial: dataset for the trials of patient attached

  • Using new technology during an OSCE of medical student

    Date Submitted: Jan 7, 2020

    Open Peer Review Period: Jan 7, 2020 - Mar 3, 2020

    Background: The objective structured clinical examination (OSCE) is a new test that evaluates the clinical competencies of sixth-year medical students in Spain. Objective: In this framework the main g...

    Background: The objective structured clinical examination (OSCE) is a new test that evaluates the clinical competencies of sixth-year medical students in Spain. Objective: In this framework the main goal is to explore possible applications and usefulness of portable eye tracking systems in the context of this new kind of tests, particularly questions related to attention and engagement. Methods: We used the portable Tobii Glasses 2 eye tracker, which monitors, in real time, the image and sounds perceived by the people wearing the device. We performed both a qualitative and quantitative analysis on the fields of vision and gaze points attracting attention as well as the visual itinerary. Results: The greatest utility of portable systems lies in patient simulators and mannequin stations. This technology proved to be useful to better identify the areas of the medical images that were provided. Portable eye trackers offer the opportunity to improve the objective evaluation of candidates and the self-evaluation of the used stations and medical simulations by examiners Conclusions: Our preliminary results suggest what elements of the OSCE are most amenable to evaluation through eye tracking and provide insights for the design of future studies in this field.

  • The effectiveness of a web-based psycho-education program in a community in Selangor, Malaysia: A randomized controlled trial

    Date Submitted: Jan 2, 2020

    Open Peer Review Period: Jan 2, 2020 - Feb 27, 2020

    Background: Background Mental health problems namely depression and anxiety are the most common problems in the community. Often patients do not seek professional care due to the stigma attached to i...

    Background: Background Mental health problems namely depression and anxiety are the most common problems in the community. Often patients do not seek professional care due to the stigma attached to it. Objective: The study aimed to determine the effectiveness of a web-based psycho-education program in managing mild depression and anxiety. Methods Methods: A two-arm randomized controlled trial of a single blinded, parallel study comparing a four weeks of web-based psycho-education intervention program versus a wait list control group was carried out. The intervention program consisted of four sessions, with each session accessed on a weekly basis. Participants aged 18 years and above, who have participated in the first phase of this study, having access to internet and who are internet literate were invited to participate in the study. By using a random number table, 119 eligible and consented participants were randomly assigned to either the intervention or the control group using random number table. The primary outcomes were depression and anxiety score while the secondary outcome was mental health literacy score, which were all assessed at baseline, week 5 and week 12. Analysis was based on intention to treat analysis. Results: Significant difference in the mental health literacy score between the intervention and the control group was observed, F (1,117) = 20.149, p<0.001, n2=0.142. No significant difference was found in the depression (p= 0.361) and anxiety scores (p= 0.797). Conclusions: The psycho-education intervention was effective in increasing the mental health literacy of the participants. Clinical Trial: The trial is registered in International Standard Randomized Controlled Trial, ISRCTN 39656144.