RefCheck Maintenance Notice

On Monday, December 3, 2018, from 16:00-18:00 EST, RefCheck will be undergoing maintenance. RefCheck is the process where, during copyediting, all references are extracted from the manuscript file, parsed, matched against various databases (eg, PubMed and CrossRef), and automatically corrected. For more information on RefCheck, please visit our Knowledge Base.

Who will be affected?


Journal Description

The Journal of Medical Internet Research (JMIR), now in its 20th year, is the pioneer open access eHealth journal and is the flagship journal of JMIR Publications. It is the leading digital health journal globally in terms of quality/visibility (Impact Factor 2017: 4.671, ranked #1 out of 22 journals) and in terms of size (number of papers published). The journal focuses on emerging technologies, medical devices, apps, engineering, and informatics applications for patient education, prevention, population health and clinical care. As leading high-impact journal in its' disciplines (health informatics and health services research), it is selective, but it is now complemented by almost 30 specialty JMIR sister journals, which have a broader scope. Peer-review reports are portable across JMIR journals and papers can be transferred, so authors save time by not having to resubmit a paper to different journals. 

As open access journal, we are read by clinicians, allied health professionals, informal caregivers, and patients alike, and have (as all JMIR journals) a focus on readable and applied science reporting the design and evaluation of health innovations and emerging technologies. We publish original research, viewpoints, and reviews (both literature reviews and medical device/technology/app reviews).

We are also a leader in participatory and open science approaches, and offer the option to publish new submissions immediately as preprints, which receive DOIs for immediate citation (eg, in grant proposals), and for open peer-review purposes. We also invite patients to participate (eg, as peer-reviewers) and have patient representatives on editorial boards.

Be a widely cited leader in the digitial health revolution and submit your paper today!


Recent Articles:

  • The SensiumVitals patch. Source: Sensium; Copyright: Sensium; URL:; License: Licensed by JMIR.

    Continuous Versus Intermittent Vital Signs Monitoring Using a Wearable, Wireless Patch in Patients Admitted to Surgical Wards: Pilot Cluster Randomized...


    Background: Vital signs monitoring is a universal tool for the detection of postoperative complications; however, unwell patients can be missed between traditional observation rounds. New remote monitoring technologies promise to convey the benefits of continuous monitoring to patients in general wards. Objective: The aim of this pilot study was to evaluate whether continuous remote vital signs monitoring is a practical and acceptable way of monitoring surgical patients and to optimize the delivery of a definitive trial. Methods: We performed a prospective, cluster-randomized, parallel-group, unblinded, controlled pilot study. Patients admitted to 2 surgical wards at a large tertiary hospital received either continuous and intermittent vital signs monitoring or intermittent monitoring alone using an early warning score system. Continuous monitoring was provided by a wireless patch, worn on the patient’s chest, with data transmitted wirelessly every 2 minutes to a central monitoring station or a mobile device carried by the patient’s nurse. The primary outcome measure was time to administration of antibiotics in sepsis. The secondary outcome measures included the length of hospital stay, 30-day readmission rate, mortality, and patient acceptability. Results: Overall, 226 patients were randomized between January and June 2017. Of 226 patients, 140 were randomized to continuous remote monitoring and 86 to intermittent monitoring alone. On average, patients receiving continuous monitoring were administered antibiotics faster after evidence of sepsis (626 minutes, n=22, 95% CI 431.7-820.3 minutes vs 1012.8 minutes, n=12, 95% CI 425.0-1600.6 minutes), had a shorter average length of hospital stay (13.3 days, 95% CI 11.3-15.3 days vs 14.6 days, 95% CI 11.5-17.7 days), and were less likely to require readmission within 30 days of discharge (11.4%, 95% CI 6.16-16.7 vs 20.9%, 95% CI 12.3-29.5). Wide CIs suggest these differences are not statistically significant. Patients found the monitoring device to be acceptable in terms of comfort and perceived an enhanced sense of safety, despite 24% discontinuing the intervention early. Conclusions: Remote continuous vital signs monitoring on surgical wards is practical and acceptable to patients. Large, well-controlled studies in high-risk populations are required to determine whether the observed trends translate into a significant benefit for continuous over intermittent monitoring. Trial Registration: International Standard Randomised Controlled Trial Number ISRCTN60999823; /ISRCTN60999823 (Archived by WebCite at

  • A dementia patient and his caregiver. Source: Pixabay; Copyright: truthseeker08; URL:; License: Public Domain (CC0).

    Mobile Health, Information Preferences, and Surrogate Decision-Making Preferences of Family Caregivers of People With Dementia in Rural Hispanic Communities:...


    Background: Mobile health (mHealth) technology holds promise for promoting health education and reducing health disparities and inequalities in underserved populations. However, little research has been done to develop mHealth interventions for family caregivers of people with dementia, particularly those in rural Hispanic communities, who often serve as surrogate decision makers for their relatives with dementia. Objective: As part of a larger project to develop and test a novel, affordable, and easy-to-use mHealth intervention to deliver individually tailored materials in rural Hispanic communities, in this pilot study, we aimed to examine (1) characteristics of people with dementia and their family caregivers in rural Hispanic communities, (2) caregivers’ preferences for types and amounts of health information and participation in surrogate decision making, and (3) caregivers’ mobile device usage and their desire for receiving information via mobile devices. Methods: This was a cross-sectional survey. A convenience sample of 50 caregivers of people with dementia was recruited from rural health care facilities in Southwest Texas during 3 weeks of April 2017 to May 2017 via word-of-mouth and flyers posted at the facilities. Results: More women than men were in the patient group (χ21=17.2, P<.001) and in the caregiver group (χ21=22.2, P<.001). More patients were on Medicare and Medicaid; more caregivers had private insurance (P<.001 in all cases). Overall, 42% of patients did not have a power of attorney for their health care; 40% did not have a living will or advance directive. Caregivers were interested in receiving all types of information and participating in all types of decisions, although on subscales for diagnosis, treatment, laboratory tests, self-care, and complementary and alternative medicine, their levels of interest for decision-making participation were significantly lower than those for receiving information. On the psychosocial subscale, caregivers’ desire was greater for surrogate decision-making participation than for information. Caregivers did not differ in their interests in information and participation in decision making on the health care provider subscale. All but 1 caregiver (98%) owned a mobile phone and 84% had a smartphone. Two-thirds wanted to receive at least a little dementia-related information via a smartphone or tablet. The amount of dementia-related information caregivers wanted to receive via a mobile device was significantly greater for women than for men (U=84.50, P=.029). Caregivers who owned a tablet were more likely to want to receive dementia-related information via a mobile device than those who did not own a tablet (U=152.0, P=.006). Conclusions: Caregivers in rural Hispanic communities were interested in receiving a wide range of information as well as participating in making decisions for their relatives with dementia. There is much need for effective mHealth interventions that can provide information tailored to the needs and preferences of these caregivers.

  • Source: iStock by Getty Images; Copyright: Pixelfit; URL:; License: Licensed by the authors.

    Creating Engaging Health Promotion Campaigns on Social Media: Observations and Lessons From Fitbit and Garmin


    Background: The popularity and reach of social media make it an ideal delivery platform for interventions targeting health behaviors, such as physical inactivity. Research has identified a dose-response relationship whereby greater engagement and exposure are positively associated with intervention effects, hence enhancing engagement will maximize the potential of these interventions. Objective: This study examined the social media activity of successful commercial activity tracker brands to understand which creative elements (message content and design) they use in their communication to their audience, which social media platforms attract the most engagement, and which creative elements prompted the most engagement. Methods: Posts (n=509) made by Fitbit and Garmin on Facebook, Twitter, and Instagram over a 3-month period were coded for the presence of creative elements. User engagement regarding the total number of likes, comments, or shares per post was recorded. Negative binomial regression analyses were used to identify creative elements associated with higher engagement. Results: Engagement on Instagram was 30-200 times higher than on Facebook, or Twitter. Fitbit and Garmin tended to use different creative elements from one another. A higher engagement was achieved by posts featuring an image of the product, highlighting new product features and with themes of self-improvement (P<.01). Conclusions: Findings suggest that Instagram may be a particularly promising platform for delivering engaging health messaging. Health messages which incorporate inspirational imagery and focus on a tangible product appear to achieve the highest engagement. Fitbit and Garmin employed difference creative elements, which is likely to reflect differences in their target markets. This underscores the importance of market segmentation in health messaging campaigns.

  • The iREST app (on the mobile phone) connected in realtime with the iREST clinician portal (on the desktop). Source: Image created by the Authors; Copyright: The Authors; URL:; License: Creative Commons Attribution + Noncommercial + NoDerivatives (CC-BY-NC-ND).

    Clinical Feasibility of a Just-in-Time Adaptive Intervention App (iREST) as a Behavioral Sleep Treatment in a Military Population: Feasibility Comparative...


    Background: Although evidence-based cognitive behavioral sleep treatments have been shown to be safe and effective, these treatments have limited scalability. Mobile health tools can address this scalability challenge. iREST, or interactive Resilience Enhancing Sleep Tactics, is a mobile health platform designed to provide a just-in-time adaptive intervention (JITAI) in the assessment, monitoring, and delivery of evidence-based sleep recommendations in a scalable and personalized manner. The platform includes a mobile phone–based patient app linked to a clinician portal. Objective: The first aim of the pilot study was to evaluate the effectiveness of JITAI using the iREST platform for delivering evidence-based sleep interventions in a sample of military service members and veterans. The second aim was to explore the potential effectiveness of this treatment delivery form relative to habitual in-person delivery. Methods: In this pilot study, military service members and veterans between the ages of 18 and 60 years who reported clinically significant service-related sleep disturbances were enrolled as participants. Participants were asked to use iREST for a period of 4 to 6 weeks during which time they completed a daily sleep/wake diary. Through the clinician portal, trained clinicians offered recommendations consistent with evidence-based behavioral sleep treatments on weeks 2 through 4. To explore potential effectiveness, self-report measures were used, including the Insomnia Severity Index (ISI), the Pittsburgh Sleep Quality Index (PSQI), and the PSQI Addendum for Posttraumatic Stress Disorder. Results: A total of 27 participants completed the posttreatment assessments. Between pre- and postintervention, clinically and statistically significant improvements in primary and secondary outcomes were detected (eg, a mean reduction on the ISI of 9.96, t26=9.99, P<.001). At posttreatment, 70% (19/27) of participants met the criteria for treatment response and 59% (16/27) achieved remission. Comparing these response and remission rates with previously published results for in-person trials showed no significant differences. Conclusion: Participants who received evidence-based recommendations from their assigned clinicians through the iREST platform showed clinically significant improvements in insomnia severity, overall sleep quality, and disruptive nocturnal disturbances. These findings are promising, and a larger noninferiority clinical trial is warranted.

  • From 2005 to 2016, 7712 pregnant women completed the NINFEA (Nascita e Infanzia: gli Effetti dell’Ambiente) baseline questionnaire. Source: Pixabay; Copyright: shaila19; URL:; License: Public Domain (CC0).

    Questionnaire Breakoff and Item Nonresponse in Web-Based Questionnaires: Multilevel Analysis of Person-Level and Item Design Factors in a Birth Cohort


    Background: Web-based questionnaires are increasingly used in epidemiologic studies, as traditional methods are facing a decrease in response rates and an increase in costs. However, few studies have investigated factors related to the level of completion of internet-based epidemiologic questionnaires. Objective: Our objective was to identify person-level characteristics and item design factors associated with breakoff (not finishing the questionnaire) and item nonresponse in a Web-based questionnaire. Methods: This study was a cross-sectional analysis of the baseline questionnaire, applied from 2005 to 2016, of the Italian NINFEA (Nascita e Infanzia: gli Effetti dell’Ambiente) birth cohort. The baseline questionnaire was administered to enrolled women, who could register at any time during pregnancy. We used logistic regression to analyze the influence of person-level factors on questionnaire breakoff, and a logistic multilevel model (first level: items of the questionnaire; second level: sections of the questionnaire; third level: study participants) to analyze the influence of person-level and item design factors on item nonresponse. Since the number of applicable items depended on the respondent’s characteristics and breakoff, we used inverse probability weighting to deal with missing by design. Results: Of 5970 women, 519 (8.69%) did not finish the questionnaire. Older age (adjusted odds ratio 1.40, 95% CI 1.05-1.88), lower educational level (adjusted odds ratio [OR] 1.53, 95% CI 1.23-1.90), and earlier stage of pregnancy (adjusted OR 3.01, 95% CI 2.31-3.92) were positively associated with questionnaire breakoff. Of the 1,062,519 applicable items displayed for the participants, 22,831 were not responded to (overall prevalence of item nonresponse 2.15%). Item nonresponse was positively associated with older age (adjusted OR 1.25, 95% CI 1.14-1.38), being in the first trimester of pregnancy (adjusted OR 1.18, 95% CI 1.06-1.31), and lower educational level (adjusted OR 1.23, 95% CI 1.14-1.33). Dropdown menu items (adjusted OR 1.77, 95% CI 1.56-2.00) and items organized in grids (adjusted OR 1.69, 95% CI 1.49-1.91) were positively associated with item nonresponse. Conclusions: It is important to use targeted strategies to keep participants motivated to respond. Item nonresponse in internet-based questionnaires is affected by person-level and item design factors. Some item types should be limited to reduce item nonresponse.

  • Source: Pixabay; Copyright: Holger Langmaier; URL:; License: Public Domain (CC0).

    Exploring the Utility of Community-Generated Social Media Content for Detecting Depression: An Analytical Study on Instagram


    Background: The content produced by individuals on various social media platforms has been successfully used to identify mental illness, including depression. However, most of the previous work in this area has focused on user-generated content, that is, content created by the individual, such as an individual’s posts and pictures. In this study, we explored the predictive capability of community-generated content, that is, the data generated by a community of friends or followers, rather than by a sole individual, to identify depression among social media users. Objective: The objective of this research was to evaluate the utility of community-generated content on social media, such as comments on an individual’s posts, to predict depression as defined by the clinically validated Patient Health Questionnaire-8 (PHQ-8) assessment questionnaire. We hypothesized that the results of this research may provide new insights into next generation of population-level mental illness risk assessment and intervention delivery. Methods: We created a Web-based survey on a crowdsourcing platform through which participants granted access to their Instagram profiles as well as provided their responses to PHQ-8 as a reference standard for depression status. After data quality assurance and postprocessing, the study analyzed the data of 749 participants. To build our predictive model, linguistic features were extracted from Instagram post captions and comments, including multiple sentiment scores, emoji sentiment analysis results, and meta-variables such as the number of likes and average comment length. In this study, 10.4% (78/749) of the data were held out as a test set. The remaining 89.6% (671/749) of the data were used to train an elastic-net regularized linear regression model to predict PHQ-8 scores. We compared different versions of this model (ie, a model trained on only user-generated data, a model trained on only community-generated data, and a model trained on the combination of both types of data) on a test set to explore the utility of community-generated data in our predictive analysis. Results: The 2 models, the first trained on only community-generated data (area under curve [AUC]=0.71) and the second trained on a combination of user-generated and community-generated data (AUC=0.72), had statistically significant performances for predicting depression based on the Mann-Whitney U test (P=.03 and P=.02, respectively). The model trained on only user-generated data (AUC=0.63; P=.11) did not achieve statistically significant results. The coefficients of the models revealed that our combined data classifier effectively amalgamated both user-generated and community-generated data and that the 2 feature sets were complementary and contained nonoverlapping information in our predictive analysis. Conclusions: The results presented in this study indicate that leveraging community-generated data from social media, in addition to user-generated data, can be informative for predicting depression among social media users.

  • Source: Flickr; Copyright: Neeta Lind; URL:; License: Creative Commons Attribution (CC-BY).

    eHealth Engagement as a Response to Negative Healthcare Experiences: Cross-Sectional Survey Analysis


    Background: eHealth provides individuals with new means of accessing health information and communicating with providers through online channels. Prior evidence suggests that patients use eHealth to find information online when they receive care that is low in patient centeredness. However, it is unclear how other problems with the healthcare-delivery system motivate the use of eHealth, how these problems relate to different kinds of eHealth activities, and which populations are most likely to use eHealth when they receive low-quality care. Objective: We aimed to determine how two types of negative care experiences—low patient centeredness and care coordination problems—motivate the use of different eHealth activities, and whether more highly educated individuals, who may find these tools easier to use, are more likely to use eHealth following negative experiences than less highly educated individuals. Methods: Using nationally representative data from the 2017 Health Information National Trends Survey, we used factor analysis to group 25 different eHealth activities into categories based on the correlation between respondents’ reports of their usage. Subsequently, we used multivariate negative binomial generalized linear model regressions to determine whether negative healthcare experiences predicted greater use of these resulting categories. Finally, we stratified our sample based on education level to determine whether the associations between healthcare experiences and eHealth use differed across groups. Results: The study included 2612 individuals. Factor analysis classified the eHealth activities into two categories: provider-facing (eg, facilitating communication with providers) and independent (eg, patient-driven information seeking and communication with non-providers). Negative care experiences were not associated with provider-facing eHealth activity in the overall population (care coordination: P=.16; patient centeredness: P=.57) or among more highly educated respondents (care coordination: P=.73; patient centeredness: P=.32), but respondents with lower education levels who experienced problems with care coordination used provider-facing eHealth more often (IRR=1.40, P=.07). Individuals engaged in more independent eHealth activities if they experienced problems with either care coordination (IRR=1.15 P=.01) or patient-centered communication (IRR=1.16, P=.01). Although care coordination problems predicted independent eHealth activity across education levels (higher education: IRR=1.13 P=.01; lower education: IRR=1.19, P=.07), the relationship between low perceived patient centeredness and independent activity was limited to individuals with lower education levels (IRR=1.25, P=.02). Conclusions: Individuals use a greater number of eHealth activities, especially activities that are independent of healthcare providers, when they experience problems with their healthcare. People with lower levels of education seem particularly inclined to use eHealth when they have negative healthcare experiences. To maximize the potential for eHealth to meet the needs of all patients, especially those who are traditionally underserved by the healthcare system, additional work should be performed to ensure that eHealth resources are accessible and usable to all members of the population.

  • Source: Unsplash; Copyright: Claudia / @kaimantha; URL:; License: Licensed by JMIR.

    Accessibility and Applicability of Currently Available e-Mental Health Programs for Depression for People With Poststroke Aphasia: Scoping Review


    Background: Depression affects approximately 60% of people with aphasia 1 year post stroke and is associated with disability, lower quality of life, and mortality. Web-delivered mental health (e-mental health) programs are effective, convenient, and cost-effective for the general population and thus are increasingly used in the management of depression. However, it is unknown if such services are applicable and communicatively accessible to people with poststroke aphasia. Objective: The aim of this study was to identify freely available e-mental health programs for depression and determine their applicability and accessibility for people with poststroke aphasia. Methods: A Web-based search was conducted to identify and review freely available e-mental health programs for depression. These programs were then evaluated in terms of their (1) general features via a general evaluation tool, (2) communicative accessibility for people with aphasia via an aphasia-specific communicative accessibility evaluation tool, and (3) empirical evidence for the general population and stroke survivors with and without aphasia. The program that met the most general evaluation criteria and aphasia-specific communicative accessibility evaluation criteria was then trialed by a small subgroup of people with poststroke aphasia. Results: A total of 8 programs were identified. Of these, 4 had published evidence in support of their efficacy for use within the general population. However, no empirical evidence was identified that specifically supported any programs’ use for stroke survivors with or without aphasia. One evidence-based program scored at least 80% (16/19 and 16/20, respectively) on both the general and aphasia-specific communicative accessibility evaluation tools and was subject to a preliminary trial by 3 people with poststroke aphasia. During this trial, participants were either unable to independently use the program or gave it low usability scores on a post-trial satisfaction survey. On this basis, further evaluation was considered unwarranted. Conclusions: Despite fulfilling majority of the general evaluation and aphasia-specific evaluation criteria, the highest rated program was still found to be unsuitable for people with poststroke aphasia. Thus, e-mental health programs require substantial redevelopment if they are likely to be useful to people with poststroke aphasia.

  • An experimental setting where participants conduct virtual reality tasks in the passive auditory oddball paradigm. Source: Image created by the Authors; Copyright: The Authors; URL:; License: Creative Commons Attribution (CC-BY).

    Involuntary Attention Restoration During Exposure to Mobile-Based 360° Virtual Nature in Healthy Adults With Different Levels of Restorative Experience:...


    Background: With the global trend of urbanization, there are increasing reports of a possible association between decreased exposure to nature and increased occurrence of mental disorders. New 360° virtual reality (VR) technology using smartphones and portable VR glasses can overcome spatial and temporal limitations to help people deal with mental fatigue in everyday life. Objective: On the basis of attention restoration theory (ART), this study aimed to investigate whether the amplitude of the mismatch negativity (MMN)/P3a complex could act as an event-related potential (ERP) biomarker of involuntary attention restoration during exposure to 360° virtual nature in healthy young adults with different levels of restorative VR experience. Methods: A total of 40 healthy adults completed prequestionnaires on demographics and simulator sickness and postquestionnaires on simulator sickness and perceived restorativeness before and after exposure to virtual nature, respectively. During the VR exposure, brain activity was measured by electroencephalography as participants were asked to conduct a 2-tone passive auditory oddball task. Results: The amplitude and latency of the MMN/P3a complex were compared between individuals reporting a highly restorative experience and those reporting a less restorative experience. Although viewing a virtual nature environment, the high restorative group (N=19) exhibited significantly reduced P3a amplitudes compared with the low restorative group (N=20; t38=2.57; P=.02; d=0.59). Particularly, a moderate but significant negative correlation was found between the self-reported restorativeness scores and the P3a amplitudes at the fronto-central region (r=−.38; P=.02). However, the latency of the MMN/P3a complex did not significantly differ between the 2 groups (auditory mismatch negativity: t38=−1.47; P=.15 and P3a: t38=−0.31; P=.76). Conclusions: Considering individuals’ restorative experience, the amplitude of the fronto-central MMN/P3a complex can potentially be employed as a distinct ERP component of interest in involuntary attention restoration during virtual nature experience in healthy young adults. The findings for the 360° virtual nature experience seem to be consistent with those of previous ERP studies on the effects of meditation practice. This study extends the findings of previous ART and ERP studies of real-world meditation, restoration, and mental fatigue management into the virtual world created by mobile phone–based VR glasses and 360° video content.

  • Source: US Department of Defense (Andrew D Sarver); Copyright: US Air Force; URL:; License: Public Domain (CC0).

    Mining Open Payments Data: Analysis of Industry Payments to Thoracic Surgeons From 2014-2016


    Background: The financial relationship between physicians and industries has become a hotly debated issue globally. The Physician Payments Sunshine Act of the US Affordable Care Act (2010) promoted transparency of the transactions between industries and physicians by making remuneration data publicly accessible in the Open Payments Program database. Meanwhile, according to the World Health Organization, the majority of all noncommunicable disease deaths were caused by cardiovascular disease. Objective: This study aimed to investigate the distribution of non-research and non-ownership payments made to thoracic surgeons, to explore the regularity of financial relationships between industries and thoracic surgeons. Methods: Annual statistical data were obtained from the Open Payments Program general payment dataset from 2014-2016. We characterized the distribution of annual payments with single payment transactions greater than US $10,000, quantified the major expense categories (eg, Compensation, Consulting Fees, Travel and Lodging), and identified the 30 highest-paying industries. Moreover, we drew out the financial relations between industries to thoracic surgeons using chord diagram visualization. Results: The three highest categories with single payments greater than US $10,000 were Royalty or License, Compensation, and Consulting Fees. Payments related to Royalty or License transferred from only 5.38% of industries to 0.75% of surgeons with the highest median (US $13,753, $11,992, and $10,614 respectively) in 3-year period. In contrast, payments related to Food and Beverage transferred from 93.50% of industries to 98.48% of surgeons with the lowest median (US $28, $27, and $27). The top 30 highest-paying industries made up approximately 90% of the total payments (US $21,036,972, $23,304,996, and $28,116,336). Furthermore, just under 9% of surgeons received approximately 80% of the total payments in each of the 3 years. Specifically, the 100 highest cumulative payments, accounting for 52.69% of the total, transferred from 27 (6.05%) pharmaceutical industries to 86 (1.89%) thoracic surgeons from 2014-2016; 7 surgeons received payments greater than US $1,000,000; 12 surgeons received payments greater than US $400,000. The majority (90%) of these surgeons received tremendous value from only one industry. Conclusions: There exists a great discrepancy in the distribution of payments by categories. Royalty or License Fees, Compensation, and Consulting Fees are the primary transferring channels of single large payments. The massive transfer from industries to surgeons has a strong “apical dominance” and excludability. Further research should focus on discovering the fundamental driving factors for the strong concentration of certain medical devices and how these payments will affect the industry itself.

  • Source: PxHere; Copyright:; URL:; License: Public Domain (CC0).

    A Practical Do-It-Yourself Recruitment Framework for Concurrent eHealth Clinical Trials: Identification of Efficient and Cost-Effective Methods for Decision...


    Background: The ability to successfully recruit participants for electronic health (eHealth) clinical trials is largely dependent on the use of efficient and effective recruitment strategies. Determining which types of recruitment strategies to use presents a challenge for many researchers. Objective: The aim of this study was to present an analysis of the time-efficiency and cost-effectiveness of recruitment strategies for eHealth clinical trials, and it describes a framework for cost-effective trial recruitment. Methods: Participants were recruited for one of 5 eHealth trials of interventions for common mental health conditions. A multipronged recruitment approach was used, including digital (eg, social media and Craigslist), research registry-based, print (eg, flyers and posters on public transportation), clinic-based (eg, a general internal medicine clinic within an academic medical center and a large nonprofit health care organization), a market research recruitment firm, and traditional media strategies (eg, newspaper and television coverage in response to press releases). The time costs and fees for each recruitment method were calculated, and the participant yield on recruitment costs was calculated by dividing the number of enrolled participants by the total cost for each method. Results: A total of 777 participants were enrolled across all trials. Digital recruitment strategies yielded the largest number of participants across the 5 clinical trials and represented 34.0% (264/777) of the total enrolled participants. Registry-based recruitment strategies were in second place by enrolling 28.0% (217/777) of the total enrolled participants across trials. Research registry-based recruitment had a relatively high conversion rate from potential participants who contacted our center for being screened to be enrolled, and it was also the most cost-effective for enrolling participants in this set of clinical trials with a total cost per person enrolled at US $8.99. Conclusions: On the basis of these results, a framework is proposed for participant recruitment. To make decisions on initiating and maintaining different types of recruitment strategies, the resources available and requirements of the research study (or studies) need to be carefully examined.

  • Clinician and patient using Internetpsykiatrien. Source: Internetpsykiatrien; Copyright: Internetpsykiatrien; URL:; License: Licensed by JMIR.

    Guided Internet-Based Cognitive Behavioral Therapy for Adult Depression and Anxiety in Routine Secondary Care: Observational Study


    Background: Internet-based cognitive behavioral therapy (iCBT) is a promising new treatment method for depression and anxiety. However, it is important to determine whether its results can be replicated in routine care before its implementation on a large scale. Although many studies have demonstrated the efficacy of iCBT under controlled conditions, only a few studies have investigated its effectiveness in routine care. Furthermore, several effects of iCBT such as treatment effects in routine care are unclear. Objective: This study aimed to evaluate the clinical effectiveness of iCBT for depression and anxiety in routine secondary care. Methods: n a retrospective cohort study, we analysed patients treated for depression or anxiety in a dedicated iCBT clinic in secondary care in Denmark. Patients were examined before treatment and weekly thereafter by using the Patient Health Questionnaire-9 and the Generalized Anxiety Disorder-7 scales for the diagnoses of depression and anxiety, respectively. Primary analyses were conducted using a linear mixed-effects model with random slope and intercept. Secondary analyses were conducted using baseline characteristics as predictors (gender, age, highest level of education, occupational status, marital status, psychotropic medication use, consumption of alcohol, and leisure drugs). Additionally, logistic regression analyses were used to predict noncompletion of treatment. Results: A total of 203 (depression, N=60; anxiety, N=143) patients were included. Participants were mainly female (78.3% with depression and 65.7% with anxiety), with a mean age of 36.03 (SD 10.97) years (range, 19-67 years) for patients with depression and 36.80 (SD 13.55) years (range, 19-69 years) for patients with anxiety. The completion rates were 62% (37) and 40% (57) for depression and anxiety treatments, respectively. The primary analyses revealed large and significant reductions in the symptom levels of depression (beta=-6.27, SE 0.83, P<.001, d=1.0) and anxiety (beta=-3.78, SE 0.43, P<.001, d=1.1). High baseline severity of the primary disorder was associated with high treatment gains (r=-0.31 for depression; r=-0.41 for anxiety). In patients with anxiety, high baseline severity also predicted a high risk of noncompletion (odds ratio=1.08, CI=1.01-1.16, P=.03). An increase in the baseline severity of the comorbid disorder slightly increased the risk of noncompletion for both disorders (depression: odds ratio=1.03, CI=1.01-1.06, P=.02; anxiety: odds ratio=1.08, CI=1.01-1.16, P=.03). Conclusions: iCBT can be clinically effective in routine care. Since depression and anxiety are costly and debilitating disorders that are vastly undertreated, this finding is important. Additionally, iCBT may help bridge the gap between the need for treatment and its provision. Our results are comparable to the within-group results of efficacy and effectiveness studies. Our noncompletion rates are similar to those observed in psychotherapy but are higher than those reported in similar clinics. Multiple factors predicted outcome and noncompletion. However, all predictor effects were statistically weak.

Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Latest Submissions Open for Peer-Review:

View All Open Peer Review Articles
  • Measuring regional quality of care using unsolicited online data: creating more detailed insight using text analyses

    Date Submitted: Dec 7, 2018

    Open Peer Review Period: Dec 11, 2018 - Feb 5, 2019

    Background: Regional population health management (PM) initiatives require insight into experienced quality of care at the regional level. Unsolicited online provider ratings have shown potential for...

    Background: Regional population health management (PM) initiatives require insight into experienced quality of care at the regional level. Unsolicited online provider ratings have shown potential for this use. This study explored the addition of comments accompanying unsolicited online ratings to regional analyses. Objective: The goal was to create additional insight for each PM initiatives as well as overall comparisons between these initiatives by attempting to determine the reasoning and rationale behind a rating. Methods: The Dutch Zorgkaart database provided the unsolicited ratings (period: 2008 – 2017) for analyses. All ratings included both quantitative ratings as well as qualitative text comments. Nine PM regions were used to aggregate ratings geographically. Sentiment analyses were performed by categorizing ratings into negative, neutral and positive ratings. Per category, as well as per PM initiative, word frequencies (unigrams and bigrams) were explored. Machine learning (naive Bayes) was applied to identify the most important predictors for rating overall sentiment and for identifying PM initiatives. Results: 449,263 unsolicited ratings were available in the Zorgkaart database, 303,930 positive ratings, 97,739 neutral and 47,592 negative ratings. Bigrams illustrated that feeling like not being “taken serious” was the dominant bigram in negative ratings, while bigrams in positive ratings were mostly related to listening, explaining and perceived knowledge. Comparing bigrams between PM initiatives showed a lot of overlap, but several differences were identified. The naive Bayes machine learning was able to predict sentiments of comments, but unable to distinguish between specific PM initiatives. Conclusions: Adding information from text comments that accompany online ratings to regional evaluations provides insight for PM initiatives into the underlying reasoning. They provide useful overarching information for healthcare policy makers, but due to a lot of overlap it adds little region specific information. Specific outliers for some initiatives are insightful. Clinical Trial: The Medical Research Involving Human Subjects Act (WMO) does not apply to this study, and official approval was not required [1]. Participants agreed to the terms of service of Zorgkaart Nederland, which states that their submission can be used anonymously for research purposes [2].

  • Health professions’ digital education: a review of learning theories in RCTs by the Digital Health Education collaboration

    Date Submitted: Dec 11, 2018

    Open Peer Review Period: Dec 11, 2018 - Dec 21, 2018

    Background: Learning theory is an essential component for designing an effective educational curriculum. Reviews of existing literature consistently lack sufficient evidence to support the effectivene...

    Background: Learning theory is an essential component for designing an effective educational curriculum. Reviews of existing literature consistently lack sufficient evidence to support the effectiveness of digital interventions for health professions’ education, which may reflect disconnections between learning theories, curriculum design, use of technology, and outcome evaluation. Objective: We aim to identify, map and evaluate the use of learning-theories in designing and implementing intervention trials of health professions’ digital education, as well as to highlight areas for future research on technology enhanced education via the establishment of a development framework for practice and research. Methods: Systematic search of MEDLINE, Embase, Cochrane Central Register of Controlled Trials (Cochrane Library), PsycINFO, CINAHL, ERIC and Web of Science for randomized controlled trials (RCTs) published between 2007 and 2016 (ten years). Results: A total 874 RCTs on digital health education were identified and categorized into online-offline, mobile digital education, and simulation-based modalities for pre and post-registration health professions’ education. Of these, 242 studies were randomly selected for methodological review and thematic analysis. Data were extracted by one author using a standardized form, with a 20% random sample extracted by a second author in duplicate. One-third (33.4%; 81/242) studies reported single or multiple learning theories in design, assessment, conceptualization or interpretation of outcomes of the digital education interventions. Commonly reported learning theories were problem-based learning (19.7%), social learning theory (13.5%), and cognitive theory of multimedia learning (12.3%). Most of these studies assessed knowledge (48.8%), skills (25.6%), and performance (24.3%) as primary outcomes with non-validated assessment tools (62.4%). Studies with reported learning theories (p=0.002) and validated instruments (p=0.006) have shown effective acquisition of learning outcomes. Conclusions: We proposed a development framework to safeguard robustness and integrity of the design and implementation of future digital education programmes for the training of health professions.

  • The design of a Care Pathway for preventive blood pressure monitoring

    Date Submitted: Dec 6, 2018

    Open Peer Review Period: Dec 10, 2018 - Feb 4, 2019

    Background: eHealth services could provide a solution for monitoring the blood pressure of at-risk patients while also decreasing expensive doctor visits. However, a major barrier that posed their imp...

    Background: eHealth services could provide a solution for monitoring the blood pressure of at-risk patients while also decreasing expensive doctor visits. However, a major barrier that posed their implementation is the lack of integration into organisations. Objective: To design a Care Pathway for monitoring blood pressure of at-risk patients, in order to increase the implementation of eHealth in secondary preventive care. Methods: A qualitative design study was performed for this research. Data was collected by conducting visual mapping sessions including semi-structured interviews with hypertension patients and doctors. The data was transcribed and coded and thereafter mapped into a Care Pathway. Results: Common agreement on four themes: (1) the current approach to blood pressure measuring has disadvantages; (2) risk and lifestyle factors of blood pressure measuring need to be considered; (3) there are certain influences of the at-home context on measuring blood pressure; and (4) new touchpoints between actors need to be designed. These the in-depth insights combined with the visualisation of the current blood pressure process resulted in the Care Pathway design for monitoring the blood pressure of at-risk patients as secondary preventive care. Conclusions: The Care Pathway guides the implementation of the SBPM eHealth devices. It showcases the pathway of at-risk patients and increases their involvement in managing their blood pressure. It serves as a basis for a new service using eHealth.

  • Data Challenges with Real-time Safety Event Detection & Clinical Decision Support

    Date Submitted: Dec 6, 2018

    Open Peer Review Period: Dec 10, 2018 - Feb 4, 2019

    Background: The continued digitization and maturation of healthcare information technology has made access to real-time data easier and more feasible for more healthcare organizations. With this incre...

    Background: The continued digitization and maturation of healthcare information technology has made access to real-time data easier and more feasible for more healthcare organizations. With this increased availability, the promise of using data to algorithmically detect healthcare-related events in real-time has become more of a reality. However, as more researchers and clinicians have utilized real-time data delivery capabilities, it has become apparent that simply gaining access to the data is not a panacea and some unique data challenges have come to the forefront in the process. Objective: To highlight some of the challenges that are germane to real-time processing of healthcare system-generated data and the accurate interpretation of the results. Methods: Distinct challenges related to the use and processing of real-time data for safety event detection were compiled and reported by several informatics and clinical experts at a quaternary pediatric academic institution. The challenges were collated from the experiences of the researchers implementing real-time event detection on more than half a dozen distinct projects. The challenges are presented in a challenge category/specific challenge/example format. Results: Eight major types of challenge categories are reported, with 13 specific challenges and 9 specific examples detailed to provide a context for the challenges. The examples reported are anchored to a specific project using medication order, medication administration record, and smart infusion pump data to detect discrepancies and errors between the three data sets. Conclusions: The use of real-time data to drive safety event detection and clinical decision support is extremely powerful, but presents its own set of challenges including data quality and technical complexity. These challenges must be recognized and accommodated for if the full promise of accurate, real-time safety event clinical decision support is to be realized.

  • Sleep monitoring and wearables : a systematic review of clinical trials and future applications

    Date Submitted: Dec 6, 2018

    Open Peer Review Period: Dec 10, 2018 - Feb 4, 2019

    Background: Sleep disorders are a major public health issue. Nearly one in two people will experience sleep disturbances during lifetime with a potential harmful impact on well-being, physical and me...

    Background: Sleep disorders are a major public health issue. Nearly one in two people will experience sleep disturbances during lifetime with a potential harmful impact on well-being, physical and mental health. The rise of connected objects is bringing new opportunities in sleep monitoring. Objective: To better understand the clinical value wearables-based sleep monitoring, we conducted a review of the literature, including feasibility studies and clinical trials on this topic. Methods: We searched PubMed, PsycINFO, ScienceDirect, the Cochrane Library, Scopus, and Web of Science up to June 2017. We created the list of keywords based on two domains: wearables and sleep. We used the Preferred Reporting Items for Systematic Reviews and Meta-Analyses to identify, select, and critically appraise relevant research while minimizing bias. Results: The initial research collected 255 articles. 18 articles meeting the inclusion criteria were included in the final analysis. Out of the selected articles, four categories appeared. Feasibility studies propose testing new connected tools during sleep, on small samples of subjects. Population comparison studies propose to evaluate the sleep of patients compared to that of healthy subjects. Several studies evaluated connected objects in comparison with polysomnography, a reference test in sleep assessment. Finally, an article evaluates the impact of sleep disorders in the clinic. Conclusions: We conducted a broad analysis of studies on the clinical and technical aspects of the use of wearables for sleep monitoring. This review of the literature showed that wearables are acceptable and promising monitoring tools in a wide range of clinical applications for sleep monitoring.

  • A Scalable Data Science Platform for Healthcare and Precision Medicine Research

    Date Submitted: Dec 6, 2018

    Open Peer Review Period: Dec 10, 2018 - Feb 4, 2019

    Background: Healthcare data is increasing in volume and complexity. Storing and analyzing this data to implement precision medicine initiatives and data driven research has exceeded the capabilities o...

    Background: Healthcare data is increasing in volume and complexity. Storing and analyzing this data to implement precision medicine initiatives and data driven research has exceeded the capabilities of traditional computer systems. Modern big data platforms must be adapted to the specific demands of healthcare and designed for scalability and growth. Objective: To (1) demonstrate the implementation of a data science platform built on open-source technology within a large, academic healthcare system and (2) describe two computational healthcare applications built on such a platform. Methods: A data science platform based on several open source technologies was deployed to support real-time, big data workloads. Data acquisition workflows for Apache Storm and NiFi were developed in Java and Python to capture patient monitoring and laboratory data for downstream analytics. Results: The use of emerging data management approaches along with open-source technologies such as Hadoop can be used to create integrated data lakes to store large, real-time data sets. This infrastructure also provides a robust analytics platform where healthcare and biomedical research data can be analyzed in near real-time for precision medicine and computational healthcare use cases. Conclusions: The implementation and use of integrated data science platforms offer organizations the opportunity to combine traditional data sets, including data from the electronic health record, with emerging big data sources, such as continuous patient monitoring and real-time laboratory results. These platforms can enable cost-effective and scalable analytics for the information that will be key to the delivery of precision medicine initiatives. Organizations that can take advantage of the technical advances found in data science platforms will have the opportunity to provide comprehensive access to healthcare data for computational healthcare and precision medicine research.