RefCheck Maintenance Notice

On Monday, December 3, 2018, from 16:00-18:00 EST, RefCheck will be undergoing maintenance. RefCheck is the process where, during copyediting, all references are extracted from the manuscript file, parsed, matched against various databases (eg, PubMed and CrossRef), and automatically corrected. For more information on RefCheck, please visit our Knowledge Base.

Who will be affected?


Journal Description

The Journal of Medical Internet Research (JMIR), now in its 20th year, is the pioneer open access eHealth journal and is the flagship journal of JMIR Publications. It is the leading digital health journal globally in terms of quality/visibility (Impact Factor 2017: 4.671, ranked #1 out of 22 journals) and in terms of size (number of papers published). The journal focuses on emerging technologies, medical devices, apps, engineering, and informatics applications for patient education, prevention, population health and clinical care. As leading high-impact journal in its' disciplines (health informatics and health services research), it is selective, but it is now complemented by almost 30 specialty JMIR sister journals, which have a broader scope. Peer-review reports are portable across JMIR journals and papers can be transferred, so authors save time by not having to resubmit a paper to different journals. 

As open access journal, we are read by clinicians, allied health professionals, informal caregivers, and patients alike, and have (as all JMIR journals) a focus on readable and applied science reporting the design and evaluation of health innovations and emerging technologies. We publish original research, viewpoints, and reviews (both literature reviews and medical device/technology/app reviews).

We are also a leader in participatory and open science approaches, and offer the option to publish new submissions immediately as preprints, which receive DOIs for immediate citation (eg, in grant proposals), and for open peer-review purposes. We also invite patients to participate (eg, as peer-reviewers) and have patient representatives on editorial boards.

Be a widely cited leader in the digitial health revolution and submit your paper today!


Recent Articles:

  • Searching for digital interventions to improve asthma self-management outcomes. Source: The Authors; Copyright: Helen J Lycett; URL:; License: Licensed by JMIR.

    Theory-Based Digital Interventions to Improve Asthma Self-Management Outcomes: Systematic Review


    Background: Asthma is a chronic disease requiring effective self-management to control it and prevent mortality. The use of theory-informed digital interventions promoting asthma self-management is increasing. However, there is limited knowledge concerning how and to what extent psychological theory has been applied to the development of digital interventions, or how using theory impacts outcomes. Objective: The study aimed to examine the use and application of theory in the development of digital interventions to enhance asthma self-management and to evaluate the effectiveness of theory-based interventions in improving adherence, self-management, and clinical outcomes. Methods: Electronic databases (CENTRAL, MEDLINE, EMBASE, and PsycINFO) were searched systematically using predetermined terms. Additional studies were identified by scanning references within relevant studies. Two researchers screened titles and abstracts against predefined inclusion criteria; a third resolved discrepancies. Full-text review was undertaken for relevant studies. Those meeting inclusion criteria were assessed for risk of bias using the Cochrane Collaboration tool. The review was conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses statement. Study outcomes were classified as medication adherence, self-management, asthma control, clinical markers of health, quality of life, other quality of life outcomes, and health care utilization. Effectiveness was calculated as an average outcome score based on the study’s reported significance. The Theory Coding Scheme (TCS) was used to establish the extent to which each intervention had applied theory and which theoretical constructs or behavioral determinants were addressed. Associations between TCS scores and asthma outcomes were described within a narrative synthesis. Results: Fourteen studies evaluating 14 different digital interventions were included in this review. The most commonly cited theories were Social Cognitive Theory, Health Belief Model, and Self-Efficacy Theory. A greater use of theory in the development of interventions was correlated with effective outcomes (r=.657; P=.01): only the 3 studies that met >60% of the different uses of theory assessed by the TCS were effective on all behavioral and clinical outcomes measured. None of the 11 studies that met ≤60% of the TCS criteria were fully effective; however, 3 interventions were partially effective (ie, the intervention had a significant impact on some, but not all, of the outcomes measured). Most studies lacked detail on the theoretical constructs and how they were applied to the development and application of the intervention. Conclusions: These findings suggest that greater use of theory in the development and application of digital self-management interventions for asthma may increase their effectiveness. The application of theory alone may not be enough to yield a successful intervention, and other factors (eg, the context in which the intervention is used) should be considered. A systematic approach to the use of theory to guide the design, selection, and application of intervention techniques is needed.

  • The SensiumVitals patch. Source: Sensium; Copyright: Sensium; URL:; License: Licensed by JMIR.

    Continuous Versus Intermittent Vital Signs Monitoring Using a Wearable, Wireless Patch in Patients Admitted to Surgical Wards: Pilot Cluster Randomized...


    Background: Vital signs monitoring is a universal tool for the detection of postoperative complications; however, unwell patients can be missed between traditional observation rounds. New remote monitoring technologies promise to convey the benefits of continuous monitoring to patients in general wards. Objective: The aim of this pilot study was to evaluate whether continuous remote vital signs monitoring is a practical and acceptable way of monitoring surgical patients and to optimize the delivery of a definitive trial. Methods: We performed a prospective, cluster-randomized, parallel-group, unblinded, controlled pilot study. Patients admitted to 2 surgical wards at a large tertiary hospital received either continuous and intermittent vital signs monitoring or intermittent monitoring alone using an early warning score system. Continuous monitoring was provided by a wireless patch, worn on the patient’s chest, with data transmitted wirelessly every 2 minutes to a central monitoring station or a mobile device carried by the patient’s nurse. The primary outcome measure was time to administration of antibiotics in sepsis. The secondary outcome measures included the length of hospital stay, 30-day readmission rate, mortality, and patient acceptability. Results: Overall, 226 patients were randomized between January and June 2017. Of 226 patients, 140 were randomized to continuous remote monitoring and 86 to intermittent monitoring alone. On average, patients receiving continuous monitoring were administered antibiotics faster after evidence of sepsis (626 minutes, n=22, 95% CI 431.7-820.3 minutes vs 1012.8 minutes, n=12, 95% CI 425.0-1600.6 minutes), had a shorter average length of hospital stay (13.3 days, 95% CI 11.3-15.3 days vs 14.6 days, 95% CI 11.5-17.7 days), and were less likely to require readmission within 30 days of discharge (11.4%, 95% CI 6.16-16.7 vs 20.9%, 95% CI 12.3-29.5). Wide CIs suggest these differences are not statistically significant. Patients found the monitoring device to be acceptable in terms of comfort and perceived an enhanced sense of safety, despite 24% discontinuing the intervention early. Conclusions: Remote continuous vital signs monitoring on surgical wards is practical and acceptable to patients. Large, well-controlled studies in high-risk populations are required to determine whether the observed trends translate into a significant benefit for continuous over intermittent monitoring. Trial Registration: International Standard Randomised Controlled Trial Number ISRCTN60999823; /ISRCTN60999823 (Archived by WebCite at

  • A dementia patient and his caregiver. Source: Pixabay; Copyright: truthseeker08; URL:; License: Public Domain (CC0).

    Mobile Health, Information Preferences, and Surrogate Decision-Making Preferences of Family Caregivers of People With Dementia in Rural Hispanic Communities:...


    Background: Mobile health (mHealth) technology holds promise for promoting health education and reducing health disparities and inequalities in underserved populations. However, little research has been done to develop mHealth interventions for family caregivers of people with dementia, particularly those in rural Hispanic communities, who often serve as surrogate decision makers for their relatives with dementia. Objective: As part of a larger project to develop and test a novel, affordable, and easy-to-use mHealth intervention to deliver individually tailored materials in rural Hispanic communities, in this pilot study, we aimed to examine (1) characteristics of people with dementia and their family caregivers in rural Hispanic communities, (2) caregivers’ preferences for types and amounts of health information and participation in surrogate decision making, and (3) caregivers’ mobile device usage and their desire for receiving information via mobile devices. Methods: This was a cross-sectional survey. A convenience sample of 50 caregivers of people with dementia was recruited from rural health care facilities in Southwest Texas during 3 weeks of April 2017 to May 2017 via word-of-mouth and flyers posted at the facilities. Results: More women than men were in the patient group (χ21=17.2, P<.001) and in the caregiver group (χ21=22.2, P<.001). More patients were on Medicare and Medicaid; more caregivers had private insurance (P<.001 in all cases). Overall, 42% of patients did not have a power of attorney for their health care; 40% did not have a living will or advance directive. Caregivers were interested in receiving all types of information and participating in all types of decisions, although on subscales for diagnosis, treatment, laboratory tests, self-care, and complementary and alternative medicine, their levels of interest for decision-making participation were significantly lower than those for receiving information. On the psychosocial subscale, caregivers’ desire was greater for surrogate decision-making participation than for information. Caregivers did not differ in their interests in information and participation in decision making on the health care provider subscale. All but 1 caregiver (98%) owned a mobile phone and 84% had a smartphone. Two-thirds wanted to receive at least a little dementia-related information via a smartphone or tablet. The amount of dementia-related information caregivers wanted to receive via a mobile device was significantly greater for women than for men (U=84.50, P=.029). Caregivers who owned a tablet were more likely to want to receive dementia-related information via a mobile device than those who did not own a tablet (U=152.0, P=.006). Conclusions: Caregivers in rural Hispanic communities were interested in receiving a wide range of information as well as participating in making decisions for their relatives with dementia. There is much need for effective mHealth interventions that can provide information tailored to the needs and preferences of these caregivers.

  • Source: iStock by Getty Images; Copyright: Pixelfit; URL:; License: Licensed by the authors.

    Creating Engaging Health Promotion Campaigns on Social Media: Observations and Lessons From Fitbit and Garmin


    Background: The popularity and reach of social media make it an ideal delivery platform for interventions targeting health behaviors, such as physical inactivity. Research has identified a dose-response relationship whereby greater engagement and exposure are positively associated with intervention effects, hence enhancing engagement will maximize the potential of these interventions. Objective: This study examined the social media activity of successful commercial activity tracker brands to understand which creative elements (message content and design) they use in their communication to their audience, which social media platforms attract the most engagement, and which creative elements prompted the most engagement. Methods: Posts (n=509) made by Fitbit and Garmin on Facebook, Twitter, and Instagram over a 3-month period were coded for the presence of creative elements. User engagement regarding the total number of likes, comments, or shares per post was recorded. Negative binomial regression analyses were used to identify creative elements associated with higher engagement. Results: Engagement on Instagram was 30-200 times higher than on Facebook, or Twitter. Fitbit and Garmin tended to use different creative elements from one another. A higher engagement was achieved by posts featuring an image of the product, highlighting new product features and with themes of self-improvement (P<.01). Conclusions: Findings suggest that Instagram may be a particularly promising platform for delivering engaging health messaging. Health messages which incorporate inspirational imagery and focus on a tangible product appear to achieve the highest engagement. Fitbit and Garmin employed difference creative elements, which is likely to reflect differences in their target markets. This underscores the importance of market segmentation in health messaging campaigns.

  • The iREST app (on the mobile phone) connected in realtime with the iREST clinician portal (on the desktop). Source: Image created by the Authors; Copyright: The Authors; URL:; License: Creative Commons Attribution + Noncommercial + NoDerivatives (CC-BY-NC-ND).

    Clinical Feasibility of a Just-in-Time Adaptive Intervention App (iREST) as a Behavioral Sleep Treatment in a Military Population: Feasibility Comparative...


    Background: Although evidence-based cognitive behavioral sleep treatments have been shown to be safe and effective, these treatments have limited scalability. Mobile health tools can address this scalability challenge. iREST, or interactive Resilience Enhancing Sleep Tactics, is a mobile health platform designed to provide a just-in-time adaptive intervention (JITAI) in the assessment, monitoring, and delivery of evidence-based sleep recommendations in a scalable and personalized manner. The platform includes a mobile phone–based patient app linked to a clinician portal. Objective: The first aim of the pilot study was to evaluate the effectiveness of JITAI using the iREST platform for delivering evidence-based sleep interventions in a sample of military service members and veterans. The second aim was to explore the potential effectiveness of this treatment delivery form relative to habitual in-person delivery. Methods: In this pilot study, military service members and veterans between the ages of 18 and 60 years who reported clinically significant service-related sleep disturbances were enrolled as participants. Participants were asked to use iREST for a period of 4 to 6 weeks during which time they completed a daily sleep/wake diary. Through the clinician portal, trained clinicians offered recommendations consistent with evidence-based behavioral sleep treatments on weeks 2 through 4. To explore potential effectiveness, self-report measures were used, including the Insomnia Severity Index (ISI), the Pittsburgh Sleep Quality Index (PSQI), and the PSQI Addendum for Posttraumatic Stress Disorder. Results: A total of 27 participants completed the posttreatment assessments. Between pre- and postintervention, clinically and statistically significant improvements in primary and secondary outcomes were detected (eg, a mean reduction on the ISI of 9.96, t26=9.99, P<.001). At posttreatment, 70% (19/27) of participants met the criteria for treatment response and 59% (16/27) achieved remission. Comparing these response and remission rates with previously published results for in-person trials showed no significant differences. Conclusion: Participants who received evidence-based recommendations from their assigned clinicians through the iREST platform showed clinically significant improvements in insomnia severity, overall sleep quality, and disruptive nocturnal disturbances. These findings are promising, and a larger noninferiority clinical trial is warranted.

  • From 2005 to 2016, 7712 pregnant women completed the NINFEA (Nascita e Infanzia: gli Effetti dell’Ambiente) baseline questionnaire. Source: Pixabay; Copyright: shaila19; URL:; License: Public Domain (CC0).

    Questionnaire Breakoff and Item Nonresponse in Web-Based Questionnaires: Multilevel Analysis of Person-Level and Item Design Factors in a Birth Cohort


    Background: Web-based questionnaires are increasingly used in epidemiologic studies, as traditional methods are facing a decrease in response rates and an increase in costs. However, few studies have investigated factors related to the level of completion of internet-based epidemiologic questionnaires. Objective: Our objective was to identify person-level characteristics and item design factors associated with breakoff (not finishing the questionnaire) and item nonresponse in a Web-based questionnaire. Methods: This study was a cross-sectional analysis of the baseline questionnaire, applied from 2005 to 2016, of the Italian NINFEA (Nascita e Infanzia: gli Effetti dell’Ambiente) birth cohort. The baseline questionnaire was administered to enrolled women, who could register at any time during pregnancy. We used logistic regression to analyze the influence of person-level factors on questionnaire breakoff, and a logistic multilevel model (first level: items of the questionnaire; second level: sections of the questionnaire; third level: study participants) to analyze the influence of person-level and item design factors on item nonresponse. Since the number of applicable items depended on the respondent’s characteristics and breakoff, we used inverse probability weighting to deal with missing by design. Results: Of 5970 women, 519 (8.69%) did not finish the questionnaire. Older age (adjusted odds ratio 1.40, 95% CI 1.05-1.88), lower educational level (adjusted odds ratio [OR] 1.53, 95% CI 1.23-1.90), and earlier stage of pregnancy (adjusted OR 3.01, 95% CI 2.31-3.92) were positively associated with questionnaire breakoff. Of the 1,062,519 applicable items displayed for the participants, 22,831 were not responded to (overall prevalence of item nonresponse 2.15%). Item nonresponse was positively associated with older age (adjusted OR 1.25, 95% CI 1.14-1.38), being in the first trimester of pregnancy (adjusted OR 1.18, 95% CI 1.06-1.31), and lower educational level (adjusted OR 1.23, 95% CI 1.14-1.33). Dropdown menu items (adjusted OR 1.77, 95% CI 1.56-2.00) and items organized in grids (adjusted OR 1.69, 95% CI 1.49-1.91) were positively associated with item nonresponse. Conclusions: It is important to use targeted strategies to keep participants motivated to respond. Item nonresponse in internet-based questionnaires is affected by person-level and item design factors. Some item types should be limited to reduce item nonresponse.

  • Source: Pixabay; Copyright: Holger Langmaier; URL:; License: Public Domain (CC0).

    Exploring the Utility of Community-Generated Social Media Content for Detecting Depression: An Analytical Study on Instagram


    Background: The content produced by individuals on various social media platforms has been successfully used to identify mental illness, including depression. However, most of the previous work in this area has focused on user-generated content, that is, content created by the individual, such as an individual’s posts and pictures. In this study, we explored the predictive capability of community-generated content, that is, the data generated by a community of friends or followers, rather than by a sole individual, to identify depression among social media users. Objective: The objective of this research was to evaluate the utility of community-generated content on social media, such as comments on an individual’s posts, to predict depression as defined by the clinically validated Patient Health Questionnaire-8 (PHQ-8) assessment questionnaire. We hypothesized that the results of this research may provide new insights into next generation of population-level mental illness risk assessment and intervention delivery. Methods: We created a Web-based survey on a crowdsourcing platform through which participants granted access to their Instagram profiles as well as provided their responses to PHQ-8 as a reference standard for depression status. After data quality assurance and postprocessing, the study analyzed the data of 749 participants. To build our predictive model, linguistic features were extracted from Instagram post captions and comments, including multiple sentiment scores, emoji sentiment analysis results, and meta-variables such as the number of likes and average comment length. In this study, 10.4% (78/749) of the data were held out as a test set. The remaining 89.6% (671/749) of the data were used to train an elastic-net regularized linear regression model to predict PHQ-8 scores. We compared different versions of this model (ie, a model trained on only user-generated data, a model trained on only community-generated data, and a model trained on the combination of both types of data) on a test set to explore the utility of community-generated data in our predictive analysis. Results: The 2 models, the first trained on only community-generated data (area under curve [AUC]=0.71) and the second trained on a combination of user-generated and community-generated data (AUC=0.72), had statistically significant performances for predicting depression based on the Mann-Whitney U test (P=.03 and P=.02, respectively). The model trained on only user-generated data (AUC=0.63; P=.11) did not achieve statistically significant results. The coefficients of the models revealed that our combined data classifier effectively amalgamated both user-generated and community-generated data and that the 2 feature sets were complementary and contained nonoverlapping information in our predictive analysis. Conclusions: The results presented in this study indicate that leveraging community-generated data from social media, in addition to user-generated data, can be informative for predicting depression among social media users.

  • Source: Flickr; Copyright: Neeta Lind; URL:; License: Creative Commons Attribution (CC-BY).

    eHealth Engagement as a Response to Negative Healthcare Experiences: Cross-Sectional Survey Analysis


    Background: eHealth provides individuals with new means of accessing health information and communicating with providers through online channels. Prior evidence suggests that patients use eHealth to find information online when they receive care that is low in patient centeredness. However, it is unclear how other problems with the healthcare-delivery system motivate the use of eHealth, how these problems relate to different kinds of eHealth activities, and which populations are most likely to use eHealth when they receive low-quality care. Objective: We aimed to determine how two types of negative care experiences—low patient centeredness and care coordination problems—motivate the use of different eHealth activities, and whether more highly educated individuals, who may find these tools easier to use, are more likely to use eHealth following negative experiences than less highly educated individuals. Methods: Using nationally representative data from the 2017 Health Information National Trends Survey, we used factor analysis to group 25 different eHealth activities into categories based on the correlation between respondents’ reports of their usage. Subsequently, we used multivariate negative binomial generalized linear model regressions to determine whether negative healthcare experiences predicted greater use of these resulting categories. Finally, we stratified our sample based on education level to determine whether the associations between healthcare experiences and eHealth use differed across groups. Results: The study included 2612 individuals. Factor analysis classified the eHealth activities into two categories: provider-facing (eg, facilitating communication with providers) and independent (eg, patient-driven information seeking and communication with non-providers). Negative care experiences were not associated with provider-facing eHealth activity in the overall population (care coordination: P=.16; patient centeredness: P=.57) or among more highly educated respondents (care coordination: P=.73; patient centeredness: P=.32), but respondents with lower education levels who experienced problems with care coordination used provider-facing eHealth more often (IRR=1.40, P=.07). Individuals engaged in more independent eHealth activities if they experienced problems with either care coordination (IRR=1.15 P=.01) or patient-centered communication (IRR=1.16, P=.01). Although care coordination problems predicted independent eHealth activity across education levels (higher education: IRR=1.13 P=.01; lower education: IRR=1.19, P=.07), the relationship between low perceived patient centeredness and independent activity was limited to individuals with lower education levels (IRR=1.25, P=.02). Conclusions: Individuals use a greater number of eHealth activities, especially activities that are independent of healthcare providers, when they experience problems with their healthcare. People with lower levels of education seem particularly inclined to use eHealth when they have negative healthcare experiences. To maximize the potential for eHealth to meet the needs of all patients, especially those who are traditionally underserved by the healthcare system, additional work should be performed to ensure that eHealth resources are accessible and usable to all members of the population.

  • Source: Unsplash; Copyright: Claudia / @kaimantha; URL:; License: Licensed by JMIR.

    Accessibility and Applicability of Currently Available e-Mental Health Programs for Depression for People With Poststroke Aphasia: Scoping Review


    Background: Depression affects approximately 60% of people with aphasia 1 year post stroke and is associated with disability, lower quality of life, and mortality. Web-delivered mental health (e-mental health) programs are effective, convenient, and cost-effective for the general population and thus are increasingly used in the management of depression. However, it is unknown if such services are applicable and communicatively accessible to people with poststroke aphasia. Objective: The aim of this study was to identify freely available e-mental health programs for depression and determine their applicability and accessibility for people with poststroke aphasia. Methods: A Web-based search was conducted to identify and review freely available e-mental health programs for depression. These programs were then evaluated in terms of their (1) general features via a general evaluation tool, (2) communicative accessibility for people with aphasia via an aphasia-specific communicative accessibility evaluation tool, and (3) empirical evidence for the general population and stroke survivors with and without aphasia. The program that met the most general evaluation criteria and aphasia-specific communicative accessibility evaluation criteria was then trialed by a small subgroup of people with poststroke aphasia. Results: A total of 8 programs were identified. Of these, 4 had published evidence in support of their efficacy for use within the general population. However, no empirical evidence was identified that specifically supported any programs’ use for stroke survivors with or without aphasia. One evidence-based program scored at least 80% (16/19 and 16/20, respectively) on both the general and aphasia-specific communicative accessibility evaluation tools and was subject to a preliminary trial by 3 people with poststroke aphasia. During this trial, participants were either unable to independently use the program or gave it low usability scores on a post-trial satisfaction survey. On this basis, further evaluation was considered unwarranted. Conclusions: Despite fulfilling majority of the general evaluation and aphasia-specific evaluation criteria, the highest rated program was still found to be unsuitable for people with poststroke aphasia. Thus, e-mental health programs require substantial redevelopment if they are likely to be useful to people with poststroke aphasia.

  • An experimental setting where participants conduct virtual reality tasks in the passive auditory oddball paradigm. Source: Image created by the Authors; Copyright: The Authors; URL:; License: Creative Commons Attribution (CC-BY).

    Involuntary Attention Restoration During Exposure to Mobile-Based 360° Virtual Nature in Healthy Adults With Different Levels of Restorative Experience:...


    Background: With the global trend of urbanization, there are increasing reports of a possible association between decreased exposure to nature and increased occurrence of mental disorders. New 360° virtual reality (VR) technology using smartphones and portable VR glasses can overcome spatial and temporal limitations to help people deal with mental fatigue in everyday life. Objective: On the basis of attention restoration theory (ART), this study aimed to investigate whether the amplitude of the mismatch negativity (MMN)/P3a complex could act as an event-related potential (ERP) biomarker of involuntary attention restoration during exposure to 360° virtual nature in healthy young adults with different levels of restorative VR experience. Methods: A total of 40 healthy adults completed prequestionnaires on demographics and simulator sickness and postquestionnaires on simulator sickness and perceived restorativeness before and after exposure to virtual nature, respectively. During the VR exposure, brain activity was measured by electroencephalography as participants were asked to conduct a 2-tone passive auditory oddball task. Results: The amplitude and latency of the MMN/P3a complex were compared between individuals reporting a highly restorative experience and those reporting a less restorative experience. Although viewing a virtual nature environment, the high restorative group (N=19) exhibited significantly reduced P3a amplitudes compared with the low restorative group (N=20; t38=2.57; P=.02; d=0.59). Particularly, a moderate but significant negative correlation was found between the self-reported restorativeness scores and the P3a amplitudes at the fronto-central region (r=−.38; P=.02). However, the latency of the MMN/P3a complex did not significantly differ between the 2 groups (auditory mismatch negativity: t38=−1.47; P=.15 and P3a: t38=−0.31; P=.76). Conclusions: Considering individuals’ restorative experience, the amplitude of the fronto-central MMN/P3a complex can potentially be employed as a distinct ERP component of interest in involuntary attention restoration during virtual nature experience in healthy young adults. The findings for the 360° virtual nature experience seem to be consistent with those of previous ERP studies on the effects of meditation practice. This study extends the findings of previous ART and ERP studies of real-world meditation, restoration, and mental fatigue management into the virtual world created by mobile phone–based VR glasses and 360° video content.

  • Source: US Department of Defense (Andrew D Sarver); Copyright: US Air Force; URL:; License: Public Domain (CC0).

    Mining Open Payments Data: Analysis of Industry Payments to Thoracic Surgeons From 2014-2016


    Background: The financial relationship between physicians and industries has become a hotly debated issue globally. The Physician Payments Sunshine Act of the US Affordable Care Act (2010) promoted transparency of the transactions between industries and physicians by making remuneration data publicly accessible in the Open Payments Program database. Meanwhile, according to the World Health Organization, the majority of all noncommunicable disease deaths were caused by cardiovascular disease. Objective: This study aimed to investigate the distribution of non-research and non-ownership payments made to thoracic surgeons, to explore the regularity of financial relationships between industries and thoracic surgeons. Methods: Annual statistical data were obtained from the Open Payments Program general payment dataset from 2014-2016. We characterized the distribution of annual payments with single payment transactions greater than US $10,000, quantified the major expense categories (eg, Compensation, Consulting Fees, Travel and Lodging), and identified the 30 highest-paying industries. Moreover, we drew out the financial relations between industries to thoracic surgeons using chord diagram visualization. Results: The three highest categories with single payments greater than US $10,000 were Royalty or License, Compensation, and Consulting Fees. Payments related to Royalty or License transferred from only 5.38% of industries to 0.75% of surgeons with the highest median (US $13,753, $11,992, and $10,614 respectively) in 3-year period. In contrast, payments related to Food and Beverage transferred from 93.50% of industries to 98.48% of surgeons with the lowest median (US $28, $27, and $27). The top 30 highest-paying industries made up approximately 90% of the total payments (US $21,036,972, $23,304,996, and $28,116,336). Furthermore, just under 9% of surgeons received approximately 80% of the total payments in each of the 3 years. Specifically, the 100 highest cumulative payments, accounting for 52.69% of the total, transferred from 27 (6.05%) pharmaceutical industries to 86 (1.89%) thoracic surgeons from 2014-2016; 7 surgeons received payments greater than US $1,000,000; 12 surgeons received payments greater than US $400,000. The majority (90%) of these surgeons received tremendous value from only one industry. Conclusions: There exists a great discrepancy in the distribution of payments by categories. Royalty or License Fees, Compensation, and Consulting Fees are the primary transferring channels of single large payments. The massive transfer from industries to surgeons has a strong “apical dominance” and excludability. Further research should focus on discovering the fundamental driving factors for the strong concentration of certain medical devices and how these payments will affect the industry itself.

  • Source: PxHere; Copyright:; URL:; License: Public Domain (CC0).

    A Practical Do-It-Yourself Recruitment Framework for Concurrent eHealth Clinical Trials: Identification of Efficient and Cost-Effective Methods for Decision...


    Background: The ability to successfully recruit participants for electronic health (eHealth) clinical trials is largely dependent on the use of efficient and effective recruitment strategies. Determining which types of recruitment strategies to use presents a challenge for many researchers. Objective: The aim of this study was to present an analysis of the time-efficiency and cost-effectiveness of recruitment strategies for eHealth clinical trials, and it describes a framework for cost-effective trial recruitment. Methods: Participants were recruited for one of 5 eHealth trials of interventions for common mental health conditions. A multipronged recruitment approach was used, including digital (eg, social media and Craigslist), research registry-based, print (eg, flyers and posters on public transportation), clinic-based (eg, a general internal medicine clinic within an academic medical center and a large nonprofit health care organization), a market research recruitment firm, and traditional media strategies (eg, newspaper and television coverage in response to press releases). The time costs and fees for each recruitment method were calculated, and the participant yield on recruitment costs was calculated by dividing the number of enrolled participants by the total cost for each method. Results: A total of 777 participants were enrolled across all trials. Digital recruitment strategies yielded the largest number of participants across the 5 clinical trials and represented 34.0% (264/777) of the total enrolled participants. Registry-based recruitment strategies were in second place by enrolling 28.0% (217/777) of the total enrolled participants across trials. Research registry-based recruitment had a relatively high conversion rate from potential participants who contacted our center for being screened to be enrolled, and it was also the most cost-effective for enrolling participants in this set of clinical trials with a total cost per person enrolled at US $8.99. Conclusions: On the basis of these results, a framework is proposed for participant recruitment. To make decisions on initiating and maintaining different types of recruitment strategies, the resources available and requirements of the research study (or studies) need to be carefully examined.

Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Latest Submissions Open for Peer-Review:

View All Open Peer Review Articles
  • Association between Physical Activity (PA) Intervention Website Use and PA Levels among Spanish-speaking Latinas

    Date Submitted: Dec 9, 2018

    Open Peer Review Period: Dec 12, 2018 - Feb 6, 2019

    Background: The Internet’s low cost and potential for high reach makes web-based channels prime for delivering evidence-based physical activity (PA) interventions. Despite the well-studied success o...

    Background: The Internet’s low cost and potential for high reach makes web-based channels prime for delivering evidence-based physical activity (PA) interventions. Despite the well-studied success of Internet-based PA interventions in primarily non-Hispanic white populations, evidence about Spanish-speaking Latinas’ use of such interventions is lacking. The recent rise in technology use among Latinas in the US, a population at heightened risk for low PA levels and related conditions, suggests that they may benefit from web-based PA interventions tailored to their cultural and language preferences. Objective: These analyses examined engagement with the website and explored how use was associated with adoption and maintenance of PA behavior. Methods: Pasos Hacia la Salud tested a Spanish-language, culturally adapted, individually tailored, Internet-based PA intervention vs. a Spanish-lanugage Internet-based Wellness Contact Control condition for under-active Latinas (N=205, Mage=39.2 (SD=10.5), 84% Mexican-American). Results: Overall, participants logged on to the website an average of 22 times (SD=28) over 12 months, with Intervention participants logging on significantly more than Controls (29 vs. 14.7, p<.001). On average, participants spent more time on the website at months 1, 4, and 6 compared to all other months, with maximum use at month 4. Both logins and time spent on the website were significantly related to intervention success (achieving higher mean minutes of MVPA/week at follow-up: b=.48, SE=.20, p=.02 for objectively measured MVPA and b=.74, SE=.34, p=.03 for self-reported MVPA at 12 months, controlling for baseline). Furthermore, those meeting CDC guidelines for PA at 12 months (>=150 min/week of MVPA) logged on significantly more than those not meeting guidelines (35 vs 20 over 12 months, p=.002). Among participants in the intervention arm, goal setting features, personal PA reports, and PA tips were the most utilized portions of the website. Higher use of these features was associated with greater success in the program (more minutes of self-reported MVPA at 12 months controlling for baseline, p’s<.05). Specifically, one additional use of these features/month over 12 months translated into an additional 34 min/week of MVPA (goals feature), 12 min/week (PA tips), and 42 min/week (PA reports). Conclusions: These results demonstrate that greater use of a tailored, web-based PA intervention, particularly certain features on the site, was significantly related to increased PA levels in Latinas. Clinical Trial: identifier NCT01834287;

  • Recruiting to a randomised controlled trial of an online program for people with type 2 diabetes and depression: Lessons learned at the intersection of e-mental health and primary care

    Date Submitted: Dec 9, 2018

    Open Peer Review Period: Dec 12, 2018 - Feb 6, 2019

    Background: e-mental health (eMH) interventions are now widely available and they have the potential to revolutionise the way that health care is delivered. As most health care is currently delivered...

    Background: e-mental health (eMH) interventions are now widely available and they have the potential to revolutionise the way that health care is delivered. As most health care is currently delivered by primary care, there is enormous potential for eMH interventions to support, or in some cases substitute, for services currently delivered face to face in the community setting. However randomised trials of eMH interventions have tended to recruit participants using online recruitment methods. Consequently, it is difficult to know whether participants recruited online differ from those who attend primary care. Objective: This paper aims to document the experience of recruiting to an eMH trial through primary care and compare the characteristics of participants recruited through this and other recruitment methods. Methods: Recruitment to the SpringboarD randomised controlled trial was initially focused on general practices in two states of Australia. Over 15 months we employed a comprehensive approach to engaging practice staff and supporting them to recruit patients, including face to face site visits, regular contact via telephone and trial newsletters, and development of an online patient registration portal. Nevertheless, it became apparent that these efforts would not yield the required sample size and we therefore supplemented recruitment through national online advertising and promoting the study through existing networks. Baseline characteristics of participants recruited to the trial through general practice, online, or other sources were compared using ANOVA and chi square. Results: Between November 2015 and October 2017, 780 people enrolled in SpringboarD, of whom 740 provided information on recruitment source. Of these, just 24 were recruited through general practice, while 520 were recruited online and 196 through existing networks. Key barriers to general practice recruitment included perceived mismatch between trial design and diabetes population, prioritisation of acute health issues, and disruptions posed by events at the practice and community level. Participants recruited through the three different approaches differed on age, gender, employment status, depressive symptoms and diabetes distress, with participants online distinguished from those recruited through general practice or other sources. However most differences reached only a small effect size and are unlikely to be of clinical importance. Conclusions: Time, labour, and cost intensive efforts did not translate into successful recruitment through general practice in this instance, with barriers identified at several different levels. Online recruitment yielded more participants, who were similar to those recruited via general practice. Clinical Trial: ACTRN12615000931572

  • A Comparison of Smartphone Ownership, Social Media Use and Willingness to use Digital Interventions for Substance Use among Generation Z and Millennials in Substance Use Treatment

    Date Submitted: Dec 11, 2018

    Open Peer Review Period: Dec 12, 2018 - Dec 21, 2018

    Background: Problematic substance use in adolescence and emerging adulthood is a significant public health concern in the United States due to high recurrence of use rates and unmet treatment needs co...

    Background: Problematic substance use in adolescence and emerging adulthood is a significant public health concern in the United States due to high recurrence of use rates and unmet treatment needs coupled with increased use. Consequently, there is a need for both improved service utilization and availability of recovery supports. Given the ubiquitous use of the Internet and social media via smartphones, a viable option is to design digital treatments and recovery support services to include Internet and social media platforms. Objective: While digital treatments delivered through social media and the Internet are a possibility, it is unclear how to tailor interventions using these tools for groups with problematic substance use. There is limited research comparing consumer trends of use of social media platforms, use of platform features, and vulnerability of exposure to drug cues online. The goal of this study was to compare digital platforms used among adolescents (Generation Zs) and emerging adults (Millennials) attending outpatient substance use treatment and to examine receptiveness to using these platforms to support substance use treatment and recovery. Methods: Generation Zs and Millennials enrolled in outpatient substance use treatment (n = 164) completed a survey examining social media use, digital intervention acceptability, frequency of substance exposure, and substance use experiences. Generation Zs (n = 77) completed the survey in July 2018. Millennials (n = 114) completed the survey in May 2016. Results: Generation Zs had an average age of 15.66 years (SD = 1.18) and primarily identified as male (50.9%). Millennials had an average age of 27.66 years (SD = 5.12) and also primarily identified as male (74.8%). Most participants owned a social media account (M: 82.0%, Z: 94.3%), and used it daily (M: 67.6%, Z: 79.2%); however, Generation Zs were more likely to use Instagram and Snapchat while Millennials were more likely to use Facebook. Further, Generation Zs were more likely to use the features within social media platforms (e.g., instant messaging M: 55.0%, Z: 79.2%, and watching videos (M: 56.8%, Z: 81.1%)). Many participants observed drug cues on social media (M: 76.4%, Z: 79.2%). However, fewer observed recovery information on social media (M: 30.6%, Z: 34.0%). Participants felt that social media (M:55.0%, Z: 48.1%), a cell phone application (M: 36.9%, Z: 45.3%), texting (M: 28.8%, Z: 45.3%), or a website (M:39.6%, Z: 32.1%) would be useful in delivering recovery support. Conclusions: Given the high rates of exposure to drug cues on social media, disseminating recovery supports within a social media platform may be the ideal just-in-time intervention necessary to decrease rates of recurrent use. However, our results suggest that cross-platform solutions capable of transcending generational preferences is necessary and one-size-fits-all digital interventions should be avoided.

  • Automated Analysis of Domestic Violence Police Reports to Explore Abuse Types and Victim Injuries

    Date Submitted: Dec 11, 2018

    Open Peer Review Period: Dec 12, 2018 - Dec 21, 2018

    Background: Police attend numerous domestic violence (DV) events each year, recording details of these events as both structured (coded) data and unstructured free text narratives. Abuse types (includ...

    Background: Police attend numerous domestic violence (DV) events each year, recording details of these events as both structured (coded) data and unstructured free text narratives. Abuse types (including physical, psychological, emotional and financial) along with any injuries sustained by victims are typically recorded in long descriptive narratives. Objective: In this paper we investigate if an automated text mining method can identify abuse types and any injuries sustained by DV victims in narratives contained in a large police data set from the New South Wales Police Force. Methods: We used a training set of 200 DV recorded events to design a knowledge-driven approach based on syntactical patterns in the text and then applied this to a large set of police reports. Results: Testing our approach on an evaluation set of 100 DV events returned a 90.2% and 85.0% precision for abuse type and victim injuries respectively. In a set of 492,393 DV reports, we found 71.32% (351,178) of events with mentions of the abuse type(s) and more than one third (35.97%; 177,117) contained victim injuries. ‘Emotional/verbal abuse’ (33.46%; 117,488) was the most common abuse type, followed by ‘punching’ (86,322; 24.58%) and ‘property damage’ (22.27%; 78,203). ‘Bruising’ was the most common form of injury sustained (29.03%; 51,455 events) with ‘cut/abrasion’ (28.93%; 51,284 events) and ‘red marks/signs’ (23.71%; 42,038 events) ranking second and third respectively. Conclusions: The results suggest that text mining can automatically extract information from police-recorded DV events that can support further public health research into domestic violence, such as examining the relationship between abuse types and victim injuries, between gender and abuse types and risk escalation for victims of DV. Potential also exists from this extracted information to be linked to information on mental health status.

  • Measuring regional quality of care using unsolicited online data: creating more detailed insight using text analyses

    Date Submitted: Dec 7, 2018

    Open Peer Review Period: Dec 11, 2018 - Feb 5, 2019

    Background: Regional population health management (PM) initiatives require insight into experienced quality of care at the regional level. Unsolicited online provider ratings have shown potential for...

    Background: Regional population health management (PM) initiatives require insight into experienced quality of care at the regional level. Unsolicited online provider ratings have shown potential for this use. This study explored the addition of comments accompanying unsolicited online ratings to regional analyses. Objective: The goal was to create additional insight for each PM initiatives as well as overall comparisons between these initiatives by attempting to determine the reasoning and rationale behind a rating. Methods: The Dutch Zorgkaart database provided the unsolicited ratings (period: 2008 – 2017) for analyses. All ratings included both quantitative ratings as well as qualitative text comments. Nine PM regions were used to aggregate ratings geographically. Sentiment analyses were performed by categorizing ratings into negative, neutral and positive ratings. Per category, as well as per PM initiative, word frequencies (unigrams and bigrams) were explored. Machine learning (naive Bayes) was applied to identify the most important predictors for rating overall sentiment and for identifying PM initiatives. Results: 449,263 unsolicited ratings were available in the Zorgkaart database, 303,930 positive ratings, 97,739 neutral and 47,592 negative ratings. Bigrams illustrated that feeling like not being “taken serious” was the dominant bigram in negative ratings, while bigrams in positive ratings were mostly related to listening, explaining and perceived knowledge. Comparing bigrams between PM initiatives showed a lot of overlap, but several differences were identified. The naive Bayes machine learning was able to predict sentiments of comments, but unable to distinguish between specific PM initiatives. Conclusions: Adding information from text comments that accompany online ratings to regional evaluations provides insight for PM initiatives into the underlying reasoning. They provide useful overarching information for healthcare policy makers, but due to a lot of overlap it adds little region specific information. Specific outliers for some initiatives are insightful. Clinical Trial: The Medical Research Involving Human Subjects Act (WMO) does not apply to this study, and official approval was not required [1]. Participants agreed to the terms of service of Zorgkaart Nederland, which states that their submission can be used anonymously for research purposes [2].

  • Health professions’ digital education: a review of learning theories in RCTs by the Digital Health Education collaboration

    Date Submitted: Dec 11, 2018

    Open Peer Review Period: Dec 11, 2018 - Dec 21, 2018

    Background: Learning theory is an essential component for designing an effective educational curriculum. Reviews of existing literature consistently lack sufficient evidence to support the effectivene...

    Background: Learning theory is an essential component for designing an effective educational curriculum. Reviews of existing literature consistently lack sufficient evidence to support the effectiveness of digital interventions for health professions’ education, which may reflect disconnections between learning theories, curriculum design, use of technology, and outcome evaluation. Objective: We aim to identify, map and evaluate the use of learning-theories in designing and implementing intervention trials of health professions’ digital education, as well as to highlight areas for future research on technology enhanced education via the establishment of a development framework for practice and research. Methods: Systematic search of MEDLINE, Embase, Cochrane Central Register of Controlled Trials (Cochrane Library), PsycINFO, CINAHL, ERIC and Web of Science for randomized controlled trials (RCTs) published between 2007 and 2016 (ten years). Results: A total 874 RCTs on digital health education were identified and categorized into online-offline, mobile digital education, and simulation-based modalities for pre and post-registration health professions’ education. Of these, 242 studies were randomly selected for methodological review and thematic analysis. Data were extracted by one author using a standardized form, with a 20% random sample extracted by a second author in duplicate. One-third (33.4%; 81/242) studies reported single or multiple learning theories in design, assessment, conceptualization or interpretation of outcomes of the digital education interventions. Commonly reported learning theories were problem-based learning (19.7%), social learning theory (13.5%), and cognitive theory of multimedia learning (12.3%). Most of these studies assessed knowledge (48.8%), skills (25.6%), and performance (24.3%) as primary outcomes with non-validated assessment tools (62.4%). Studies with reported learning theories (p=0.002) and validated instruments (p=0.006) have shown effective acquisition of learning outcomes. Conclusions: We proposed a development framework to safeguard robustness and integrity of the design and implementation of future digital education programmes for the training of health professions.