This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
Remote measurement technology refers to the use of mobile health technology to track and measure change in health status in real time as part of a person’s everyday life. With accurate measurement, remote measurement technology offers the opportunity to augment health care by providing personalized, precise, and preemptive interventions that support insight into patterns of health-related behavior and self-management. However, for successful implementation, users need to be engaged in its use.
Our objective was to systematically review the literature to update and extend the understanding of the key barriers to and facilitators of engagement with and use of remote measurement technology, to guide the development of future remote measurement technology resources.
We conducted a systematic review using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines involving original studies dating back to the last systematic review published in 2014. We included studies if they met the following entry criteria: population (people using remote measurement technology approaches to aid management of health), intervention (remote measurement technology system), comparison group (no comparison group specified), outcomes (qualitative or quantitative evaluation of the barriers to and facilitators of engagement with this system), and study design (randomized controlled trials, feasibility studies, and observational studies). We searched 5 databases (MEDLINE, IEEE Xplore, EMBASE, Web of Science, and the Cochrane Library) for articles published from January 2014 to May 2017. Articles were independently screened by 2 researchers. We extracted study characteristics and conducted a content analysis to define emerging themes to synthesize findings. Formal quality assessments were performed to address risk of bias.
A total of 33 studies met inclusion criteria, employing quantitative, qualitative, or mixed-methods designs. Studies were conducted in 10 countries, included male and female participants, with ages ranging from 8 to 95 years, and included both active and passive remote monitoring systems for a diverse range of physical and mental health conditions. However, they were relatively short and had small sample sizes, and reporting of usage statistics was inconsistent. Acceptability of remote measurement technology according to the average percentage of time used (64%-86.5%) and dropout rates (0%-44%) was variable. The barriers and facilitators from the content analysis related to health status, perceived utility and value, motivation, convenience and accessibility, and usability.
The results of this review highlight gaps in the design of studies trialing remote measurement technology, including the use of quantitative assessment of usage and acceptability. Several processes that could facilitate engagement with this technology have been identified and may drive the development of more person-focused remote measurement technology. However, these factors need further testing through carefully designed experimental studies.
International Prospective Register of Systematic Reviews (PROSPERO) CRD42017060644; https://www.crd.york.ac.uk/PROSPERO/display_record.php?RecordID=60644 (Archived by WebCite at http://www.webcitation.org/70K4mThTr)
Global smartphone ownership has increased, which provides ready access to the internet, and a means of actively logging information and passively gathering big data [
Engagement is defined as the extent to and manner in which people actively use a resource and has been operationalized as a multistage process involving the point of engagement, a period of sustained engagement, disengagement, and reengagement [
The purpose of this systematic review was to update and extend the understanding of the barriers to and facilitators of engagement with RMT systems for target users. We defined RMT following Davis et al [
Following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, we conducted a systematic review of studies to answer the question “What are the barriers to and facilitators of engagement with remote measurement technology?” We registered the trial with the International Prospective Register of Systematic Reviews (PROSPERO registration number CRD42017060644).
We included studies if they met the following criteria: (1) were published in English; (2) included health care RMT, defined as any mobile technology that enables monitoring of a person’s health status through a remote interface, with the data then either transmitted to a health care provider for review or to be used as a means of education for the user themselves [
We searched Ovid MEDLINE, IEEE Xplore, EMBASE, Web of Science, and the Cochrane Library using the combined terms “remote” or “mobile” and “technology” or “devices,” along with “telemedicine” and “mHealth.”
We extracted the following data: (1) device type and RMT system (including active and passive data); (2) population characteristics, including diagnostic categories, sample size, time using RMT, and the country in which the study was conducted; and (3) methods used to gather qualitative information on the feasibility and acceptability, grouped as follows: usage statistics, questionnaires, structured or semistructured interviews, focus groups, and descriptive feedback.
One author (SS) read and reread the results reported in articles published from January 2014 to July 2016 to extract individual barriers and facilitators (defined as “a circumstance or obstacle that may prevent the adoption of remote measurement technology” or “make adoption easy or easier”). The coding frame was developed by 3 authors (SS, BG, and HC) using these data. It consisted of the following themes: health status, usability, convenience and accessibility, perceived utility, and motivation, with subthemes. This coding frame was then tested on a further batch of articles published from June 2016 to May 2017 (coded by authors BG and HC and discrepancies evaluated by SS). This replication test allowed for a validation and potential extension of the initial coding frame.
Methodological quality was assessed by 2 independent raters using the Mixed Methods Appraisal Tool (MMAT) [
Of the 3187 abstracts and titles identified, 33 original articles met our inclusion criteria (see the PRISMA flow diagram in
Studies varied in their sample size (7-365 participants), as well as the age (8-95 years) and sex of participants (30 studies included both male and female participants).
Studies were conducted in 10 countries: the United States (n=24), United Kingdom (n=1), Canada (n=1), Taiwan (n=1), Sweden (n=1), Poland (n=1), Australia (n=1), Switzerland (n=1), Germany (n=1), and New Zealand (n=1). Study durations ranged from 1 to 13 months, and 3 studies consisted of only a single individual or group session.
A total of 6 studies used passive RMT, including wearable pedometers and accelerometers, and built-in smartphone activity monitors (see
RMT systems provided feedback to users (n=17), members of the users’ health care team (n=7), or both (n=9). Feedback was provided in various forms, including visual displays (eg, graphs), report summaries, historic reporting patterns, and messages (eg, health advice and motivational feedback).
The studies covered many health conditions, with most concentrating on 1 condition (n=17). A total of 2 studies featured more than 1 physical health diagnosis (diabetes and obesity, and multiple genetic blood disorders). Only 4 studies related to mental health conditions such as psychosis and posttraumatic stress disorder, and 2 studies included both physical and mental health conditions (eg, depression and type 2 diabetes, HIV, and substance use disorders). The remaining studies supported general health and well-being (n=7), and smoking cessation (n=1).
In total, 27 studies employed quantitative methods to identify barriers to and facilitators of using RMT systems, including usage statistics (n=20) and questionnaires (n=19). Most questionnaires (15/19, 79%) were unvalidated measures developed for the study. Only 4 studies used validated measures, including the System Usability Scale, the Telehealth Usability Questionnaire, and the Technology Acceptance Model Questionnaire. Similarly, types of usage statistics reported varied greatly between studies. Of these 27 studies, 9 employed a mixed-methods design and asked for qualitative information (ie, from semistructured interviews and focus groups) and quantitative information from their users; 6 studies employed purely qualitative methods.
Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flow diagram of study selection. RMT: remote measurement technology.
Of the reviewed studies, 2 obtained the maximum score of 1 on the MMAT [
Of the 5 studies that reported on the average number of times the RMT system was used, 3 reported the total number of interactions and 2 reported the number of days that people interacted with the app; 2 reported on the percentage of people who wore the wearable device for the whole study; and 4 set a threshold for the appropriate level of adherence (which varied between studies) and reported the percentage of people meeting these requirements. The remaining studies reported idiosyncratic usage statistics that were not comparable across studies. This variability severely limited quantified conclusions. For the few studies that reported the average percentage of time used, this ranged from 64% to 86.5% [
Reasons for dropout across studies.
Reason for dropout | Frequency | Related theme |
Lost or stolen smartphone | 23 | Usability |
Technical malfunction (eg, smartphone corrupted, not receiving texts, or delivery delays) | 7 | Usability |
Exacerbation of health condition, including participants who were injured or died during the course of the study | 6 | Health status |
Deleted app | 3 | Usability |
App not compatible with existing smartphone | 3 | Convenience and accessibility |
Unexpected usage patterns (eg, switched smartphone off in between answering surveys, left smartphone plugged into charger, used smartphone in airplane mode) | 3 | Perceived utility |
Moved out of area or was discharged from hospital | 3 | Convenience and accessibility |
Sold smartphone | 2 | Perceived utility |
Changed mobile phone or service plan | 2 | Convenience and accessibility |
Practical technical difficulties (eg, not being able to download the app) | 2 | Usability |
Broken smartphone | 1 | Usability |
Inconsistent wireless network | 1 | Convenience and accessibility |
App consumed too much battery | 1 | Usability |
System too slow | 1 | Usability |
Unspecified reason | 11 | Not applicable |
We divided themes into 5 major categories that made up a coding frame for structuring the minor themes. The two batches of articles (2014-2015 and 2016-2017) yielded subthemes that fitted within the same coding frame, with all major themes represented across the two time periods providing evidence of validity. No new themes arose in the later studies. The following section describes the findings for each major theme, with barriers and facilitators in italics.
Ben-Zeev et al [
Where technical malfunctions and complexities in terms of usability arose, practical support was sometimes necessary. Some studies reported that problems such as “creating user accounts, answering intake question and navigating content due to unexpected behavior of keyboards, scroll bars, buttons, and other interface widgets” could be addressed with minor adjustments [
In addition to technical functionality and clarity of information, we grouped other subthemes under the broader theme of usability.
Where users were required to actively engage with data collection (active RMT), the
Other major barriers were related to participants’ access to resources such as websites and videos due to a poor internet connection or lack of a Wi-Fi connection, and use of old computer systems [
Other barriers within this theme included RMT systems not being adequately
The results of 4 studies demonstrated a positive and motivating effect of feedback [
Further
The value of the RMT system appeared to be affected by people’s
Dropout is a clear indicator of problems with engagement. Reasons for dropout spanned several of the qualitative themes, with problems related to usability of the wearable device and the smartphones apps being the most frequent. Convenience and accessibility was the second most frequent theme. The study that reported the greatest percentage of dropouts included one of the largest samples (n=342) and followed people with a diagnosis of psychosis for 6 months. Studies that reported no dropouts or the odd person dropping out were much smaller (ranging from 8 to 51 participants), and dropout may not be possible to understand here, as the sample might have been highly selective. There was no significant relationship between the percentage of people who dropped out and the length of the intervention in days (
A total of 10 studies reported on the impact of variables on adherence in terms of compliance and use of an mHealth device over time. The themes included health status, with greater physical disability [
Many of the factors discovered are consistent with the engagement attributes previously reported by O’Brien and Toms [
Engagement in our model is moderated by health status, usability, convenience and accessibility, perceived utility, and motivation to engage. Engagement may be at its strongest when the user is able to use the technology, perceives the technology to be useful, and wants to use the technology.
Of particular importance to RMT systems for management of health outcomes is the health status of the user. Health status will inevitably have an impact on what constitutes a usable, convenient, accessible, or valuable feature of an RMT system. As an example, being unwell and outside of one’s usual environment or routine (eg, in the hospital) led to disruptions in engagement and dropout [
At the heart of this proposed model is usability. There may be individual differences that moderate usability, including variables such as age, past experience with technology, and exacerbations in health conditions and disability status, as well as the influence of how the system is designed. Problems with usability were the most common reasons for dropout from the studies. There is evidence that older adults were harder to engage [
Model of barriers to and facilitators of engagement with remote measurement technology.
However, the specific parameters for this support are unknown and need further research with clearly quantifiable outcomes. In addition, involvement of user experience methods is important for the development of usable mHealth tools for RMT systems in the future, with coproduction and user-centered design processes to validate choices [
The need to be able to integrate the RMT system into a user’s normal routine was clear. Participants preferred tools that fit in with daily routines and tools that have already been adopted, with some disengaging and dropping out if unacceptable alternatives were offered. Personalization and demonstrating flexibility, in terms of taking into account the specific disabilities and needs of clinical groups, may be key in the design of usable RMT systems. This may include individual goal setting of dates and times for study activities, opting in or out of certain tracking activities (eg, reducing intrusiveness), or accommodating for health-related differing abilities. It may be important to note that forgetfulness emerged as a key barrier to engagement, which may suggest that the cognitive burden placed on individuals to remember to complete RMT schedules, in these studies, was too great. The value of notifications and reminders to carry out tasks has been demonstrated through usage statistics. That said, the magnitude of the effect varied between studies, with 1 study demonstrating a much bigger impact of notifications. This suggests that other factors moderate the likelihood of self-initiated engagement. Prompts have been mentioned to help aid memory, but there was some suggestion that the timing of these strategies may be important [
We propose that increasing the rewards of using RMT increases the overall perceived value of the system in the face of some potential costs. Costs included financial costs of purchasing equipment, as well as concerns about privacy and reliability or accuracy of the data collected. As a strategy for increasing rewards associated with RMT systems, feedback is generally accepted, tolerated, and, in some cases, actively sought by users of RMT systems. In this context, feedback is considered to be additional information that participants receive from an RMT system about their health, their participation, or the larger program from which users and participants can derive value. This could include health information, rates of participation or adherence, metrics defined in goal-setting exercises, positive reinforcements, or general information about the study or their health condition. It was commonly reported that participants would like to receive more feedback [
Motivation was a smaller but important category emerging from the analysis of the results of previous studies using RMT systems for the management of health outcomes. Without motivation, participants may not engage with the initial process of learning how to use a new system, and this category is inextricably linked to all other factors discussed previously. Even if users are familiar with mHealth tools such as smartphones and wearable devices, they may need additional motivation to integrate a new set of behaviors, such as responding to surveys. Lack of motivation is therefore a fundamental barrier to engagement. The factors presented thus far should be considered not just at the initiation of the study, but also as engagement is managed over time, because perceptions of the technology’s value or usability may change with prolonged use (eg, if expectations are not met). Therefore, we recommend steps to increase, or mitigate decreased, motivation with an RMT system to maintain motivation, and therefore engagement, over time.
Facilitators identified include convenience and accessibility, perceived utility, and motivation, but these factors are drawn from of pool of studies that varied greatly in terms of their quality. In addition, we conceptualized engagement as a process that should include disengagement and reengagement when required, but most findings reported in the studies included in this review relate to moderators of initial and sustained engagement. Although in our model we tentatively propose a feedback loop between the point of disengagement and the same barriers and facilitators affecting initial and sustained engagement, it is possible that factors affecting reengagement may be different, and this was not the focus of the studies. Future research should focus on the entire engagement process and quantify the impact of specific variables on engagement in terms of observable changes in usage statistics in rigorous experimental design. Some examples might be looking at the impact of different types of support (automated messages vs personalized messages vs direct human support) on the number of interactions and overall time spent using a smartphone app or wearable device. The impact of different types of feedback (immediate vs delayed vs no feedback) and data visualization or communication methods (graphs vs text messages vs discussion with a study coordinator) or environment (hospital vs home-based use) also need to be explicitly tested. Careful experimental manipulation is missing from the literature to date and, to be able to compare across these conditions, quantitative measures and usage statistics also require more standardization. A similar conclusion has also been drawn when considering adherence [
It is not enough for software developers to consider their systems in isolation from the individuals who may be using them. One of the main ways to develop engaged systems is to begin with codesign with those individuals who will be using the system. This is especially important for those involved in providing RMT for improving health. Before RMT systems are tested, there needs to be an iterative design process that explores acceptability, such as following the principles of user-centered design [
The themes discovered in this review emerged across two different time periods providing validity information, but this evidence suggests that we are continuing to make the same mistakes. There is a great potential for RMT systems to augment and extend health care, but there remain clear challenges that still need to be overcome. Two suggestions are, first, to improve how we measure the impact of modifiable variables on engagement in order to understand the magnitude of effects. Second, several studies suggest working with the target users directly to coproduce systems that are acceptable and feasible to use over long periods of time. Our model indicates the interrelationship between key facilitators on the one hand, and the person and RMT factors on the other, that could act as a prototype for the development of RMT in the future.
Search strategy
Facilitators of and barriers to engagement in active RMT.
Facilitators of and barriers to engagement in passive and combination RMT.
Systematic review quotes
Characteristics of the original studies included in the systematic review.
Mixed Methods Appraisal Tool
Preferred Reporting Items for Systematic Reviews and Meta-Analyses
remote measurement technology
This paper was written as part of the development of useful mHealth and remote measurement technology systems in the Remote Assessment of Disease and Relapse – Central Nervous System (RADAR-CNS) project. We acknowledge all partners in the RADAR-CNS consortium (www.radar-cns.org) for overall discussion of the results. The RADAR-CNS project has received funding from the Innovative Medicines Initiative 2 Joint Undertaking under grant agreement no. 115902. This Joint Undertaking receives support from the European Union’s Horizon 2020 research and innovation program and European Federation of Pharmaceutical Industries and Associations (EFPIA; www.imi.europa.eu). This communication reflects the views of the RADAR-CNS consortium and neither the Innovative Medicines Initiative nor the European Union and EFPIA are liable for any use that may be made of the information contained herein. This paper also presents independent research funded in part by the UK National Institute for Health Research (NIHR) Biomedical Research Centre at South London and Maudsley National Health Service (NHS) Foundation Trust and King’s College London. The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR, or the UK Department of Health and Social Care. TW would also like to acknowledge support from the NIHR Biomedical Research Centre at the South London and Maudsley Foundation Trust and King’s College London, as well as the NIHR Senior Investigator Awards.
None declared.