The Karma system is currently undergoing maintenance (Monday, January 29, 2018).
The maintenance period has been extended to 8PM EST.

Karma Credits will not be available for redeeming during maintenance.
Advertisement

Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 17.04.18 in Vol 20, No 4 (2018): April

Preprints (earlier versions) of this paper are available at http://preprints.jmir.org/preprint/9897, first published Jan 20, 2018.

This paper is in the following e-collection/theme issue:

    Original Paper

    Real-World Implementation of Video Outpatient Consultations at Macro, Meso, and Micro Levels: Mixed-Method Study

    1Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, United Kingdom

    2Barts Health NHS Trust, London, United Kingdom

    3Tower Hamlets Clinical Commissioning Group, London, United Kingdom

    *these authors contributed equally

    Corresponding Author:

    Trisha Greenhalgh, FMedSci

    Nuffield Department of Primary Care Health Sciences

    University of Oxford

    Radcliffe Observatory Quarter

    Oxford, OX2 6GG

    United Kingdom

    Phone: 44 1865289293

    Email:


    ABSTRACT

    Background: There is much interest in virtual consultations using video technology. Randomized controlled trials have shown video consultations to be acceptable, safe, and effective in selected conditions and circumstances. However, this model has rarely been mainstreamed and sustained in real-world settings.

    Objective: The study sought to (1) define good practice and inform implementation of video outpatient consultations and (2) generate transferable knowledge about challenges to scaling up and routinizing this service model.

    Methods: A multilevel, mixed-method study of Skype video consultations (micro level) was embedded in an organizational case study (meso level), taking account of national context and wider influences (macro level). The study followed the introduction of video outpatient consultations in three clinical services (diabetes, diabetes antenatal, and cancer surgery) in a National Health Service trust (covering three hospitals) in London, United Kingdom. Data sources included 36 national-level stakeholders (exploratory and semistructured interviews), longitudinal organizational ethnography (300 hours of observations; 24 staff interviews), 30 videotaped remote consultations, 17 audiotaped face-to-face consultations, and national and local documents. Qualitative data, analyzed using sociotechnical change theories, addressed staff and patient experience and organizational and system drivers. Quantitative data, analyzed via descriptive statistics, included uptake of video consultations by staff and patients and microcategorization of different kinds of talk (using the Roter interaction analysis system).

    Results: When clinical, technical, and practical preconditions were met, video consultations appeared safe and were popular with some patients and staff. Compared with face-to-face consultations for similar conditions, video consultations were very slightly shorter, patients did slightly more talking, and both parties sometimes needed to make explicit things that typically remained implicit in a traditional encounter. Video consultations appeared to work better when the clinician and patient already knew and trusted each other. Some clinicians used Skype adaptively to respond to patient requests for ad hoc encounters in a way that appeared to strengthen supported self-management. The reality of establishing video outpatient services in a busy and financially stretched acute hospital setting proved more complex and time-consuming than originally anticipated. By the end of this study, between 2% and 22% of consultations were being undertaken remotely by participating clinicians. In the remainder, clinicians chose not to participate, or video consultations were considered impractical, technically unachievable, or clinically inadvisable. Technical challenges were typically minor but potentially prohibitive.

    Conclusions: Video outpatient consultations appear safe, effective, and convenient for patients in situations where participating clinicians judge them clinically appropriate, but such situations are a fraction of the overall clinic workload. As with other technological innovations, some clinicians will adopt readily, whereas others will need incentives and support. There are complex challenges to embedding video consultation services within routine practice in organizations that are hesitant to change, especially in times of austerity.

    J Med Internet Res 2018;20(4):e150

    doi:10.2196/jmir.9897

    KEYWORDS



    Introduction

    Background

    Outpatient services, particularly for people with long-term conditions, have changed little in recent decades. Yet, in many countries, population demographics, disease epidemiology, and care priorities have changed a great deal, with, for example, an aging population, rising rates of chronic illness and multimorbidity, and an increasing emphasis on multidisciplinary team care and supported self-management. Outpatient nonattendance rates in some patient groups, especially the disadvantaged and those with multiple and complex needs, are high and may be associated with poor disease control, increased use of emergency services, and high costs [1,2]. Patient-borne costs (in terms of time and travel) of attending outpatient appointments are high, especially for tertiary care [3-5].

    There is a strong policy push in the United Kingdom [6-9] and elsewhere [10-12] to harness the potential of digital technologies to improve care models and redesign care pathways in a way that improves the accessibility and efficiency of services and maximizes the potential for patient self-management. The UK’s National Information Board recently argued that a different kind of health service is needed, in which the traditional outpatient consultation will become increasingly obsolete [8]. Remote video consulting using Skype or FaceTime is one potential solution.

    Published research on video outpatient consultations has been summarized in several recent narrative reviews [13-15]. Randomized controlled trials (RCTs) have shown such consultations to be acceptable, safe, and effective (and, when measured, to reduce patient-borne costs) in patients deemed clinically eligible in a range of conditions, including adult and teenage diabetes [16-18], chronic kidney disease [19], chronic obstructive pulmonary disease [20,21], mental health conditions [22,23], chronic pain [24-26], support after premature birth [27], support of patients in care homes [28], postoperative follow-up for orthopedics [29-32], plastic surgery [33], and prostate cancer [34].

    However, all these trials were small; publication bias cannot be excluded, uptake rates were not always reported, and the cost of establishing and maintaining the remote service was rarely measured. Although no specific safety issues or critical events were reported in any of the above studies, exclusion or withdrawal rates in some studies were high [18,23,25].

    Notwithstanding the positive findings of randomized trials, audits of actual practice suggest that video outpatient consultations, in common with other forms of telehealth, account for only a tiny fraction of encounters in any specialty [35,36]. Nonadoption, abandonment, and failure of scale-up, spread, and sustainability (the NASSS framework) are the norm when technology-supported service models are introduced in real-world settings for multiple and complex reasons [37-39]. In the words of one critic, the benefits of Skype- and FaceTime-supported outpatient services demonstrated in proof-of-concept studies and experimental trials should be weighed against contextual realities, including “the vagaries of technology, negative views among clinicians, poor uptake by providers, and legal, ethical and administrative barriers” [40].

    Some authors have queried whether video consultations might be less reliable [41-43], less safe [44], and less cost-effective than traditional encounters [40]. The online environment is known to produce subtle alterations in the dynamics of human interaction, with a potential risk that clinical clues will be missed or the clinician-patient dynamic altered adversely [45]. The introduction of video outpatient consultations also brings operational and cultural challenges, including the need to develop new ways of organizing clinical and administrative work and train and support both staff and patients in technology use [37,46].

    In short, there is a growing mismatch between the positive evidence base emerging from experimental trials and the variable (mostly negative) experiences of teams who try to introduce video outpatient services in the real world [35,36]. The discussion sections of randomized trial reports often suggest that further research is needed into implementation challenges—especially national policy and economic context, the practicalities and costs of organizational change, and the fine-grained detail of how video consultations unfold (in particular, how “quality” and “safety” are achieved and assured) [17,25]. No previous studies have addressed all these issues and their interdependencies rigorously and prospectively. Yet, such research is critical to enrich our understanding of video consulting and to inform and support the development and scaling up of such services. For this reason, we undertook an in-depth, real-world case study of an attempt to introduce video outpatient consultations across several clinical services (in three different hospitals) in a London-based acute trust.

    Aim, Objectives, and Research Questions

    Our aims were (1) to define good practice and inform its implementation in relation to video outpatient consultations via Skype and similar media and (2) to generate transferable new knowledge about challenges to scaling up and routinizing this service model in health care organizations.

    Specifically, our objectives were as follows:

    1. At macro level, to build relationships with key national stakeholders, identify from their perspective how to overcome policy and legal barriers, and create a receptive context for our findings.
    2. At meso level, to illuminate and explore the sociotechnical system that supports the video consultation at organizational level and identify how organizations can best support the introduction and sustainability of this service model where appropriate.
    3. At micro level, to study the clinician-patient interaction in a maximum variety sample of video consultations and a comparator sample of face-to-face consultations, exploring both effective and less effective communication.

    Our research questions were as follows:

    1. Macro level: what is the national-level context for the introduction of video outpatient consultations in the United Kingdom, and what measures might incentivize and make such consultations easier to achieve?
    2. Meso level: how can a video consultation service best be established, routinized, and sustained?
    3. Micro level: what defines “quality” in a video consultation, and what are the barriers to achieving this?

    Methods

    Outline

    We have published a detailed study protocol previously [14]; this section provides a summary and refinement of our sampling and analytic approach.

    Study Design

    The multilevel, mixed-method study of video outpatient consultations in three hospital departments (diabetes, antenatal diabetes, and cancer surgery: micro level), embedded in an organizational case study of the introduction and rollout of this new service model (meso level), taking account of the evolving national context (macro level), was used.

    Setting

    The research took place from 2015 to 2017 in Barts Health, a large, multisite acute trust in a socioeconomically deprived and multiethnic borough in inner-city London, United Kingdom. The organization was under pressure to deliver services more cost-effectively while responding to rising need and demand; outpatient clinics were crowded, and travel to and between its multiple sites was difficult, time-consuming, and (for patients on low incomes) costly.

    Clinicians in the diabetes service had been working for several years to establish a remote video option for outpatient consultations to improve accessibility and address high “did not attend” rates. In an early pilot study, we found high acceptance rates by staff and patients for the video option, a significant reduction in “did not attend” rates and small efficiency savings [47,48]. We subsequently demonstrated greater engagement, improved self-management, and better glycemic control in patients with challenging social circumstances and a history of defaulting from appointments who were offered the option of video consultations (see Multimedia Appendix 1 for our 2014 Diabetes Review, Engagement and Management via Skype [DREAMS] report).

    This study occurred at a time when the trust was seeking to learn from the diabetes pilot and make video outpatient consultations part of business as usual whenever clinically appropriate. We worked mainly with three services on separate sites in east London: diabetes (adult and young adult), which had been piloting the virtual consultation model; antenatal diabetes, which sought to use such consultations to support close (sometimes daily) monitoring of diabetes in pregnancy; and hepatobiliary and pancreatic cancer, a tertiary care service to which patients were sometimes referred from beyond London, requiring a round trip of several hours. By the end of the study period, other specialties at Barts Health (including neurology, rheumatology, hematology, and endocrinology) had begun to introduce video outpatient consultations.

    Project Management and Governance

    The study was delivered by a core working group (TG, SV, JW, JM, and SS), supported by a 6-monthly independent steering group and a patient advisory group (see below). The steering group had a lay chair and cross-sector stakeholder representation, including patients, National Health Service (NHS) stakeholders, and national-level decision makers (details in Multimedia Appendix 2). The study received ethics approval from City Road and Hampstead NHS Research Ethics Committee on December 9, 2014 (reference 14/LO/1883) and subsequent amendments.

    Participants and Data Sources

    We collected data over a 28-month period from 36 national-level stakeholders; 24 staff at Barts Health (9 clinicians, 5 managers, 3 technical support staff, 7 administrative support staff); and a total of 50 patients. Data sources are summarized in Table 1 and described in more detail below.

    Sampling and Data Collection: Macro Level

    We began with individuals charged with delivering information technology (IT) strategy in NHS England, as well as those leading on patient participation. Alongside review of policy documents (from 2000), we used snowball sampling (asking each interviewee to nominate a colleague) to build up a picture of the national context. We invited a maximum variety sample of 39 stakeholders from across government (eg, NHS England, Care Quality Commission, and NHS Improvement), professional organizations (eg, Royal College of Physicians and Medical Protection Society), patient groups (eg, National Voices), industry (eg, Microsoft), and charitable and third sector organizations (eg, Health Foundation), of whom 36 agreed to talk informally with the study team (3 were uncontactable). We then undertook audiotaped, semistructured interviews with a purposive sample of 12 of these stakeholders, ensuring variation in the different institutions, groups, and perspectives represented. Stakeholder details and interview guides are available in Multimedia Appendix 2.

    Table 1. Overview of multilevel data collection and analysis in Virtual Online Consultations: Advantages and Limitations (VOCAL) study.
    View this table

    Sampling and Data Collection: Meso Level

    The goal of sampling was to map the people, interactions, and organizational routines that support the virtual consultation with a view to building a rich “ecological” picture of the sociotechnical microsystem [49] (and its wider embedding in the organization) needed to make this service model work as business as usual. We began from participating clinics, mapped the individuals and technologies involved there, and then moved outwards to include, for example, finance and clinical informatics departments.

    In total, we conducted over 300 hours of observation of consultations and the clinical and administrative work supporting them, combined with semistructured or naturalistic interviews (asking people “on the job” what they are doing and why they are doing it, as people often find it easier to talk about the detail of their job while they are actually doing it [50]) with 24 staff. We also collected 16 local documents (business plans, informal guides, emails, and minutes of meetings) and descriptive statistics from each clinic.

    Sampling and Data Collection: Micro Level

    We used audio, video, and screen capture to produce rich multimodal data on a total of 30 virtual consultations and made audio recordings of 17 face-to-face recordings matched for clinical condition (in which the clinician stated the encounter could have happened virtually). Details of these consultations are shown in Table 1. We sought maximum variation in age; ethnicity; and clinical, social, and personal circumstances. It was a precondition of ethical approval that clinicians were able to exercise judgment about which patients to invite to join the study.

    Specific exclusion criteria were as follows: no 3G access at home, lack of familiarity (by patient or carer) with the relevant technology, clinical inappropriateness (eg, need for direct physical examination), inability to give informed consent, and comorbidity preventing participation (eg, severe visual impairment). Clinic populations included a high proportion of limited English speakers. In the young adult diabetes clinic, bilingual health advocates were available and trained in the use of remote consulting, so limited English was not an exclusion criterion there. In the antenatal diabetes and cancer clinics, a remote interpreting service was not available, but patients comfortable with a family member interpreter were included.

    Our micro-level dataset consisted of video recordings of consultations incorporating two video streams: one from the clinic and one from the remote site (typically the patient’s home). We recorded consultations using a small digital camcorder with wide-angle lens and remote control (Sony Handycam DCR-SR72), mounted on a mini tripod. We used a commercially available screen capture software tool (ACA Systems) run directly from an encrypted Universal Serial Bus (USB) stick to record screen images showing on each party’s computer screen as a video file. A researcher started and stopped the recordings but left the room during the consultation. When the patient used a mobile, tablet, or Mac computer (which could not run the encrypted USB device), the researcher positioned a second digital camera to capture the screen. In 12 cases, the consultation was recorded at the clinic end but not at the patient end for logistical or patient preference reasons.

    Each end of the consultation resulted in 2 digital files, one screen capture and one video. We synchronized these into one file using video editing software (Sony Movie Studio)—allowing us to play the video of the computer screen exactly in parallel with a video of the patient looking at the screen and to view the patient and clinician “ends” in parallel. Face-to-face consultations were audiorecorded using a digital dictaphone. Further details of the informed consent process are given in Multimedia Appendix 2.

    The micro-level dataset also included contemporaneous field notes from patients’ homes (eg, material circumstances) and the clinic (eg, physical surroundings and use of paper or electronic records). We also collected demographic data (age, gender, and ethnicity) on patients using the Skype option for remote consulting and (for comparison) on the clinic population as a whole over a 12-month period at each clinic setting.

    Theoretical Framework

    In our original study protocol [14], we drew on a technology-enriched version of Giddens’ structuration theory [14], which proposes a dynamic and reciprocal link between (1) the external social environment, (2) human interpretations and action, and (3) technologies; it considers how the relationship between these evolves over time, each shaping the others. For example, the theory explores how human action reproduces and reinforces social norms; how societal expectations (including professional norms and codes of practice) influence the “scripts” built into technologies; how technologies, through their functionality and affordances, make some actions possible but others impossible; and how laws, regulatory restrictions, policy priorities, and professional codes of conduct mean that even when an action is technically and physically possible, it is not in reality an option [51].

    The health care setting is heavily institutionalized (ie, our behavior is influenced primarily by expectations of how we should or must behave in this setting rather than simply by economic or personal concerns, such as maximizing efficiency or pleasure). In such circumstances, behavior is often ritualized (ie, we know and play out the roles expected of us as doctors, nurses, patients, and so on). A key question driving our data collection and analysis was how would the technological and material aspects of the remote consultation shape, enable, and constrain the playing out of these institutionalized roles and behaviors.

    As the study unfolded, we enriched and extended this initial theoretical framework with additional material on, for example, clinical aspects of the illness or condition, the kind of knowledge that is foregrounded (or backgrounded) by the technology, and commercial and regulatory considerations. The resulting theoretical framework (NASSS) has recently been published [37].

    Action Research

    As described in detail previously [14], our study was informed by the principles of action research [52], defined as “a mutual learning process within which people work together to discover what the issues are, why they exist, and how they might be addressed” [53]. Data collected by and with the research team were fed back formatively to inform development of the service (for instance, where appropriate, we sought to support plans for rollout of virtual consultations across the hospital). In the early stages of the study, we held two formative learning workshops to feed back our findings. As the study progressed, we were welcomed into the trust’s existing governance structures (including an outpatient strategy group set up to drive the rollout of virtual consultations and the local information governance department to develop standards and guidance for such consultations) so bespoke feedback meetings became unnecessary. We also fed back emerging findings periodically to national-level decision makers (for instance, relating to national payment systems) both via bespoke meetings and also because a national policy maker with responsibility for NHS IT was on the VOCAL steering group and another worked closely with us on a related project.

    Data Analysis: Macro Level

    Interviews with national stakeholders were initially analyzed thematically to provide context for the statements, actions, and interpretations made by organizational actors at local level. We also used interpretive policy analysis [54,55] to identify the key “storylines” [56] shaping policy and debate around remote consultations and to surface inconsistencies and ambiguities between local and national perspectives.

    Data Analysis: Meso Level

    Our approach to mapping the sociotechnical health care ecosystem [49] provided detailed data about the logistical and technical barriers involved in introducing and running remote consultation services (in diabetes and cancer clinics, as well as the wider hospital such as IT and information governance departments) and workarounds to overcome them. This included data about issues related to technology, clinic management, administrative processes, patient enrollment, and the exercise of clinical judgment. We used diagrams and narrative as synthesizing devices to draw together a visual representation and linked verbal account of these human and technical interactions and interdependencies.

    We also drew on the notion of “organizational routines” [57,58] defined as “recognizable, repetitive patterns of interdependent action carried out by multiple actors” [59]. Routines are how organizational life is patterned, and hence, studying them provides important insights into how innovations such as remote consultations may (or may not) be assimilated in health care and how that assimilation changes over time. We identified the work required (at clinic, departmental, and executive levels) to “routinize” aspects of the virtual consultation service; examined the dynamics within and across different routines; and analyzed the convergence between stated (or proxy) routines, clinician and staff understandings about how to enact it (ostensive routine), and the range of ways in which it is then carried out (performative routine). This allowed us to reveal the tension between current business as usual and the new ways of working implied by a video consultation model.

    Data Analysis: Micro Level

    Our initial analysis of micro-level data involved repeated viewing of selected virtual consultations and discussion in our interdisciplinary team (including sociology, linguistics, human computer interaction, and medicine), alongside review of interview data with patients and clinicians. This led us to identify a number of questions relating, for instance, to the ways in which the context of the consultation (often involving patients in their home setting and clinician at the clinic) shaped communication; whether the usual format of the medical consultation (opening, history taking, examination, diagnosis, and review) might shift when conducted remotely; how talk about technology might reorient patient-clinician interaction; and how sensitive topics (such as breaking bad news) might play out differently.

    On the basis of these early emerging themes, we sought a methodology to add depth and detail to our findings and identified the Roter interaction analysis system (RIAS), a widely used method for coding medical dialogue [60]. Broadly derived from social exchange theories related to interpersonal influence, problem-solving, and reciprocity [61-63], RIAS offers a validated coding system [60], allowing researchers to systematically quantify the occurrence of different types of talk that occur during medical encounters that reflects accepted patient and provider roles and obligations in a “meeting between experts” [60]. It has been used extensively to analyze face-to-face consultations but rarely in remote settings. We identified one paper (a conference proceeding) that explored the theoretical potential of RIAS in technology-mediated consultations [64], 3 small empirical studies in different clinical conditions [65-67], and a validation study of new RIAS codes for technology-related talk [68].

    Roter’s original taxonomy distinguishes three main categories of talk: “task-focused” (eg, application of medical expertise to diagnose and manage disease), “socioemotional” (eg, greeting, building rapport, and showing concern), and “process” (eg, inviting the patient to sit down). In this study, we also used a fourth category: “technology talk” (eg, that the picture is fuzzy), initially introduced by other researchers [66,68] and adapted and extended by our own team. Table A1 in Multimedia Appendix 2 shows the high-level clusters and more detailed categories used in RIAS with examples drawn from our data.

    Following a 3-day training course delivered by the team that originally developed RIAS, we applied the RIAS coding scheme to our micro-level data, addressing five questions about the differences between virtual and face-to-face consultations for the same clinical condition:

    1. Are remote consultations shorter and more “to the point” than face-to-face ones?
    2. How do they differ in the different kinds of (nontechnology-related) talk that occurs?
    3. What kind of technology-related talk occurs?
    4. What kinds of breaches (misunderstandings, “repairs,” and so on) of talk occur in virtual consultations, when do such breaches occur, to what extent do they matter, and how might they be reduced?
    5. How do interruptions (in the patient’s home and/or in the clinician’s office) affect the flow of talk in the virtual consultation?

    The assumptions for normal distribution of the data were not accepted (Shapiro-Wilk normality test was significant at P<.05). Therefore, Mann-Whitney U tests (nonparametric) were used to compare interactions for virtual and face-to-face consultations.

    Patient Involvement Statement

    The initial impetus for introducing virtual outpatient consultations in the diabetes service was from service users (many from deprived socioeconomic backgrounds and/or minority ethnic groups) who found it difficult to attend face-to-face appointments. As noted above, the VOCAL steering group had a lay chair and patient representation. In addition, we sought ongoing patient feedback on both the research process and the developing video outpatient services at Barts Health via a dedicated patient advisory group, with 12 patients (and one spouse) representing a wide range of ages and ethnic groups and facilitated by an anthropologist with close knowledge of the local community. This group met three times throughout the study, supplemented with ad hoc contact with individual members (eg, to invite input on lay summaries). In one of the meetings, the patient advisory group was shown (with the written consent of participants) two video clip recordings of virtual consultations as the basis for discussion. In addition, some preexisting lay groups (one antenatal group consisting of 9 mothers and one spouse, a peer support group for gestational diabetes, and a young people’s peer support group for diabetes) and a cancer support charity were consulted to extend the range of patient and lay input.


    Results

    Macro-Level Findings

    The external context for technological innovation in the UK public sector is currently extremely challenging. There is a strong policy push to develop UK’s digital economy [69-73], digital government [74-77], and digital health [7,8,70,78-85]. We found no national policy specifically relating to the introduction of virtual consultations. As one senior decision maker said, “There are pockets of success, and there are certain vanguards exploring it, there’s bits and bobs. But there’s not actively a digital fund for telehealth.” Rather, the policy of using technologies to support new service models appears to be folded into other programs such as the NHS Five-Year Forward View (2015-20) [2] and the General Practice Forward View (2016-21) [86]. The former includes the NHS “vanguards” to test 50 local innovative service models supported by a dedicated tranche of innovation funding [87]. An independent review in 2016 of IT in the NHS called for “new national funding to help trusts go digital and achieve maximum benefit from digitisation” [7] and led to the appointment of 12 NHS hospital trusts as “digital exemplars” with generous additional funding for introducing new technologies [88].

    Notwithstanding these initiatives, constraints imposed by financial austerity—spending plans, for example, indicate a decreasing share of gross domestic product being devoted to the NHS from around 7.3% in 2016 and 2017 to 6.9% by 2022 and 2023 [89]—have meant that there has been a little slack in supporting the piloting, organizational learning, and extensive groundwork that is often needed to routinize new technologies or practices. Low growth in NHS budgets [81] combined with sustained increases in demand are taking their toll on providers [90-92].

    National-level stakeholders and documents depicted technological innovation as imminently poised to deliver efficiency savings in the NHS “at scale” and “at pace” [81,93,94], thereby helping solve the growing challenge of rising demand and falling budgets and also producing “a virtuous circle of economic growth for the UK” [8]. The industry sector (whether global companies seeking long-term strategic partnerships or small- and medium-sized enterprises offering “niche” products) is depicted in these documents as the innovator and producer of “transformative technologies” [81], with the implication that the technologies, once produced, will drive transformation of the NHS. The potential time lag between adoption of technology and realization of productivity gains (if any) was rarely acknowledged either by interviewees or in policy documents.

    Few interviewees mentioned low digital literacy and digital access. In the United Kingdom, 9% of all citizens (and a disproportionate number of the poor, sick, and elderly) have never used the Internet [95]. The UK government has an active digital inclusion strategy [70,71,96], which appears to be predicated on a “deficit” model (ie, it assumes that with support and training, everyone will be able and willing to connect digitally to public services). An alternative perspective, taken by patient and advocacy organizations, is that digital exclusion has complex underlying causes linked to the social determinants of health (eg, poverty, social exclusion, language, and literacy issues) and that “training” alone will not overcome these [97].

    There was broad consensus among national-level stakeholders that the current evidence base for technology-supported new service models is weak. However, there were striking sectoral differences in what kind of evidence stakeholders believed was needed. Industry interviewees expressed confidence in the “fail early, fail often” approach of iterating software design to optimize the use of a technology in a particular setting. Interviewees from regulatory bodies and professional organizations appeared keen on qualitative evidence (they wanted to know more about, for instance, what makes a good quality remote consultation). In contrast, clinicians and policy makers placed high value on RCT evidence generated elsewhere but assumed to be transferable to the current setting [2].

    The introduction of virtual consultations was viewed by industry informants as a particularly difficult and risky aspect of NHS IT development because it required major changes to service models and institutional embedding. Suppliers lamented slow time frames, “decision making by committee,” “so much duplication,” “consultation about everything,” and the “need for everyone to go out and evaluate every single thing.”

    The prevailing emphasis by NHS England and local providers on one-off procurement contracts for particular technologies contrasts with the strategic desire of many industry stakeholders to develop mature partnerships in which industry commits to supporting an evolving service via an evolving package of technology and support. Industry informants were therefore, perhaps reluctantly, prioritizing business initiatives that rested on “off-the-shelf” technologies that could be bought, installed, and made to work with minimal ongoing work to embed, routinize, and sustain them. They saw greater potential, for example, in what one industry executive called the “wellness and wearables market”—technologies that were more or less freestanding and could often be promoted direct to consumers.

    A key issue repeatedly raised by interviewees but rarely evident in published documents was how reimbursement for virtual consultations would be implemented. As one senior decision maker in NHS England told us, “we have a drug tariff that does prescriptions very well, but we don’t have anything for digital.”

    Our analysis highlighted several strands of work being undertaken by national-level bodies (NHS Improvement and NHS England) to review payment and pricing. The development of a national Innovation and Technology Tariff for England has gone some way to addressing this by removing the need for multiple local price negotiations (instead assuring reimbursement when an approved innovation is used) but is currently limited to 6 specific medical devices or apps and does not include virtual consultation technologies [98]. The need for individual provider organizations to negotiate payment for virtual consultations with local clinical commissioning groups (even as a temporary “pass through” solution) is likely to be a significant barrier to national rollout and might also compromise the long-term sustainability of existing virtual consultation services. The proposed shift away from activity-based funding to capitated payments in the NHS could potentially overcome this problem but is likely to be several years in development [2,99,100].

    Meso-Level Findings

    The action research element of the study meant that we experienced first-hand the reality of setting up and delivering the virtual consultation service in a busy public-sector hospital trust. Despite the fact that Barts Health had been an early adopter of virtual consultations and a willing partner in the research proposal, implementation proved far more complex and difficult than anticipated. One key barrier to progress was the extreme pressure on human and financial resources. Clinicians, managers, and technical staff in every department were under pressure; key posts were unfilled or frozen, and clinics were very heavily booked.

    Early success of video consultations in the diabetes service had been achieved partly through workarounds (eg, installation and support of Skype on a small number of machines as a “favor”). Mainstreaming this same service as business as usual across the trust—a shift that required many processes and activities to be formalized and managed centrally—was initially strongly resisted by the IT department because it threatened to “open the floodgates” in an overstretched department and risked impacting the network bandwidth. Yet, support from the IT department (to set up machines, troubleshoot, provide on-the-job training, and configure electronic booking systems to log “video” appointments) was mission-critical, as was the development and implementation of an information governance protocol to ensure compliance with legal and regulatory standards around privacy and data protection. The latter proved extremely time-consuming even with support from the research team who worked with key local staff to develop standard operating procedures and help navigate these through the trust’s approval mechanisms. The resulting document (see Multimedia Appendix 2) was reviewed and endorsed by the UK Information Governance Alliance (IGA), who used it as the starting point for developing national policy guidelines on the use of Skype and FaceTime across the NHS. We also developed a patient information leaflet with input from our patient advisory group and a guidance document for staff on setting up Skype services, which was used to support rollout across the trust (see Multimedia Appendix 3).

    Although some clinicians embraced the new technology with enthusiasm, many others were unwilling to try it (claiming to be “too busy”). Those who did join the study talked positively in interviews about the high quality of most video consultations and believed (correctly as it turned out) that video consultations were generally shorter than the equivalent face-to-face encounter. However, they also commented that they had had to take on a number of new roles and practices. These included triage (judging a patient’s suitability for virtual consultation), finding appropriate physical space for “private” Skype interactions outside clinic hours, troubleshooting IT, patient setup (ensuring the technology worked and supporting patients with its use), and medical documentation (adjusting to the different ways in which electronic and paper documents and other artefacts were used in consultations).

    Virtual consultations appeared to work best for long-term conditions in which the clinician and patient had a preexisting relationship with a high degree of mutual trust and “common ground” when interdepartmental coordination over clinical care was not required, when the need for close physical examination could be excluded in advance, when there was a clear relative advantage to virtual consulting (eg, a history of defaulting from face-to-face appointments or clinical or practical barriers to the patient traveling), when both parties were confident and competent with technical issues, and/or when there was a pressing clinical need to have repeated contacts with the patient. In the (relatively rare) circumstances in which key clinical, technical, and practical preconditions were met, video consultations appeared to be safe and popular with both patients and staff (patients were particularly positive about convenience and not having to take time off work).

    The many fine-grained differences between the clinic routine for a face-to-face consultation (see example Figure 1) and a video one (Figure 2) illustrate why the process of “embedding” generated both new work for immediate staff and also knock-on effects elsewhere in the organization. Embedding work broadly related to four key processes or subroutines: generating data or information (highlighted gray in Figures 1 and 2), enabling access to data or information (highlighted yellow), facilitating patient access through the clinic (highlighted blue), and tracking the patient through the clinic and care pathway (highlighted green). Crucially, each of these processes was supported by technical and physical artefacts, the movement of artefacts across (virtual and physical) spaces, and the input of multiple clinical and nonclinical actors.

    The physical presence of the patient within the clinic setting was fundamental to existing ways of identifying, scheduling, conducting, rebooking, and monitoring patient appointments. For instance, in Figure 1, the physical presence of the patient at reception prompted “check in” and generation of the clinic outcome form: this enabled the nurse assistant to identify the patient, conduct relevant tests, and record the results on the form, which in turn enabled the remainder of the consultation to take place. Embedding virtual consultations in the work of the clinic involved significant reworking of those processes in ways that took account of the “virtual” presence of the patient—as illustrated in Figures 1 and 2 and the additional examples in figures A1 to A4 in Multimedia Appendix 2. The extent to which face-to-face consultation routines needed to be reoriented, and how this reorientation was managed by staff, varied across the three clinic settings, depending on the people, technologies, material artefacts, physical and spatial arrangements, clinical pathways, and assessment procedures already in place.

    Another key challenge to the introduction of video outpatient consultations was the reconfiguration of the electronic patient record system to allow the booking of video appointments on the clinic appointment schedules. Each consultant had a “profile” on the electronic record through which appointments were booked by the administration teams. The available appointment types and time slots that could be booked were configured according to their existing clinic schedule. The reconfiguration of the booking schedule to enable the video appointment option was managed by the trust’s information and communication technology (ICT) support department, which developed protocols for the types of appointments that could be integrated into staff profiles and specifications for the payment tariff allocated to these appointments.

    Figure 1. Routine for face-to-face consultation in diabetes adult or young adult clinic.
    View this figure
    Figure 2. Routine for a virtual consultation in the diabetes adult or young adult clinic.
    View this figure

    The introduction of the video consultation option in any clinical service involved a lengthy process of form filling, enquiries (email or phone), and discussions and agreements among senior service managers.

    By the end of the study, the video option had to some extent become business as usual in the diabetes adult and young adult clinic (see Figures 1 and 2) but had evolved in a different way from the original plan. Although only 3.6% of prebooked outpatient appointments for the consultant diabetologist were formally coded on the electronic record as having occurred via Skype, the actual proportion was much higher because not all remote consultations were coded as such. This was partly because a new electronic record system (without a code for remote consultations) was introduced as part of the service part-way through the study and partly because of ad hoc consultations, discussed below.

    A significant use of video consultation by Skype was for supplementary clinician-initiated and/or spontaneous patient-initiated encounters (eg, as an ad hoc measure for keeping in close touch with patients who were undergoing a temporary period of instability or heightened need). If ad hoc encounters are added to the denominator, approximately 11% of all outpatient consultations in the diabetes clinic (and 22% of the lead diabetologist’s) were undertaken by video. Clinicians liked the ease with which vulnerable and “hard to reach” patients could send a message via Skype requesting a virtual encounter, allowing prompt clinical input that clinicians believed improved patients’ confidence in self-management, and which in some cases may have averted a serious complication or hospital admission. It was not possible to produce a reliable estimate of the extent to which these ad hoc encounters replaced rather than added to the clinic’s conventional workload or prevented a hospital admission.

    In the antenatal diabetes clinic (whose detailed routines are represented diagrammatically in figures A1 and A2 in Multimedia Appendix 2), only one clinician ever used the Skype service (for 2% of her encounters), and it was abandoned after a pilot period. In this (extremely busy) clinic, virtual consultations aligned poorly with a context involving multidisciplinary teams (patients were typically consulting multiple clinicians across departments) with a relatively short-term but high-risk condition (gestational or preexisting diabetes in pregnancy) and in the absence of integrated medical records (paper medical notes being held by the patient and so not physically present at the clinician end).

    In the hepatobiliary and pancreatic cancer clinic (a tertiary referral service, figures A3 and A4 in Multimedia Appendix 2), virtual consultations were popular and generally unproblematic for follow-up after cancer surgery (a time when it was neither convenient nor clinically recommended for patients to make a long journey to the clinic). Clinicians reported that the dynamic of consultations was more relaxed (eg, being introduced to family members and pets), and some patients said they preferred to receive bad news (eg, signs of recurrence) in the comfort of their home without the ordeal of a long journey home afterwards. The proportion of all cancer follow-up consultations undertaken via video link rose from 7% to 20% during the course of the study.

    There was no significant difference in demographic characteristics (age, gender, or ethnicity) between patients using the Skype option and the overall demographic of the patient population in the antenatal diabetes and cancer surgery settings. For adult and young adult consultant-led appointments in diabetes, there was no significant difference in terms of gender and ethnicity. However, there was a significant difference in age profile, with underrepresentation of patients older than 55 years taking up the Skype option (χ23[N=307]=11.7, P=.01). Older patients opting for video consultations usually had a technology-savvy younger relative who offered to help.

    In all virtual consultation services, there were multiple technical issues to be addressed. These were often easily resolvable, but not all patients (or staff) were sufficiently skilled or confident to undertake the necessary “troubleshooting” to achieve and maintain the video connection or resolve audio quality problems.

    The VOCAL study came at a time when senior management at the trust were turning their attention to rollout of virtual consultations beyond a handful of clinics—a factor that was crucial to establishing an outpatient project strategy group (chaired by the chief medical officer) focused on supporting local rollout. That work needed to accommodate competing policy priorities locally and nationally and work with national policy makers, regulators, and industry partners (including the IGA, NHS England, Clinical Commissioning Groups, and Microsoft UK) to find workable ways forward through close dialogue and practical problem-solving.

    Micro-Level Findings

    Our micro-level dataset is summarized in Table 2. Key findings are described below.

    The opening sequence of a video consultation was very different from that of a face-to-face one. In the former, there was invariably a “technical set-up” phase before clinical talk began. Clinicians sometimes conducted test calls so the patient could familiarize themselves with the Skype technology and/or check that the video and audio worked before the consultation. Clinicians often talked patients through minor technical problems while consulting. Examples of video and face-to-face opening sequences are shown in Textboxes 1 and 2, respectively.

    Technical problems included lack of sound, poor sound quality, loss of picture, and patient failing to activate video. All were relatively minor and resolved satisfactorily, sometimes through “workarounds.” For example, poor audio quality in two consultations required patient and clinician to communicate via telephone, muting the sounds while simultaneously running the video display; the problem was later found to be caused by a letter dictation device plugged into the clinician’s computer. Technical issues sometimes led to conversational breaches, typically characterized by concern and/or humor, but the flow of conversation was usually quickly restored (see Textbox 3 for illustration).

    Face-to-face consultations were characterized by shared physical space (eg, across the corner of a desk); patients and clinicians typically engaged together with numerous physical artefacts (paper notes, diary, smartphone, insulin pen or pump, scraps of paper, and sticky notes) as consultations unfolded. In video consultations, both parties had to compensate for lack of shared space and artefacts (eg, by holding a page up to the screen or reading aloud from a set of home blood glucose readings).

    Apart from technical issues and differences linked to physical layout, video and face-to-face consultations within any specialty were strikingly similar. The content and flow of most video consultations in our dataset appeared to be high quality, though a small fraction appeared awkward and disjointed, with parties frequently misunderstanding or talking over each other and/or needing to seek clarification. Further analysis of these “awkward” consultations is ongoing, but no major safety concerns were identified.

    Table 2. Overview of consultations in our micro-level dataset.
    View this table

    Textbox 1. Example of opening exchange of a virtual consultation for antenatal diabetes. (xx)=length of pause in seconds.
    View this box

    Textbox 2. Example of opening exchange of a face-to-face antenatal diabetes consultation. (xx)=length of pause in seconds.
    View this box

    Textbox 3. Conversational breach related to reduced video quality during cancer surgery follow-up appointment. (()) refers to unintelligible speech.
    View this box

    Video consultations presented new possibilities for interruption. This included disruptions related to the technology (eg, loss of sound and incoming call on the mobile device being used for the consultation), as well as nontechnological interruptions in the domestic environment (eg, family members entering the room). In all cases in our dataset, flow of the consultation resumed readily after such interruptions.

    Findings from our RIAS analysis are summarized in Table 3. Consultation length (defined by RIAS as frequency of utterances) was, overall, 13.34% (584/4379) shorter than comparable face-to-face encounters in all three clinical services studied, even taking account of the small amount of initial “technical talk” to establish the connection, constituting 4.43% (168/3795) of all talk during virtual consultations. However, these differences in length were not statistically significant (U=121.5, P=.43). The extent to which the clinician did more talking (“dominance”) and exerted more control (“directedness”) was similar in both video and face-to-face consultations in each specialty (though it varied across specialties, perhaps reflecting differences in clinical scope and/or clinicians’ consulting styles). The one statistically significant difference in clinician dominance was in the diabetes antenatal setting, in which consultations were slightly more clinician-dominated during remote (median=1.2; interquartile range [IQR]=0.3) than face-to-face (median=1.7, IQR=0.5) consultations (U=3.5, P=.02). This was probably explained by patients, at the clinician’s request, reading out home blood glucose readings and insulin doses in the video consultations.

    A more fine-grained analysis of the different types of talk, which we will present in a separate publication (Wherton et al, in preparation) likewise confirmed only small and mostly nonstatistically significant differences in categories such as “verbal attentiveness,” “making requests,” “giving information,” and “counseling” (see full list of categories in Multimedia Appendix 2); significant differences were again explained by the material circumstances of the consultation.

    None of the other differences between video and face-to-face consultations in the above table were statistically significant.

    Table 3. Median and interquartile ranges (IQR) for clinician and patient talk in virtual and face-to-face consultations, based on Roter interaction analysis system.
    View this table

    The RIAS analysis did not include the “ad hoc” consultations that occurred in the diabetes clinic (in which, eg, patients sought an immediate, and often very quick, Skype encounter with a clinician known to them to sort out a problem with insulin dosage).


    Discussion

    Statement of Principal Findings 

    This study has confirmed findings from randomized trials that when clinical, technical, and practical preconditions are met, video consultations are safe, effective, and popular with participating patients and staff. In most cases, video consultations consisted of similar types of talk, in similar proportions, to comparable face-to-face consultations, and differences between different clinical specialties were more striking than those because of the technology. By the end of this study, between 2% and 22% of all consultations were being undertaken via video link by participating clinicians. In the remainder, the video option was considered impractical, technically unachievable, or clinically inadvisable for the patient. Technical challenges were typically minor but potentially prohibitive.

    Although these findings confirm that video consultations may have an important place in transforming care models, some staff members chose not to participate, and patients for whom video consultations were deemed appropriate represented a fraction of the overall clinic workload in all specialties studied.

    Notwithstanding policy interest in digital solutions, the reality of establishing video outpatient services in a busy and financially stretched acute trust proved far more complex and time-consuming than anticipated—mainly due to lack of “organizational slack” [101], disruption of traditional clinic routines, and real and perceived information governance challenges.

    Although national policy makers viewed video consultations as a driver of change (supporting new, more efficient service models), industry informants viewed this option as low priority because of anticipated (and experienced) challenges of working with the NHS on projects that required complex organizational, policy, and regulatory changes.

    These findings can be theorized using our recently published NASSS framework that was developed to explain why, despite significant investment and high expectations, five problems persist: digital technologies are either not adopted or soon abandoned by professionals and/or their patients and clients or else the technology-supported service succeeds as a small-scale demonstration project but fails to scale up locally, spread to other comparable settings, or be sustained over time. The NASSS framework analyses these problems in terms of seven interacting domains: the condition, the technology, the value proposition, individual adopters (staff and patients), the organization, the external (eg, regulatory and policy) context, and emergence over time [37]. Each domain can be simple (few components, predictable—as in making a sandwich), complicated (multiple components but still largely predictable—as in building a rocket), or complex (dynamic, composed of multiple interacting elements, and unpredictable—as in raising a child).

    To the extent that VOCAL was successful in establishing a video consultation service, this was explained by the various NASSS domains: straightforward, predictable, and low-risk clinical conditions; simple and dependable technology that was fit-for-purpose; clear benefits for both the technology supplier and the patient; acceptance of the technology by staff (who considered that the technology supported and extended their professional role) and patients (who were able and willing to develop new skills and ways of engaging); alignment with existing—or emerging—organizational routines; and a strong policy push. To the extent that efforts to introduce video consultations were unsuccessful, this can be explained by complexity and unpredictability in the clinical condition; lack of dependability and fitness-for-purpose of the technology; lack of acceptance by staff or patients; limited organizational slack, lack of shared vision, and/or clashes with long-held and difficult-to-change routines; and tricky regulatory or policy issues (eg, national concerns about information governance and lack of a national tariff for virtual consultations).

    Our study has also illustrated, through detailed multilevel analysis, the interdependence of the different domains in the NASSS framework. For example, our national-level interviews identified a reluctance among major technology vendors in the United Kingdom (not just Microsoft) to make major investments in partnerships with the NHS. This meant that, at the time of writing, the technology being used was an off-the-shelf product that had not been specially adapted for use in video consultations and that this technology was not high priority for support from the local IT department. This partly explains why significant clinician time (and an extension of the clinician role) was needed to complete such tasks as new appointment booking and management of a virtual waiting room. Our findings suggest that proactive codesign between technology suppliers, participating health care organizations, and national policy makers could potentially produce three things: video software that is more fit for purpose, organizational routines that are better aligned to support video consulting, and better incentives for major suppliers to work in a collaborative and ongoing way with health care providers.

    Although our study was not designed to generate a simple or universal “checklist” for implementing video consultations, it is worth reproducing here the five “key recommendations for practice” aimed at clinicians and managers that were coproduced through action research in this study:

    • Introduce the service slowly and incrementally with direct involvement of the team to ensure compatibility between the technology and existing practice
    • Allow plenty of time for discussion with staff and patients about how it affects the service
    • Work in collaboration with your ICT department and technical support teams to establish roles and processes to assist use of the technology
    • Use with an understanding of the patients’ lives and how the technology relates to the management of their health condition
    • Support flexible use, allowing scope to fit the service around the needs of the patient

    Strengths and Weaknesses of the Study 

    To our knowledge, this is the first research study in any clinical field to have taken an in-depth, mixed-methods, and multilevel approach to the study of video outpatient consultations. We succeeded in our goal of collecting rich qualitative data that exposed the “messy reality” of establishing a virtual consultation service and illuminated the pros and cons of using this medium for clinical interaction in different settings. Using action research, we were able to inform and facilitate the work of embedding the new service model and gain detailed insight into organizational complexities and how these changed over time. Working with front-line clinical and technical services, we have developed significant expertise, standard operating procedures, information governance and technical guidance documents, and protocols for setting up and running video outpatient clinics. National policy makers have been engaged from the outset, and the study has attracted interest from other hospitals. A rollout phase continues within the trust, and further work is also ongoing to extend the model to other NHS organizations across the United Kingdom.

    The main limitation of the study is its focus on a single (albeit large and multisite) health care organization. Barts Health was even more financially stretched than most acute trusts in the United Kingdom; hospitals under less tight financial and staffing pressures may have found the implementation work easier. In addition, the sample size for the detailed analysis of virtual consultations was relatively small.

    Comparison With Other Studies

    Almost all previous research on video consultations in health care has either addressed the technical detail of the remote connection or undertaken an RCT of virtual vs face-to-face consultations [13-18,20-34]. Such studies lend support to the conclusion that in selected patients, video consultations are noninferior to face-to-face ones—but (often by their own admission) they leave unanswered the question of how to establish the service as a real-world option and/or move from a small-scale research or demonstration project to sustainable business as usual. Our finding that, in contrast with concerns raised by previous authors, the technical quality of Skype interactions via available broadband in London was almost always adequate affirms a recent study by others of 4G mobile technology [43].

    Some critics will view it as a limitation that we did not emulate the experimental methodology of previous studies. This was deliberate. Our findings have confirmed that virtual consultations cannot be treated like a drug or even as a complex behavioral intervention to be tested “on” patients. Rather, they are the result of a hugely complex sociotechnical system in which “successful” virtual consulting is contingent on multiple factors at multiple levels. If we appear to have produced ambiguous findings, this is perhaps because ambiguity and tension are inherent to complex sociotechnical systems. To questions such as “do virtual consultations work?,” “are virtual consultations safe?,” and “are virtual consultations cost effective?,” we suggest the answer will always be “it depends.”

    Meaning of the Study

    In the context of a strong policy push to develop digital alternatives to the traditional consultation, delivering video outpatient services at scale is likely to be far from straightforward, as rollout in any locality will be influenced by (among other things) prevailing organizational culture, financial and human resources and priorities for allocating these, existing organizational and technical infrastructure, the nature and causes of professional resistance, information governance challenges, and the logistics of payment. Video consultations, although safe and effective for selected patients, fundamentally change the nature of outpatient care and require clinician buy-in (which may or may not be forthcoming). Industry, although not opposed to the idea of developing software to support video consultations, appears to view organizationally embedded technical solutions as relatively low priority.

    The finding that efforts to implement a video consultation service met with multiple challenges in relation to workability and integration aligns with numerous previous studies of other forms of remote care (see in particular Finch and May’s work on telehealth, which formed the empirical basis for May’s normalization process theory [38,39]). Indeed, these difficulties may even have worsened in recent years as clinical work has become more protocolized and financial pressures more severe.

    Unanswered Questions and Future Research

    This study has, in some way, revealed and explored the challenges to establishing video outpatient consultations as a real-world service. Overcoming those challenges will not be easy, but further in-depth case studies in both comparable and contrasting settings are likely to enrich our understanding. As more health care organizations make the strategic decision to introduce video consultation services, research could explore the collaboration and mutual learning that occurs between them and test approaches to supporting that interorganizational interaction.

    Our macro-level interviews identified a consistent finding from industry informants that the NHS is currently a uniquely difficult setting in which to attempt to introduce technologies that imply major changes in service models. Industry’s preferred model—of long-term partnerships (for technologies plus service support to embed them) rather than one-off procurement contracts—should be introduced in test sites and carefully researched using longitudinal ethnography. The research agenda here is methodological as well empirical; it is founded (we believe) on the notion that technologies and services are continually evolving and mutually shaping; they cannot be fixed in time nor (therefore) be adequately tested using traditional randomized trial designs.

    One of the most interesting findings of this study was that the technology provided opportunities for clinicians and patients to use the technology adaptively and differently, allowing new modes of consulting to evolve (eg, patient-initiated contacts direct to the clinician via Skype messaging, which appeared to help supported self-adjustment of insulin dosage in diabetes). Further qualitative research could pursue the consequences of such adaptive usage.

    Conclusions

    This study has applied a sociological lens (specifically, an empirically oriented adaptation of Giddens’ structuration theory), as well as the recently-published NASSS framework to a real-world empirical study of video outpatient consultations across three contrasting clinical specialties.

    We found that these consultations appear safe, effective, and convenient for patients in situations where participating clinicians judge them clinically appropriate; however, such patients are a fraction of the overall clinic workload. As with other technological innovations, some clinicians will adopt video consultations readily, whereas others will need incentives and support. There are complex challenges to embedding video consultation services within routine practice in health care organizations that are hesitant to change, especially at a time of austerity.

    Acknowledgments

    The study was funded by NIHR Health Services and Delivery Research programme (HS&DR-13/59/26); TG and SS’s input was also part-funded by the NIHR Biomedical Research Centre, Oxford, grant BRC-1215-20008 to the Oxford University Hospitals NHS Foundation Trust and the University of Oxford. The authors would like to thank Barts Health NHS Trust and all of the staff, patients, and carers who have given up their time to support and/or participate in the VOCAL study. Without them the research study would not have been possible. They also thank the members of their external steering group, especially Mrs Jenni Bowley who chaired the group, and the members of our Patient Advisory Group. The authors also thank Charles Gutteridge (Chief Clinical Information Officer at Barts Health) and Emma Byrne (previously at Queen Mary University of London) who helped to shape the initial proposal and the early stages of the project and Susan Larson from RIASWorks for providing training in the RIAS approach to data analysis. Several peer reviewers provided helpful comments that improved the paper.

    Authors' Contributions

    SV had the original idea for introducing virtual consultations at Barts Health and established the service as a pilot in 2010. TG was the chief investigator of the VOCAL study and (as such) its guarantor. She was involved in all aspects of study design, data collection and analysis, and writing; she drafted the first version of the paper. SS and JW are senior academics; they were involved in all aspects of the research. SS led on the macro element of the study and produced the full project report for NIHR. JW led on ethical applications and on the meso and micro elements of the study. At Barts Health, SB, SV, and PH are all consultants; DCR and SV are nurses; and JM is a senior project manager; all were involved in refining the study design, facilitating access to the study sites, data collection, and patient engagement. AC, a community anthropologist, led the patient advisory group. IH, a general practitioner, was the lead contact with local clinical commissioning groups. All authors contributed to dissemination, report writing, and publications, and all have seen and approved the final manuscript.

    Conflicts of Interest

    None declared.

    Multimedia Appendix 1

    Final report of Health Foundation DREAMS study (2014): Diabetes Review, Engagement and Management via Skype.

    PDF File (Adobe PDF File), 539KB

    Multimedia Appendix 2

    Supplementary material.

    PDF File (Adobe PDF File), 368KB

    Multimedia Appendix 3

    Skype guidance for clinicians and managers.

    PDF File (Adobe PDF File), 233KB

    References

    1. QualityWatch. Quality at a cost: QualityWatch annual statement 2016   URL: http:/​/www.​qualitywatch.org.uk/​sites/​files/​qualitywatch/​field/​field_document/​QW%20annual%20statement%202016%20%28final%29%20WEB.​pdf [accessed 2018-03-31] [WebCite Cache]
    2. NHS England. Five Year Forward View   URL: https://www.england.nhs.uk/wp-content/uploads/2014/10/5yfv-web.pdf [accessed 2018-03-31] [WebCite Cache]
    3. Gerlach UA, Vrakas G, Holdaway L, O'Connor M, Macedo R, Reddy S, et al. Skype clinics after intestinal transplantation - follow-up beyond post codes. Clin Transplant 2016 Jul;30(7):760-766. [CrossRef] [Medline]
    4. Campbell-Richards D. Exploring diabetes non-attendance: an Inner London perspective. J Diabetes Nurs 2016;20(2):73-78.
    5. de Oliviera C, Bremner KE, Ni A, Alibhai SM, Laporte A, Krahn MD. Patient time and out-of-pocket costs for long-term prostate cancer survivors in Ontario, Canada. J Cancer Surviv 2014 Mar;8(1):9-20. [CrossRef] [Medline]
    6. The King's Fund. A Digital NHS? An introduction to the digital agenda and plans for implementation   URL: https:/​/www.​kingsfund.org.uk/​sites/​default/​files/​field/​field_publication_file/​A_digital_NHS_Kings_Fund_Sep_2016.​pdf [accessed 2018-03-31] [WebCite Cache]
    7. UK Government. Making IT Work: Harnessing the Power of Health Information Technology to Improve Care in England   URL: https:/​/www.​gov.uk/​government/​uploads/​system/​uploads/​attachment_data/​file/​550866/​Wachter_Review_Accessible.​pdf [accessed 2018-03-31] [WebCite Cache]
    8. National Information Board. 2014. Personalised Health and Care 2020: Using Data and Technology to Transform Outcomes for Patients and Citizens   URL: https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/384650/NIB_Report.pdf [accessed 2018-03-31] [WebCite Cache]
    9. Department of Health, UK Government. 2012. Digital strategy: Leading the culture change in health and care   URL: https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/213222/final-report1.pdf [accessed 2018-03-31] [WebCite Cache]
    10. DigitalHealth. Safe, seamless and secure: evolving health and care to meet the needs of modern Australia   URL: https:/​/www.​digitalhealth.gov.au/​about-the-agency/​publications/​australias-national-digital-health-strategy/​ADHA-strategy-doc-(2ndAug).​pdf [accessed 2018-03-31] [WebCite Cache]
    11. Canada Health Infoway. 2013. Opportunities for Action: A Pan-Canadian Digital Health Strategic Plan   URL: https:/​/www.​infoway-inforoute.ca/​en/​component/​edocman/​resources/​i-infoway-i-corporate/​vision/​1658-opportunities-for-action-a-pan-canadian-digital-health-strategic-plan [accessed 2018-03-31] [WebCite Cache]
    12. European Commission. 2012. Communication from the Commission to the European Parliament, The Council, The European Economic and Social Committee and the Committee of the Regions: eHealth Action Plan 2012-2020 - Innovative healthcare for the 21st century   URL: https://ec.europa.eu/health/sites/health/files/ehealth/docs/com_2012_736_en.pdf [accessed 2018-03-31] [WebCite Cache]
    13. Armfield NR, Bradford M, Bradford NK. The clinical use of Skype--For which patients, with which problems and in which settings? A snapshot review of the literature. Int J Med Inform 2015 Oct;84(10):737-742. [CrossRef] [Medline]
    14. Greenhalgh T, Vijayaraghavan S, Wherton J, Shaw S, Byrne E, Campbell-Richards D, et al. Virtual online consultations: advantages and limitations (VOCAL) study. BMJ Open 2016 Jan 29;6(1):e009388. [CrossRef]
    15. Gentles SJ, Lokker C, McKibbon KA. Health information technology to facilitate communication involving health care providers, caregivers, and pediatric patients: a scoping review. J Med Internet Res 2010;12(2):e22. [CrossRef]
    16. Freeman KA, Duke DC, Harris MA. Behavioral health care for adolescents with poorly controlled diabetes via Skype: does working alliance remain intact? J Diabetes Sci Technol 2013 May 1;7(3):727-735 [FREE Full text] [CrossRef] [Medline]
    17. Hansen CR, Perrild H, Koefoed BG, Zander M. Video consultations as add-on to standard care among patients with type 2 diabetes not responding to standard regimens: a randomized controlled trial. Eur J Endocrinol 2017 Jun;176(6):727-736. [CrossRef] [Medline]
    18. Harris MA, Freeman KA, Duke DC. Seeing is believing: using Skype to improve diabetes outcomes in youth. Diabetes Care 2015;38(8):1427-1434. [CrossRef]
    19. Katz IJ, Pirabhahar S, Williamson P, Raghunath V, Brennan F, O'Sullivan A, et al. iConnect CKD - Virtual Medical Consulting: a web-based chronic kidney disease, hypertension and diabetes integrated care program. Nephrology (Carlton) 2017 May 4:-. [CrossRef]
    20. Nield M, Hoo GW. Real-time telehealth for COPD self-management using Skype™. COPD 2012 Dec;9(6):611-619. [CrossRef] [Medline]
    21. Mark D, Ikehara C, Matsuura C, Hara K, Li D. Validating the impact of teaching pursed-lips breathing with Skype: a pilot study. J Hosp Palliat Nurs 2013;15(8):424-432. [CrossRef]
    22. Choi NG, Hegel MT, Marti N, Marinucci ML, Sirrianni L, Bruce ML. Telehealth problem-solving therapy for depressed low-income homebound older adults. Am J Geriatr Psychiatry 2014 Mar;22(3):263-271 [FREE Full text] [CrossRef] [Medline]
    23. Chavooshi B, Mohammadkhani P, Dolatshahi B. A randomized double-blind controlled trial comparing Davanloo intensive short-term dynamic psychotherapy as internet-delivered vs treatment as usual for medically unexplained pain: a 6-month pilot study. Psychosomatics 2016;57(3):292-300. [CrossRef] [Medline]
    24. Bennell KL, Nelligan R, Dobson F, Rini C, Keefe F, Kasza J, et al. Effectiveness of an internet-delivered exercise and pain-coping skills training intervention for persons with chronic knee pain: a randomized trial. Ann Intern Med 2017 Apr 4;166(7):453-462. [CrossRef] [Medline]
    25. Somers TJ, Kelleher SA, Westbrook KW, Kimmick GG, Shelby RA, Abernethy AP, et al. A small randomized controlled pilot trial comparing mobile and traditional pain coping skills training protocols for cancer patients with pain. Pain Res Treat 2016;2016:2473629 [FREE Full text] [CrossRef] [Medline]
    26. Müller KI, Alstadhaug KB, Bekkelund SI. Acceptability, feasibility, and cost of telemedicine for nonacute headaches: a randomized study comparing video and traditional consultations. J Med Internet Res 2016 May 30;18(5):e140 [FREE Full text] [CrossRef] [Medline]
    27. Gund A, Sjöqvist BA, Wigert H, Hentz E, Lindecrantz K, Bry K. A randomized controlled study about the use of eHealth in the home health care of premature infants. BMC Med Inform Decis Mak 2013 Feb 9;13:22 [FREE Full text] [CrossRef] [Medline]
    28. Orlandoni P, Jukic PN, Spazzafumo L, Venturini C, Cola C, Sparvoli D, et al. Utility of video consultation to improve the outcomes of home enteral nutrition in a population of frail older patients. Geriatr Gerontol Int 2016 Jun;16(6):762-767. [CrossRef] [Medline]
    29. Sharareh B, Schwarzkopf R. Effectiveness of telemedical applications in postoperative follow-up after total joint arthroplasty. J Arthroplasty 2014 May;29(5):918-22.e1. [CrossRef]
    30. Marsh J, Hoch JS, Bryant D, MacDonald SJ, Naudie D, McCalden R, et al. Economic evaluation of web-based compared with in-person follow-up after total joint arthroplasty. J Bone Joint Surg Am 2014 Nov 19;96(22):1910-1916. [CrossRef] [Medline]
    31. Marsh JD, Bryant DM, MacDonald SJ, Naudie DD, McCalden RW, Howard JL, et al. Feasibility, effectiveness and costs associated with a web-based follow-up assessment following total joint arthroplasty. J Arthroplasty 2014 Sep;29(9):1723-1728. [CrossRef] [Medline]
    32. Buvik A, Bugge E, Knutsen G, Småbrekke A, Wilsgaard T. Quality of care for remote orthopaedic consultations using telemedicine: a randomised controlled trial. BMC Health Serv Res 2016;16:483 [FREE Full text] [CrossRef] [Medline]
    33. Westra I, Niessen FB. Implementing real-time video consultation in plastic surgery. Aesthetic Plast Surg 2015 Oct;39(5):783-790. [CrossRef] [Medline]
    34. Viers BR, Lightner DJ, Rivera ME, Tollefson MK, Boorjian SA, Karnes RJ, et al. Efficiency, satisfaction, and costs for remote video visits following radical prostatectomy: a randomized controlled trial. Eur Urol 2015;68(4):729-735. [CrossRef]
    35. Zanaboni P, Wootton R. Adoption of routine telemedicine in Norwegian hospitals: progress over 5 years. BMC Health Serv Res 2016 Dec 20;16:496 [FREE Full text] [CrossRef] [Medline]
    36. Adler-Milstein J, Kvedar J, Bates DW. Telehealth among US hospitals: several factors, including state reimbursement and licensure policies, influence adoption. Health Aff (Millwood) 2014 Feb;33(2):207-215. [CrossRef] [Medline]
    37. Greenhalgh T, Wherton J, Papoutsi C, Lynch J, Hughes G, A'Court C, et al. Beyond adoption: a new framework for theorizing and evaluating nonadoption, abandonment, and challenges to the scale-up, spread, and sustainability of health and care technologies. J Med Internet Res 2017 Nov 1;19(11):e367 [FREE Full text] [CrossRef] [Medline]
    38. Finch T, May C, Mair F, Mort M, Gask L. Integrating service development with evaluation in telehealthcare: an ethnographic study. BMJ 2003;327:1205. [CrossRef]
    39. May C, Mort M, Williams T, Mair F, Gask L. Health technology assessment in its local contexts: studies of telehealthcare. Soc Sci Med 2003 Aug;57(4):697-710. [Medline]
    40. Chakrabarti S. Usefulness of telepsychiatry: a critical evaluation of videoconferencing-based approaches. World J Psychiatry 2015;5(3):286-304. [CrossRef]
    41. Krout RE, Baker F, Muhlberger R. Designing, piloting, and evaluating an on-line collaborative songwriting environment and protocol using Skype telecommunication technology: perceptions of music therapy student participants. Music Ther Perspect 2010;28(1):79-85. [CrossRef]
    42. Baker DC, Bufka LF. Preparing for the telehealth world: navigating legal, regulatory, reimbursement, and ethical issues in an electronic age. Prof Psychol Res Pr 2011;42(6):405-411. [CrossRef]
    43. Caffery LJ, Smith AC. Investigating the quality of video consultations performed using fourth generation (4G) mobile telecommunications. J Telemed Telecare 2015 Mar 11;6:348-354. [CrossRef] [Medline]
    44. van Eck CF. Web-based follow-up after total joint arthroplasty proves to be cost-effective, but is it safe? commentary on an article by Jacquelyn Marsh, PhD, et al.: “Economic evaluation of web-based compared with in-person follow-up after total joint arthroplasty”. J Bone Joint Surg Am 2014;96(22):e192. [CrossRef]
    45. Arminen I, Licoppe C, Spagnolli A. Respecifying mediated interaction. Res Lang Soc Interact 2016;49(4):290-309. [CrossRef]
    46. Kahn JM. Virtual visits--confronting the challenges of telemedicine. N Engl J Med 2015 Apr 30;372(18):1684-1685. [CrossRef] [Medline]
    47. Newham University NHS Trust. London: Health Foundation; 2012. Diabetes Appointments via Webcam in Newham: Final Report for Shine   URL: http://www.health.org.uk/sites/health/files/Shine2011_DAWN_poster.pdf [accessed 2018-04-01] [WebCite Cache]
    48. Vijayaraghavan S, O'Shea T, Campbell-Richards D, Sudra R, Morris J, Byrne E, et al. DAWN: Diabetes Appointments via Webcam in Newham. Br J Diabetes Vasc Dis 2015;15(3):123-126. [CrossRef]
    49. Brown MJ, Shaw NT, Mador RL. Mapping the sociotechnical healthcare ecosystem: expanding the horizons of sociotechnical inquiry. AMIA Annu Symp Proc 2008:1233-1235. [Medline]
    50. Barley SR, Kunda G. Bringing work back in. Organ Sci 2001;12(1):76-95. [CrossRef]
    51. Greenhalgh T, Stones R. Theorising big IT programmes in healthcare: strong structuration theory meets actor-network theory. Soc Sci Med 2010 May;70(9):1285-1294. [CrossRef] [Medline]
    52. Waterman H, Tillen D, Dickson R, de Koning K. Action research: a systematic review and guidance for assessment. Health Technol Assess 2001;5(23):iii-157 [FREE Full text] [Medline]
    53. Bate P. Synthesizing research and practice: using the action research approach in health care settings. Soc Policy Adm 2000;34(4):478-493. [CrossRef]
    54. Wagenaar H. Meaning in Action: Interpretation and Dialogue in Policy Analysis. New York: M. E. Sharpe; 2011.
    55. Yanow D. How Does a Policy Mean? Interpreting Policy and Organisational Actions. Washington, DC: Georgetown University Press; 1996.
    56. Hajer MA. Doing discourse analysis: coalitions, practices, meaning. In: Van der Brink M, Metze T, editors. Words Matter in Policy and Planning: Discourse Theory and Method in the Social Sciences. Utrecht: Netherlands Graduate School of Urban and Regional Research; 2006:65-74.
    57. Feldman MS. A performative perspective on stability and change in organizational routines. Ind Corp Change 2003;12(4):727-752. [CrossRef]
    58. Feldman MS, Pentland BT. Reconceptualizing organizational routines as a source of flexibility and change. Adm Sci Q 2003;48(1):94-118. [CrossRef]
    59. Feldman MS, Pentland BT, D’Adderio L, Lazaric N. Beyond routines as things: introduction to the special issue on routine dynamics. Organ Sci 2016;27(3):505-513. [CrossRef]
    60. Roter D, Larson S. The Roter interaction analysis system (RIAS): utility and flexibility for analysis of medical interactions. Patient Educ Couns 2002 Apr;46(4):243-251. [Medline]
    61. Roter D, Hall JA. Doctors Talking With Patients/Patients Talking With Doctors: Improving Communication in Medical Visits. Westport, CT: Auburn House; 1992.
    62. Bales RF. Interaction Process Analysis. Cambridge: Addison-Wesley; 1950.
    63. Emerson R. Social exchange theory. Annu Rev Sociol 1976;2:335-362 [FREE Full text] [CrossRef]
    64. Cloud-Buckner J, Gallimore JJ, Scott R. Theoretical discussion of CAMOCS and RIAS communication models to support healthcare information technology development. Proc Hum Factors Ergon Soc Annu Meet 2010;54(5):517-521. [CrossRef]
    65. Wakefield BJ, Bylund CL, Holman JE, Ray A, Scherubel M, Kienzle MG, et al. Nurse and patient communication profiles in a home-based telehealth intervention for heart failure management. Patient Educ Couns 2008 May;71(2):285-292. [CrossRef] [Medline]
    66. Nelson E, Spaulding R. Adapting the Roter interaction analysis system for telemedicine: lessons from four specialty clinics. J Telemed Telecare 2005;11 Suppl 1:105-107. [CrossRef] [Medline]
    67. Agha Z, Roter DL, Schapira RM. An evaluation of patient-physician communication style during telemedicine consultations. J Med Internet Res 2009 Sep 30;11(3):e36 [FREE Full text] [CrossRef] [Medline]
    68. Miller EA. The continuing need to investigate the nature and content of teleconsultation communication using interaction analysis techniques. J Telemed Telecare 2011;17(2):55-64. [CrossRef] [Medline]
    69. UK Government. Information Economy Strategy   URL: https:/​/www.​gov.uk/​government/​uploads/​system/​uploads/​attachment_data/​file/​206944/​13-901-information-economy-strategy.​pdf [accessed 2018-03-31] [WebCite Cache]
    70. UK Government. 2014. Government Digital Inclusion Strategy   URL: https://www.gov.uk/government/publications/government-digital-inclusion-strategy [accessed 2018-03-31] [WebCite Cache]
    71. UK Government. 2014. UK Digital Inclusion Charter   URL: https:/​/www.​gov.uk/​government/​publications/​government-digital-inclusion-strategy/​uk-digital-inclusion-charter [accessed 2018-03-31] [WebCite Cache]
    72. UK Government. 2016. Digital Economy Bill Overview   URL: https://www.gov.uk/government/publications/digital-economy-bill-overview [accessed 2018-03-31] [WebCite Cache]
    73. UK Government. 2016. Government’s first Director General for Digital and Media to help turn Britain into world-leading digital economy   URL: https:/​/www.​gov.uk/​government/​news/​governments-first-director-general-for-digital-and-media-to-help-turn-britain-into-world-leading-digital-economy [accessed 2018-03-31] [WebCite Cache]
    74. Fox M. London: Cabinet Office, Efficiency and Reform Group, and Government Digital Service; 2010. Directgov 2010 and beyond: revolution not evolution   URL: https:/​/www.​gov.uk/​government/​uploads/​system/​uploads/​attachment_data/​file/​60993/​Martha_20Lane_20Fox_s_20letter_20to_20Francis_20Maude_2014th_20Oct_202010.​pdf [accessed 2018-04-01] [WebCite Cache]
    75. UK Government. 2012. Digital Landscape Research   URL: https://www.gov.uk/government/publications/digital-landscape-research [accessed 2018-03-31] [WebCite Cache]
    76. UK Government. 2011. Government Digital Service opens for business   URL: https://www.gov.uk/government/news/government-digital-service-opens-for-business [accessed 2018-03-31] [WebCite Cache]
    77. UK Government. Government Transformation Strategy 2017 to 2020   URL: https:/​/www.​gov.uk/​government/​uploads/​system/​uploads/​attachment_data/​file/​590199/​Government_Transformation_Strategy.​pdf [accessed 2018-03-31] [WebCite Cache]
    78. House of Commons Health Committee. London: The Stationery Office; 2005. The Use of New Medical Technologies within the NHS   URL: https://publications.parliament.uk/pa/cm200405/cmselect/cmhealth/398/398i.pdf [accessed 2018-04-01] [WebCite Cache]
    79. UK Government. 2008. High Quality Care for All   URL: https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/228836/7432.pdf [accessed 2018-03-31] [WebCite Cache]
    80. UK Government. 2012. Civil Service Reform Plan   URL: https://www.gov.uk/government/publications/civil-service-reform-plan [accessed 2018-03-31] [WebCite Cache]
    81. UK Government. 2016. Accelerated Access Review: Final Report. Review of innovative medicines and medical technologies   URL: https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/565072/AAR_final.pdf [accessed 2018-03-31] [WebCite Cache]
    82. UK Government. 2015. Digital Health in the UK: An industry study for the Office of Life Sciences   URL: https:/​/www.​gov.uk/​government/​uploads/​system/​uploads/​attachment_data/​file/​461479/​BIS-15-544-digital-health-in-the-uk-an-industry-study-for-the-Office-of-Life-Sciences.​pdf [accessed 2018-03-31] [WebCite Cache]
    83. UK Government. 2016. Office for Life Sciences: a How To guide. A guide to navigating the innovation pathway in England   URL: https:/​/www.​gov.uk/​government/​uploads/​system/​uploads/​attachment_data/​file/​525787/​AAR_how_to_guides.​pdf [accessed 2018-03-31] [WebCite Cache]
    84. NHS England. London: NHS England; 2016. NHS England announces first medtech devices and apps to join fast track payment scheme for innovation   URL: https://www.england.nhs.uk/2016/11/innov-tech-tariff/ [accessed 2018-04-01] [WebCite Cache]
    85. NHS England. Securing Excellence in GP IT Services: 2016-18 Operating Model   URL: https:/​/www.​england.nhs.uk/​digitaltechnology/​wp-content/​uploads/​sites/​31/​2015/​04/​gp-it-operating-model-16-18.​pdf [accessed 2018-03-31] [WebCite Cache]
    86. NHS England. 2016. General Practice Forward View   URL: https://www.england.nhs.uk/wp-content/uploads/2016/04/gpfv.pdf [accessed 2018-03-31] [WebCite Cache]
    87. NHS England. 2016. New Care Models: Vanguards - developing a blueprint for the future of NHS and care services   URL: https://www.england.nhs.uk/wp-content/uploads/2015/11/new_care_models.pdf [accessed 2018-03-31] [WebCite Cache]
    88. NHS England. 2016. 12 NHS Hospital trusts to trail blaze NHS’s Digital Revolution   URL: https://www.england.nhs.uk/2016/09/digital-revolution/ [accessed 2018-03-31] [WebCite Cache]
    89. Nuffield Trust. 2017. NHS funding choices and the 2017 General Election   URL: https://www.nuffieldtrust.org.uk/research/nhs-funding-choices-and-the-2017-general-election [accessed 2018-03-31] [WebCite Cache]
    90. The King's Fund. 2017. Sicker patients the main reason for A&E winter pressures   URL: https://qmr.kingsfund.org.uk/2017/22/ [accessed 2018-03-31] [WebCite Cache]
    91. The King's Fund. 2017. Half of NHS areas planning to cancel or delay spending because of financial pressures   URL: https:/​/www.​kingsfund.org.uk/​press/​press-releases/​half-nhs-areas-planning-cancel-or-delay-spending-because-financial-pressures [accessed 2018-03-31] [WebCite Cache]
    92. Appleby J. 2015. NHS Spending: squeezed as never before   URL: https://www.kingsfund.org.uk/blog/2015/10/nhs-spending-squeezed-never [accessed 2018-03-31] [WebCite Cache]
    93. NHS England. Innovation into Action: Supporting delivery of the NHS Five Year Forward View   URL: https://www.england.nhs.uk/wp-content/uploads/2015/10/nhs-inovation-into-action.pdf [accessed 2018-03-31] [WebCite Cache]
    94. National Archives. 2011. Innovation, health and wealth: Accelerating adoption and diffusion in the NHS   URL: http:/​/webarchive.​nationalarchives.gov.uk/​20130107070708/​http:/​www.dh.gov.uk/​prod_consum_dh/​groups/​dh_digitalassets/​documents/​digitalasset/​dh_134597.​pdf [accessed 2018-03-31] [WebCite Cache]
    95. ONS. Internet users in the UK: 2017   URL: https://www.ons.gov.uk/businessindustryandtrade/itandinternetindustry/bulletins/internetusers/2017 [accessed 2018-03-20] [WebCite Cache]
    96. UK Government. Digital inclusion   URL: https://www.gov.uk/performance/digital-inclusion [accessed 2018-03-20] [WebCite Cache]
    97. Tinder Foundation. 2016. Health & Digital: Reducing Inequalities, Improving Society. An evaluation of the Widening Digital Participation programme   URL: http:/​/nhs.​tinderfoundation.org/​wp-content/​uploads/​2016/​07/​Improving_Digital_Health_Skills_Report_2016.​pdf [accessed 2018-03-31] [WebCite Cache]
    98. NHS England. The NHS England Innovation and Technology Tariff 2017 to 2019 Technical notes   URL: https://www.england.nhs.uk/wp-content/uploads/2017/05/innovation-tech-tariff-technical-notes.pdf [accessed 2018-03-20] [WebCite Cache]
    99. UK Government. 2014/15 National Tariff Payment System   URL: https:/​/www.​gov.uk/​government/​uploads/​system/​uploads/​attachment_data/​file/​300547/​2014-15_National_Tariff_Payment_System_-Revised_26_Feb_14.​pdf [accessed 2018-03-31] [WebCite Cache]
    100. Charlesworth A, Hawkins L, Marshall L. 2014. NHS payment reform: lessons from the past and directions for the future   URL: https://www.nuffieldtrust.org.uk/files/2017-01/2014-payment-reform-policy-response-web-final.pdf [accessed 2018-03-31] [WebCite Cache]
    101. Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q 2004;82(4):581-629 [FREE Full text] [CrossRef] [Medline]


    Abbreviations

    ICT: information and communication technology
    IGA: Information Governance Alliance
    IT: information technology
    IQR: interquartile range
    NASSS: nonadoption, abandonment, and failure of scale-up, spread, and sustainability
    NHS: National Health Service
    RCT: randomized controlled trial
    RIAS: Roter interaction analysis system
    USB: Universal Serial Bus


    Edited by G Eysenbach; submitted 20.01.18; peer-reviewed by C May, S Hinder, T Schopf, H Yu; comments to author 08.02.18; revised version received 19.02.18; accepted 20.02.18; published 17.04.18

    ©Trisha Greenhalgh, Sara Shaw, Joseph Wherton, Shanti Vijayaraghavan, Joanne Morris, Satya Bhattacharya, Philippa Hanson, Desirée Campbell-Richards, Seendy Ramoutar, Anna Collard, Isabel Hodkinson. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 17.04.2018.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.