This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
Many promising technological innovations in health and social care are characterized by nonadoption or abandonment by individuals or by failed attempts to scale up locally, spread distantly, or sustain the innovation long term at the organization or system level.
Our objective was to produce an evidence-based, theory-informed, and pragmatic framework to help predict and evaluate the success of a technology-supported health or social care program.
The study had 2 parallel components: (1) secondary research (hermeneutic systematic review) to identify key domains, and (2) empirical case studies of technology implementation to explore, test, and refine these domains. We studied 6 technology-supported programs—video outpatient consultations, global positioning system tracking for cognitive impairment, pendant alarm services, remote biomarker monitoring for heart failure, care organizing software, and integrated case management via data sharing—using longitudinal ethnography and action research for up to 3 years across more than 20 organizations. Data were collected at micro level (individual technology users), meso level (organizational processes and systems), and macro level (national policy and wider context). Analysis and synthesis was aided by sociotechnically informed theories of individual, organizational, and system change. The draft framework was shared with colleagues who were introducing or evaluating other technology-supported health or care programs and refined in response to feedback.
The literature review identified 28 previous technology implementation frameworks, of which 14 had taken a dynamic systems approach (including 2 integrative reviews of previous work). Our empirical dataset consisted of over 400 hours of ethnographic observation, 165 semistructured interviews, and 200 documents. The final nonadoption, abandonment, scale-up, spread, and sustainability (NASSS) framework included questions in 7 domains: the condition or illness, the technology, the value proposition, the adopter system (comprising professional staff, patient, and lay caregivers), the organization(s), the wider (institutional and societal) context, and the interaction and mutual adaptation between all these domains over time. Our empirical case studies raised a variety of challenges across all 7 domains, each classified as simple (straightforward, predictable, few components), complicated (multiple interacting components or issues), or complex (dynamic, unpredictable, not easily disaggregated into constituent components). Programs characterized by complicatedness proved difficult but not impossible to implement. Those characterized by complexity in multiple NASSS domains rarely, if ever, became mainstreamed. The framework showed promise when applied (both prospectively and retrospectively) to other programs.
Subject to further empirical testing, NASSS could be applied across a range of technological innovations in health and social care. It has several potential uses: (1) to inform the design of a new technology; (2) to identify technological solutions that (perhaps despite policy or industry enthusiasm) have a limited chance of achieving large-scale, sustained adoption; (3) to plan the implementation, scale-up, or rollout of a technology program; and (4) to explain and learn from program failures.
In 2004 and 2005, Greenhalgh et al published a multilevel framework for studying diffusion of innovations in health care, based on a cross-disciplinary systematic literature review [
An update of that review in 2010 focused explicitly on organizational-level adoption and mainstreaming of technological innovations [
In recent years, technological innovation has moved apace and is now widely viewed as a significant potential contributor to health and wealth [
These problems are illustrated by the paradox of telehealth (a term with contested definitions [
Poor uptake of technological innovations is often explained in terms of barriers and facilitators. In a recent review of telehealth in heart failure, for example, we identified technology barriers, patient barriers, staff barriers, team barriers, business and financial barriers, and governance and regulatory barriers [
As our 2004 review of diffusion of innovations found, it is not individual factors that make or break a technology implementation effort but the
Among others, Lupton [
We aimed to produce an evidence-based, theory-informed, but also accessible and usable framework that would enable those seeking to design, develop, implement, scale up, spread, and sustain technology-supported health or social care programs to identify and help address the key challenges in different domains and the interactions between them.
We selected a diverse sample of case studies from 2 research programs: Virtual Online Consultations: Advantages and Limitations (VOCAL) and Studies in Co-creating Assisted Living Solutions (SCALS), whose detailed methodology and ethical approval have been described elsewhere [
Case studies in VOCAL and SCALS involved qualitative interviews (with patients, clinicians, managers, technical designers, commercial partners, and—where relevant—investors), analysis of documents (correspondence, business plans, clinical records), ethnography (of technology use by patients or clients and staff, of meetings and events, and of technology design and functionality), and video recording of both ends of remote consultations [
Study flowchart.
Summary of data sources used in this analysis.
Study site or sites | Technology or technologies | Participants | Data sources | |
1A. Acute hospital trust (3 specialties—diabetes, prenatal diabetes, cancer—on different sites) |
Skype (acute hospital) and FaceTime (community hospital) | 1A. 24 staff (9 clinicians, 10 support staff, 5 managers); 30 patients |
35 formal semistructured interviews plus ~100 informal interviews; ≥150 hours of ethnographic observation; 40 videotaped remote consultations (12 diabetes, 6 prenatal diabetes, 12 cancer, 10 heart failure); ≥500 emails; 30 local documents such as business plans, protocols; 50 national-level documents | |
2A. Social care organization in deprived borough in inner London, UK | GPS tracking devices supplied by 5 different technology companies, includes GPS tracking with virtual map and geofence alert functions | 7 index cases; 8 lay caregivers; 5 formal caregivers; 3 social care staff; 3 health care staff; 3 call center staff | 22 ethnographic visits and “go-along” interviews with index cases (~50 hours); 15 ethnographic visits with health and social care staff; 6 staff interviews; 5 team meetings; 3 local protocols | |
3A. Health care commissioning organization in deprived borough in outer London, UK |
In both sites, pendant alarms and base units were supplied by multiple different technology companies and supported by local councils, each with a different set of arrangements with providers and an “arms-length management organization” alarm support service | Site 3A. 8 index cases; 7 lay caregivers; 12 professional staff |
50 semistructured and narrative interviews; 61 ethnographic visits (~80 hours of observation) including needs assessments and reviews; 20 hours of observation at team meetings | |
Acute hospital trusts in 6 different cities in United Kingdom | Tablet computer and commercially available sensing devices (blood pressure monitor, weighing scales, pulse oximeter) | 7 research staff, including principal investigator and research coordinator for SUPPORT-HFb trial; 7 clinical staff involved in trial; 4 clinical staff not involved in trial; (to date) 18 patient participants and 1 spouse | 1 patient focus group; 8 patient interviews; 24 additional semistructured interviews; SUPPORT-HF study protocol and ethics paperwork; material properties and functionality of biomarker database | |
5A. Health care commissioning organization in northern England |
5A. Web-based portal developed by small technology company for use by families to help them organize and coordinate the care of (typically) an older relative |
Product A: 2 technology developers and CEOc of technology company; 4 social care commissioners; 30 health and social care staff considering using the device; 4 users of the device, 1 nonuser. |
22 semistructured and narrative interviews; 16 hours’ ethnographic observations of meetings; autoethnographic testing of functionality and usability of devices; secondary analysis of third-party evaluation of Product B | |
1 acute hospital trust, 1 community health trust, 3 local councils, 3 health care commissioning organizations | Integrated data warehouse incorporating predictive risk modeling (in theory interoperable with record systems in participating organizations) | 14 staff; 20 patient participants | 14 semistructured interviews; 50 ethnographic visits (~80 hours); 12 hours’ shadowing community staff; 4 hours’ observation of interdisciplinary meetings; 12 local protocols or documents |
aGPS: global positioning system.
bSUPPORT-HF: Seamless User-Centred Proactive Provision of Risk-Stratified Treatment for Heart Failure.
cCEO: chief executive officer.
Articles describing technology implementation frameworks and their applications were eligible if they (1) studied a technology that was perceived as new by intended users, (2) aimed (through the uptake and use of the technology) to improve service efficiency, or patient or client outcomes in health or social care; and (3) offered some kind of conceptual or theoretical framework. We were particularly interested in patient-facing technologies such as telehealth, but we also assessed other frameworks (eg, for health information systems) for transferable insights.
We began by selecting relevant studies from our hermeneutic literature review of telehealth in heart failure (covering 32 previous systematic reviews and 60 additional articles, including many that covered conditions beyond heart failure) [
Having obtained few hits, we extended our search to the wider literature by tracking our original 2004 diffusion of innovations review [
We used a simple data extraction form to summarize key aspects of each study (both theoretical and empirical). Using the hermeneutic (interpretive) methodology described in detail previously [
All 6 case studies generated large amounts of qualitative and quantitative data, not all of which was relevant to the objective of this study. Our first task was to delineate a more focused dataset of individual index cases (patients or clients) along with relevant staff interviews, field notes, and background documents (
We developed initial versions of the nonadoption, abandonment, scale-up, spread, and sustainability (NASSS) diagram and framework part way through the empirical work to guide our action research and inform cross-case theorization. As we applied the framework to real cases, challenges occurred that were not covered by it (eg, patients unable to use technologies because of comorbidities), so we searched more specifically for articles to inform additional domains and questions. We shared a near-final version of the framework with colleagues involved in 10 further large-scale technology-supported change programs (including email and video consultations in primary care; an online peer support network for people with mental health needs; remote biomarker monitoring in transplant patients; and an online tool for people to identify local services appropriate to their health and care needs); we further refined the framework in the light of their feedback.
This case included 4 clinical services: 3 hospital-based services from VOCAL (young adult diabetes, prenatal diabetes, and cancer surgery, all using Skype; SS, unpublished data, 2017) and 1 community-based from SCALS (a nurse-led heart failure service run from 4 community hospitals, using predominantly FaceTime). In each, patients judged “appropriate” for video consultations by the doctor or nurse were offered this option. National policy makers viewed video consulting as a way of delivering health care efficiently to an aging population with rising rates of chronic illness. But the reality of establishing such services in busy and financially stretched public sector organizations proved far more complex and difficult than anticipated; progress was slow and required multiple organizational workarounds. Technical challenges in setting up video consultations with patients were typically mundane but potentially prohibitive (eg, forgotten passwords, poor connectivity, outdated software). When clinical, technical, and practical preconditions were met, video consultations appeared safe and were popular with both patients and staff, although only some clinicians agreed to participate.
By the end of the study period, video consultations had been abandoned in the prenatal diabetes service and put on hold in the community heart failure service, but the young adult diabetes and cancer surgery services were conducting around 20% of follow-up consultations remotely. In the (extremely busy) prenatal diabetes clinic, video consultations aligned poorly with a context involving multidisciplinary teams (patients were typically seeing multiple clinicians across departments) for a relatively short-term but high-risk condition and in the absence of integrated records (paper medical notes were held by the patient so not physically at hand for the clinician). In the heart failure clinic, the physical examination (eg, heart rhythm, leg edema) that the nurses considered essential was not easy in the remote environment (although sometimes possible with patient and caregiver assistance); multimorbidity and polypharmacy were common and in most cases the perceived risks and uncertainties associated with remote consulting were considered to outweigh the benefits.
Electronic tracking through GPS is used to monitor people with cognitive impairment who “wander” outside the home. We worked with a public sector social care organization to implement and adapt GPS tracking devices and a linked monitoring service for such individuals (of whom 11 were considered eligible and 7 assented). In what were typically very complex care contexts, GPS devices were useful to the extent that they aligned with a wider sociotechnical care network that included lay caregivers, call centers, and health and social care professionals. In this context, “safe” wandering was a collaborative accomplishment that depended on the technology’s material features, affordances, and aesthetic properties; a distributed knowledge of the individual and the places they wandered through; and a collective and dynamic interpretation of risk. Each index case required a high degree of tinkering (including customization of the device, liaison with the technology supplier, and adjustment of work routines) to achieve a solution that was acceptable. Despite this, only 3 individuals were still using the technology by the end of the 18-month study period.
Pendant alarms (worn around the neck or on a wrist strap and connected to a remote call center) were the only patient-facing technology in widespread use in our dataset. Both study sites had a well-established sociotechnical infrastructure that included a named care team with expertise and local knowledge. Supply of a pendant alarm was typically initiated by a public sector organization and involved a local technology supplier to fit it, with or without support from an age charity. Clients could also self-refer. The setup usually depended on a network of lay caregivers available to respond to a summons; a safe box was usually installed containing a key so the rescuer could let themselves in, or an emergency response (eg, ambulance or 24/7 social care) was summoned as required. Users paid a set-up fee (around £25; US $40) plus a small weekly support fee (around £4.50; US $7), although some local care providers offered this service free of charge (eg, to people in receipt of welfare benefits). In almost all cases, the individual had multiple and complex needs (physical, cognitive, social) and was using multiple technologies in addition to the alarm [
In many but not all cases, activation of the pendant alarm led to help arriving promptly. On some occasions, there was a mismatch: the alarm was triggered when there was no objective need (perhaps by accident) or, more commonly, not triggered when caregivers felt it should have been—because the individual did not want to trouble anyone, did not believe the problem was serious or urgent or was unable to activate the device (eg, during a fit), or was not wearing the alarm at the time of the crisis. Sometimes, call center operators made judgments and put in “emotional work” to support the caller without alerting their relative or reassured them while help was on its way [
This case study involved cardiology departments in 6 UK hospitals, each implementing biomarker monitoring (weight, blood pressure, heart rate) for heart failure as part of a multicenter randomized controlled trial (Seamless User-Centred Proactive Provision of Risk-Stratified Treatment for Heart Failure [SUPPORT-HF]). The tablet technology supplied to patient participants had been developed using a codesign methodology [
This case study followed the very different fortunes of 2 software products, each designed to help relatives and friends (and sometimes professional staff as well) organize tasks and visits for someone with health or care needs. Product A, a Web portal, had been developed in-house by a small software company, based on a previous caring experience by one of the company staff. The business model was to sell the product to care organizations who would then provide it to their clients for free. The developer did not initially anticipate that either intended end users or participating care organizations would need any training or ongoing support to use the portal. Product A was not successful during the study period; fewer than 5 families were ever identified as actively using it.
Product B was a smartphone app (with linked Web portal) that had been developed via publicly funded research and development using codesign methodology by a national caregivers’ charity. The charity had previously identified a need for such software; they worked with a specialist app developer company and carefully selected pilot families. From the outset, the charity recognized that caregivers would need to be made aware of the product through mass mailing and to be actively invited and supported to use it, and that a helpdesk service would be needed. The app was made available commercially (via the App Store) for £2.99 (about US $5). Users signed up gradually but steadily; there was no tipping point, but at the time of writing over 1000 families are using the product through the care charity; in a preliminary evaluation, most spoke highly of it (and of the charity support).
Case management is a way of organizing health and social care services through assessment and care planning by multidisciplinary teams with the aim of managing the growing challenge of emergency hospital admissions (and readmissions) in older people with multiple health and care needs. To avoid the high human and financial cost of such admissions, coordinated action and frequent dialogue between primary care providers, secondary care providers, and social care and other formal and informal caregivers is often needed. In the SCALS study, 1 site had introduced an integrated data warehouse incorporating a predictive risk modeling tool to automate the identification of people at high risk of hospital admissions through risk stratification, and to facilitate shared access to care plans in efforts to achieve integrated care. Although the data warehouse was part of business as usual in this site, in practice people at high risk of hospital admission were identified through a combination of risk stratification and clinical judgment. Care plans were shared in a range of ways, sometimes through the integrated data warehouse and sometimes bypassing it.
Our search for evidence-based approaches to guide our empirical work on the above case studies identified 28 technology implementation frameworks, informed by several theoretical perspectives, which we sorted into a simple taxonomy [
A key limitation of many previous frameworks was the lack of detailed analysis of the condition or problem for which a new technology was [part of] the intended solution. Some assumed a “textbook” condition—simple, isolated, easily characterized, and amenable to management by algorithm or protocol using a one-size-fits-all (or minimally customizable) technology. Yet there is much empirical evidence that the health and care needs of real people are extremely heterogeneous, even when they have the “same” condition. For example, Tait et al’s case-by-case analysis of patients in a heart failure clinic found that every one of them required significant customization and ongoing adaptation of the care package recommended in the guideline [
A prominent policy prediction, typically couched in the language of “empowerment,” is that remote technology will make care more efficient by encouraging self-management of chronic conditions [
No previous framework explicitly considered inequalities in access, uptake, and use of health and care technologies by age, sex, socioeconomic status, or ethnic group, although previous empirical studies have highlighted substantial differences across such groups [
Most previous frameworks addressed the material properties of the technology, such as its physical features, functionality, and interoperability. Few considered its symbolic properties (some technologies—such as mobile phones—have connotations of youth, progress, and friendship; others—such as GPS tracking devices or pendant alarms—symbolize dependence or external control). One or two frameworks considered what knowledge or skills (and hence training and support) were needed for intended users to be confident in operating the technology.
A question addressed tangentially or not at all by previous frameworks was
Very few frameworks in our sample included an assessment of whether a technology was likely to be
Lehoux et al distinguish between a health technology’s upstream value as viewed by investors (especially the business case for generating profits, further spin-offs, and highly qualified jobs), drug and device regulators (preliminary evidence of efficacy and safety), and financial regulators (auditable business processes and governance), and its downstream value as viewed by clinicians and policy makers (including its impact on patients and health care costs) [
In previous frameworks, technology adoption by health care staff was most commonly theorized using Davis’s technology acceptance model, comprising perceived usefulness, perceived ease of use, and attitude toward the technology [
In considering our own case studies, we were drawn to sociological theories because the new technologies often had implications for staff identity, professional commitments, and scope of practice. Acceptance by professional staff may be the single most important determinant of whether a new technology-supported service succeeds or fails at a local level [
One framework in our sample addressed technology acceptance by patients. The digital health engagement model was based on both burden of treatment theory and normalization process theory [
Two frameworks drew on DeLone and McLean’s classic theoretical model of information system success [
Surprisingly few frameworks considered the organizational setting. Some antecedent characteristics of organizations have been shown to support innovation at an organizational level [
Another important aspect of implementation omitted by most previous frameworks was the health or care organization’s business model for introducing the technology. This includes the resources to support the model, key partners and relationships, the transaction mechanism (how will the organization interact with the supplier?), the value structure (how and when will value, including benefits for patients, be created and investment costs recouped?), and organizational design issues (what changes in organizational structure and processes are required or presumed by the new technology?) [
Most, but not all, previous frameworks considered how a new technology would fit with existing organizational routines (defined as recurrent, collective patterns of interaction that both coordinate and control organizational work [
Work processes can be described in two ways: the way things are supposed to work and the way they do work. Software that is designed to support standard procedures can be too brittle.
Cherns’ classic theory of sociotechnical design, originally developed in the 1970s, is built on the principle that introducing technologies in an organization is a social process that depends on values, mindsets, and engagement, as well as on clear and extensive communication about what changes are occurring and why [
Innovation is not restricted to the prior design of an artefact, but continues as artefacts are implemented and used (innofusion). Supplier offerings are inevitably incomplete in relation to the complex, heterogeneous and evolving requirements of users; work needs to be done by specific users to incorporate these generic solutions to their particular contexts and practices (domestication).
Surprisingly, few previous frameworks in the health and social care literature have attempted to capture this insight, which partly explains why on-the-job training in technology use and ongoing helpdesk support are key to the implementation process [
A prominent theme in our empirical findings—but addressed tangentially or not at all by most previous frameworks—was that implementing health technology programs involves a great deal of work [
Around half of the frameworks identified in this review included a question on the wider context for technology-supported care [
While some implementation frameworks were designed around a rigid (and apparently systematic) logic model [
Abbott et al [
Van Gemert-Pijnen et al reviewed 16 frameworks for the implementation of eHealth innovations published up to 2009 (although in our own classification, 3 of these were not actually frameworks) [
Van Dyk, writing from an industrial engineering background, produced a taxonomy of eHealth implementation frameworks based on several theories, including diffusion of innovations, technology acceptance and use, e-readiness of organizations, transactional economics, and the information system life cycle, as well as considering more pragmatic barriers-and-facilitators studies [
Van Dyk drew an important insight from organizational life cycle studies (including her own work on a telemedicine service maturity model for health care organizations [
One final concept, which appears key to organizational adaptation over time but was not explicitly addressed in any of the previous frameworks, is the notion of resilience, defined as “the intrinsic ability of a system to adjust its functioning prior to, during, or following changes and disturbances so that it can sustain required operations, even after a major mishap or in the presence of continued stress” [
In sum, we found the integrated frameworks of van Gemert-Pijnen et al [
The final version of the NASSS technology implementation framework is shown in
Our case studies raised a variety of challenges across all 7 domains, each of which could be classified as simple (straightforward, predictable, few components), complicated (multiple interacting components or issues), or complex (dynamic, unpredictable, not easily disaggregated into constituent components) [
This domain addresses the clinical (question 1A) and the comorbidities and sociocultural aspects (question 1B) of the condition. It reflects the striking finding across all our case studies that only a fraction of potential end users were assessed by their clinicians as “suitable” for the technology. In the majority, the condition was considered clinically high risk, unpredictable, or atypical (eg, complicated by comorbidities or sociocultural factors, especially cognitive or health literacy considerations).
The NASSS framework for considering influences on the adoption, nonadoption, abandonment, spread, scale-up, and sustainability of patient-facing health and care technologies.
Question 2A addresses material and technical features of the technology. The technologies in our case studies were more or less freestanding. In all cases, features such as size, sounds, aesthetics, and “clunkiness” had significant impact on the technology’s actual and perceived usability and appropriateness. Many were insufficiently prototyped. Dependability of technologies was key, especially in high-risk conditions or social situations.
Question 2B considers the knowledge generated or made visible by technology. This includes not only the accuracy of the data but also the extent to which those data are accepted, trusted, and considered sufficient for decision making. Engaging with the data generated by patient-facing technologies may inform, educate, and empower patients and lay caregivers. But data may also be misinterpreted by the patient or cause distress.
Question 2C addresses the knowledge and support needed to use the technology. Some technologies are much easier to operate than others; some require frequent troubleshooting; and some assume a different organizational role—or even an altered professional identity—for the user. Some patient-facing technologies require no knowledge form the patient; others require clinical knowledge, technical knowledge, and the ability to make judgments about (for example) what counts as urgent.
Question 2D addresses issues for sustainability raised by the technology supply model—that is, how the technology was procured, the nature of the client-supplier relationship, and the level of potential substitutability via the marketplace. The telehealth device marketplace is notorious for its lack of interoperability as companies attempt to retain their market share through lock-in of customers. The consequences for health and social care services—and their clients—of market withdrawal could be significant. While procurement options may not have immediate impact on scaling up and sustainability, they are likely to have significant impact in the long term and may influence decisions on whether to adopt an innovation in the first place. Customizable, off-the-shelf technologies (COTS) [
This domain concerns whether a new technology is worth developing in the first place—and for whom it generates value. Question 3A addresses upstream value, which follows the supply-side logic of financial markets and investment decisions (and hence depends on preliminary tests of efficacy and safety, and evidence of good business practice). Question 3B addresses downstream value, which follows the demand-side logic of health technology appraisal, reimbursement, and procurement (ie, relates to evidence of benefit to patients and real-world affordability). As
Domains and questions in the nonadoption, abandonment, scale-up, spread, and sustainability (NASSS) framework.
Domain/question | Simple | Complicated | Complex | |
1A. What is the nature of the condition or illness? | Well-characterized, well-understood, predictable | Not fully characterized, understood, or predictable | Poorly characterized, poorly understood, unpredictable, or high risk | |
1B. What are the relevant sociocultural factors and comorbidities? | Unlikely to affect care significantly | Must be factored into care plan and service model | Pose significant challenges to care planning and service provision | |
2A. What are the key features of the technology? | Off-the-shelf or already installed, freestanding, dependable | Not yet developed or fully interoperable; not 100% dependable | Requires close embedding in complex technical systems; significant dependability issues | |
2B. What kind of knowledge does the technology bring into play? | Directly and transparently measures [changes in] the condition | Partially and indirectly measures [changes in] the condition | Link between data generated and [changes in] the condition is currently unpredictable or contested | |
2C. What knowledge and/or support is required to use the technology? | None or a simple set of instructions | Detailed instruction and training needed, perhaps with ongoing helpdesk support | Effective use of technology requires advanced training and/or support to adjust to new identity or organizational role | |
2D. What is the technology supply model? | Generic, “plug and play,” or COTSa solutions requiring minimal customization; easily substitutable if supplier withdraws | COTS solutions requiring significant customization or bespoke solutions; substitution difficult if supplier withdraws | Solutions requiring significant organizational reconfiguration or medium- to large scale-bespoke solutions; highly vulnerable to supplier withdrawal | |
3A. What is the developer’s business case for the technology (supply-side value)? | Clear business case with strong chance of return on investment | Business case underdeveloped; potential risk to investors | Business case implausible; significant risk to investors | |
3B. What is its desirability, efficacy, safety, and cost effectiveness (demand-side value)? | Technology is desirable for patients, effective, safe, and cost effective | Technology’s desirability, efficacy, safety, or cost effectiveness is unknown or contested | Significant possibility that technology is undesirable, unsafe, ineffective, or unaffordable | |
4A. What changes in staff roles, practices, and identities are implied? | None | Existing staff must learn new skills and/or new staff be appointed | Threat to professional identity, values, or scope of practice; risk of job loss | |
4B. What is expected of the patient (and/or immediate caregiver)—and is this achievable by, and acceptable to, them? | Nothing | Routine tasks, eg, log on, enter data, converse | Complex tasks, eg, initiate changes in therapy, make judgments, organize | |
4C. What is assumed about the extended network of lay caregivers? | None | Assumes a caregiver will be available when needed | Assumes a network of caregivers with ability to coordinate their input | |
5A. What is the organization’s capacity to innovate? | Well-led organization with slack resources and good managerial relations; risk taking encouraged | Limited slack resources; suboptimal leadership and managerial relations; risk taking not encouraged | Severe resource pressures (eg, frozen posts); weak leadership and managerial relations; risk taking may be punished | |
5B. How ready is the organization for this technology-supported change? | High tension for change, good innovation-system fit, widespread support | Little tension for change; moderate innovation-system fit; some powerful opponents | No tension for change; poor innovation-system fit; many opponents, some with wrecking power | |
5C. How easy will the adoption and funding decision be? | Single organization with sufficient resources; anticipated cost savings; no new infrastructure or recurrent costs required | Multiple organizations with partnership relationship; cost-benefit balance favorable or neutral; new infrastructure (eg, staff roles, training, kit) can mostly be found from repurposing | Multiple organizations with no formal links and/or conflicting agendas; funding depends on cost savings across system; costs and benefits unclear; new infrastructure conflicts with existing; significant budget implications | |
5D. What changes will be needed in team interactions and routines? | No new team routines or care pathways needed | New team routines or care pathways that align readily with established ones | New team routines or care pathways that conflict with established ones | |
5E. What work is involved in implementation and who will do it? | Established shared vision; few simple tasks, uncontested and easily monitored | Some work needed to build shared vision, engage staff, enact new practices, and monitor impact | Significant work needed to build shared vision, engage staff, enact new practices, and monitor impact | |
6A. What is the political, economic, regulatory, professional (eg, medicolegal), and sociocultural context for program rollout? | Financial and regulatory requirements already in place nationally; professional bodies and civil society supportive | Financial and regulatory requirements being negotiated nationally; professional and lay stakeholders not yet committed | Financial and regulatory requirements raise tricky legal or other challenges; professional bodies and lay stakeholders unsupportive or opposed | |
7A. How much scope is there for adapting and coevolving the technology and the service over time? | Strong scope for adapting and embedding the technology as local need or context changes | Potential for adapting and coevolving the technology and service is limited or uncertain | Significant barriers to further adaptation and/or coevolution of the technology or service | |
7B. How resilient is the organization to handling critical events and adapting to unforeseen eventualities? | Sense making, collective reflection, and adaptive action are ongoing and encouraged | Sense making, collective reflection, and adaptive action are difficult and viewed as low priority | Sense making, collective reflection, and adaptive action are discouraged in a rigid, inflexible implementation model |
aCOTS: customizable, off-the-shelf.
Question 4A is about adoption (and continued use) of the technology by staff. Reflecting the findings of previous studies reviewed above, some staff in each of our case studies simply did not engage with the program or use the technology. Nonadoption (or in some cases, abandonment) of the technology was occasionally explained by the technology’s attributes (such as usability or ease of use). More commonly, staff were concerned about threats to their scope of practice or to the safety and welfare of the patient—and even, in some cases, about a fear of job loss.
Question 4B addresses adoption by patients or clients, including acceptance (hence symbolic meaning and aesthetics) and the work required of them.
Question 4C addresses the assumptions that may be built into the technology (or the linked service model) about the availability and behavior of lay caregivers. We encountered many cases of nonuse of all patient-facing technologies that were explained by weak or absent social networks, limited information technology skills (and distrust of technology) among lay caregivers, or long-standing family conflicts, which the technology sometimes brought to the surface but never solved (see examples in
Questions 5A and 5B address the organization’s capacity (to embrace any service-level innovation) and readiness (for a specific technology), respectively. Our case studies included a wide range of antecedent conditions and levels of readiness that influenced uptake and internal scale-up of technology-supported programs (see examples in
Question 5C addresses the adoption decision, typically a board-level decision to allocate a budget line to support a particular technology. Our empirical data indicated three main problems with business modeling. The first problem was lack of data: it was often impossible to predict the uptake, use, and impact of the technology or the amount of investment needed to get (and keep) it up and running. Predictions were often guesstimates and typically overoptimistic, for example, in terms of numbers of potential users and potential costs and efficiency savings. The second problem was lack of money to support the program (see “organizational slack” above). Indeed, several organizations in our case studies did not appear to even recognize the need for a dedicated budget, over and above the cost of the technology, to support implementation and maintenance. The third problem was interdependencies between organizations and the teams working within and across them.
Question 5D considers the extent to which established work routines will be disrupted or made too brittle by the new technology. In some cases, there will be a transition period in which new collaborative routines linked to the new technology will be played out in parallel with existing routines—perhaps on legacy systems—before (hopefully) replacing them.
The last question in domain 5 concerns the
Question 6A relates to the wider institutional and sociocultural context, which in our case studies was often key to explaining an organization’s failure to move from a successful demonstration project (heavily dependent on particular champions and informal workarounds) to a fully mainstreamed service (scale-up) that was widely transferable (spread) and that persisted long term (sustainability). Aspects of wider context that proved pivotal in our case studies included health policy (including which service models were formally approved for funding—see example in
While the domains above can be distinguished analytically, the reality of any technology implementation project is that at an empirical level they are inextricably interlinked and dynamically evolving, often against a rapidly shifting policy context or continued evolution of the technology (and, at an individual patient level, as the condition deteriorates or fluctuates over time).
Question 7A relates to the medium- and long-term feasibility of continuing to adapt the technology and the program. Some technologies were more amenable to adaptation than others. Adaptation of staff roles and care pathways was also difficult when organizations were in subcontractor relationships (sometimes with private providers).
Question 7B concerns organizational resilience, and particularly what May and Finch called “reflexive monitoring” [
While an adaptive and reflexive approach is essential for effective scale-up, spread, and sustainability, it is impossible to be prescriptive at the outset as to how to go about this. Both formative and summative evaluation of the program’s success will need to use imagination and multiple methods, along with a narrative account of what happened and why, in order to capture its multiple interdependencies, nonlinear effects, and unintended consequences [
Colleagues found our near-final NASSS framework helpful in considering challenges to implementation of a range of technology-supported health or care programs. While some used the framework (as we ourselves did) prospectively and in real time to predict and explore the challenges of implementing an existing technology to support a new program, some used it retrospectively to explain failures and partial successes in past projects. One group (a design company) suggested that the NASSS framework might be used at an early (design) stage to focus attention first on the condition and adopter system domains (ie, the needs of intended users), which would then inform both the technology and value proposition domains.
Building on previous work by other groups, this study has developed and applied a new framework for predicting and evaluating the success of technology-supported health and social care programs. Across a diverse sample of 6 empirical case studies followed (so far) for up to 3 years and tested briefly (for what might be called face validity) in a further sample of 10 additional case studies, we identified and explained numerous examples of nonadoption and abandonment of the technology by individuals or limited success in attempts to scale up, spread, and sustain the program within and beyond the organization.
A striking finding that explained many such instances was a tendency to assume that the issues to be addressed were simple or complicated (hence knowable, predictable, and controllable) rather than
The technology and linked program had been designed around an oversimplified, textbook model of the condition.
The technology was insufficiently prototyped, insufficiently customizable, insufficiently dependable, dependent on complex knowledge to use it, or designed to generate data or knowledge that was incomplete, inappropriate, or contested in the care context.
The value proposition of the technology was unclear, in terms of a viable business venture for its developer or in terms of a clear benefit for patients and an affordable real-world service model.
The intended users of the technology had plausible personal or professional reasons to resist or reject it.
The organization(s) were inadequately set up for innovation, not ready for (or interested in) this particular innovation, unable to negotiate a viable business model with partner organizations, unable to shift to new ways of working, or unable to support the extensive work needed to implement and sustain the change (including the rhetorical “work” of making sense of an equivocal technology).
Complexity in external (financial, regulatory, legal, policy) issues—of which reimbursement seemed to be particularly key—stalled the mainstreaming and spread of the program.
The program was unable to adapt and evolve over time in a way that continued to meet the needs of its intended users and remain clinically, operationally, and financially viable.
The NASSS framework has been developed systematically to fill a key gap in the literature on technology implementation—specifically, to address not just adoption but also
NASSS was never intended to provide a simple fix for complex problems. It cannot be applied in a formulaic way or used deterministically as a tool, nor do we believe it is possible to make firm predictions about which elements of the framework will be mission-critical or how the different elements will interact (since these are likely to differ substantially in different cases and settings). With these caveats, we believe that, subject to further empirical testing, the NASSS diagram in
The empirical work to develop NASSS was based entirely in the United Kingdom, although collaborators who peer reviewed and tested the draft framework included groups from Australia, Canada, Italy, and the United States. The 6 case studies presented in this paper have been followed for up to 3 years; field work is ongoing, so additional insights about long-term sustainability may emerge in the future. We strongly encourage other research groups to explore the applicability of the framework for different purposes and to adapt and extend it if appropriate.
Previous technology implementation frameworks identified in our hermeneutic review of the literature are summarized in
First, we have added a preliminary focus on the illness or condition for which the technology is assumed to be a solution, an emphasis that emerged from our empirical data but probably also reflects the background of several authors as clinicians, psychologists, or patient-facing service coordinators. Second, we have introduced the heuristic of classifying each domain of interest as simple, complicated, or complex, and cautiously concluded that it is
Implementing new technologies as part of changes to health and social care services is inherently challenging. While policy makers are calling for technology to be implemented rapidly and at scale, the reality is that when dealing with the multiple complexities of health and care, it is extremely difficult to go beyond small-scale demonstration projects. We hope that the NASSS framework will help implementation teams—and, at an earlier stage, technology and service designers—to identify, understand, and address the interacting challenges to achieving sustained adoption, local scale-up, distant spread, and long-term sustainability of their programs.
While we believe that the NASSS framework is academically defensible, additional work is now ongoing to make it accessible to its intended users beyond academia. In the United Kingdom, the NHS National Technology Adoption Centre (NTAC) was set up in 2007 to promote the uptake of health technologies through its Technology Implementation Projects and HowToWhyTo guides (which attempt to distil lessons learned from pilots in a format designed to be easily transferable and hence reusable). One of the principal objectives of the guides has been to assist practitioner stakeholders to build a business case for innovation. However, a recent study suggests that their impact has been disappointing and points to the difficulties of making context-specific knowledge transferable and hence reusable in other contexts [
Previous implementation/evaluation frameworks for health and care technologies.
Examples from case studies to illustrate NASSS.
anchored, realistic, cocreative, human, integrated, and evaluated
customizable, off-the-shelf
global positioning system
nonadoption, abandonment, scale-up, spread, and sustainability
Studies in Co-creating Assisted Living Solutions
Seamless User-Centred Proactive Provision of Risk-Stratified Treatment for Heart Failure
Virtual Online Consultations: Advantages and Limitations
We are grateful to the participants (patients, caregivers, and staff) in the VOCAL and SCALS studies for their commitment and insights. We also thank the study steering groups, especially Mrs Jenni Bowley who chaired them. We are grateful to various colleagues who tested the NASSS framework on their own projects and provided feedback, and especially staff at Frog Design who contributed to substantial changes to the NASSS diagram. We drew on a qualitative evaluation of Product B undertaken by researchers at another university and published internally by the caregivers’ charity.
The VOCAL study was funded through the UK National Institute for Health Research Health Services and Delivery Research grant to TG (13/59/26). The SCALS program’s main funding is from a Senior Investigator Award and Public Engagement Award to TG from the Wellcome Trust in its Society and Ethics Program (WT104830MA). Additional funding for the GPS tracking case study was provided through a Program Development Grant to TG from the National Institute for Health Research (RP-DG-1213-10003). The SCALS study is registered on the Wellcome Trust’s Society and Ethics portfolio. TG, SS, JW, and NF are part-funded by the National Institute for Health Research Biomedical Research Centre, Oxford, UK (grant NIHR- BRC-1215-20008). The initial draft of this paper was written when TG was on a writing retreat at the Bellagio Center, Italy, funded by the Rockefeller Foundation.
None declared.