This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
Screening is an effective primary prevention strategy in health care, as it enables the early detection of diseases. However, the uptake of such screening remains low. Different delivery methods for screening have been developed and found to be effective in increasing the uptake of screening, including the use of web-based apps. Studies have shown that web-based apps for screening are effective in increasing the uptake of health screening among the general population. However, not much is known about the effective implementation of such web-based apps in the real-world setting. Implementation strategies are theory-based methods or techniques used to enhance the adoption, implementation, and sustainability of evidence-based interventions. Implementation strategies are important, as they allow us to understand how to implement an evidence-based intervention. Therefore, a scoping review to identify the various implementation strategies for web-based apps for screening is warranted.
This scoping review aims to identify (1) strategies used to implement web-based apps for health screening, (2) frameworks used for implementing web-based apps for health screening, (3) outcome measures of implementation strategies, and (4) effective implementation strategies.
This scoping review was conducted based on Arksey and O’Malley’s framework. After identifying the review question, two researchers independently screened and selected relevant literature from PubMed, Embase, Cochrane, Cumulative Index of Nursing and Allied Health Literature, PsycINFO, International Standard Randomised Controlled Trial Number Registry, OpenGrey, ClinicalTrials.gov, World Health Organization International Clinical Trials Registry Platform, and Web of Science. This was followed by charting the data using a standardized form. Finally, we collated, summarized, and reported the results quantitatively and qualitatively based on the review objectives.
A total of 16,476 studies were retrieved, of which 5669 were duplicates. From a total of 10,807 studies, 10,784 studies were excluded based on their titles and abstracts. There were 23 full-text articles reviewed, and 4 articles were included in the final analysis. Many studies were excluded because they focused on the effectiveness and not on the implementation of web-based apps. Facilitation was the most cited implementation strategy used, followed by reminders, clinical champions, and educational meetings and materials. Only 2 studies used implementation frameworks to guide the evaluation of their studies. Common outcome measures for implementation strategies were feasibility, fidelity, and penetration. Implementation strategies reported to be effective were quality improvement meetings, facilitation, educational meetings, and clinical champions.
There is a dearth of literature on the implementation of web-based apps for health screening. Implementation strategies were developed without any reported use of implementation theories or frameworks in most studies. More research on the development and evaluation of web-based screening app implementations is needed.
Screening is one strategy for health promotion and disease prevention. Screening is the process of identifying healthy people who may be at an increased risk of a particular disease or health problem [
Many studies have shown screening to be effective in the prevention of diseases. A study in Korea reported that participation in cardiovascular disease (CVD) health screening was associated with lower CVD and all-cause mortality from CVD. There was also an increase in the early detection of CVD and a reduction in health care use and cost [
However, even when screening is beneficial and effective, its uptake remains suboptimal [
Information and communication technology (ICT) has become an important platform to improve health care in the general population [
A web-based app is defined as a program that is accessed over a network connection using HTTP rather than existing within a device’s memory. Web-based apps often run inside a web browser. However, web-based apps may also be client-based, where a small part of the program is downloaded to a user’s desktop, but processing is done over the internet on an external server [
Web-based apps for screening use questionnaires to accomplish screening. This screening can include mental and behavioral health as well as risk factor assessment. Examples include screening for depression risk, alcoholic addiction, and smoking. Users (patients) enter their responses based on the questions, and the web app sometimes provides appropriate advice to the patient. Several studies have shown that web-based apps for screening are effective in increasing the uptake of health screening among the population [
Implementation research involves the study of “the use of strategies to adopt and integrate evidence-based health interventions and change practice patterns within specific settings” [
Nonetheless, studies have shown that many EBIs do not reach their target population [
Studies show that implementing ICT in health care settings can be complex [
Studying
This scoping review uses the framework by Arksey and O’Malley [
Among studies exploring the implementation of web-based screening apps:
What implementation strategies were used and proven effective?
What studies were informed by implementation theories, models, and frameworks, and which ones in particular?
What implementation outcomes were measured?
What factors were identified as influencing implementation effectiveness?
The literature search was conducted using PubMed, Embase, Cochrane, CINAHL (Cumulated Index to Nursing and Allied Health Literature) via EBSCOHost, PsycINFO via OvidSP, ISRCTN (International Standard Randomised Controlled Trials Number) registry, OpenGrey, ClinicalTrials.gov, World Health Organization International Clinical Trials Registry Platform, and Web of Science for peer-reviewed scientific literature. The search was conducted in August 2018 for 2 weeks and concluded on August 28, 2018. Other search methods used to supplement the literature include reference or footnote tracking, using the “related articles” function in PubMed, citation tracking, personal knowledge and personal contacts, and contacting experts in the field. The search terms and search strategies are included in
The inclusion criteria were studies that described any implementation strategies used to implement web-based apps for screening. Definition of specific terms are outlined as follows:
Implementation strategies: methods or techniques used to enhance the adoption, implementation, and sustainability of EBIs [
Web-based app: any program accessed over a network connection using HTTP rather than existing within a device’s memory [
Frameworks: A framework is a structure, overview, outline, system, or plan consisting of various descriptive categories (eg, concepts, constructs, or variables) and the relations between them that are presumed to account for a phenomenon [
The study must also include all age groups and genders; quantitative, qualitative, and mixed-method study designs will be included, and studies must be in the English language.
The exclusion criteria were as follows:
Non–web-based apps such as desktop-based, computer-based, CD-ROM interventions, mobile apps, and electronic health records
Nonempirical references such as trial protocols, book reviews, editorials, magazine articles, and theoretical or methodological articles
Non-English studies
Two independent reviewers screened the titles and the abstracts. The full texts of relevant articles were retrieved and screened by the two independent reviewers. Any disagreements were discussed among the two reviewers, and a third reviewer was consulted if the two reviewers were unable to resolve any disagreements.
Data were extracted using a standardized form. Two reviewers independently extracted the data. Any disagreements were resolved with a third reviewer. Data extracted from individual studies include author(s), year of publication, origin or country of origin (where the study was published or conducted), aims or purpose, frameworks, study population and sample size (if applicable), methodology or methods, intervention type, comparator and details of these (eg, duration of the intervention; if applicable), duration of the intervention (if applicable), outcomes and details of these (eg, measures; if applicable), and how the key findings relate to the scoping review question(s). The two reviewers met after extracting data from two articles to determine if the data extraction process was consistent with the research questions and purpose. Any inconsistencies were discussed until a consensus was reached. A third researcher was consulted when the two researchers were unable to reach a consensus.
The results of this review were divided into three distinct steps based on recommendations by Levac et al [
The consultation exercise was conducted among experts in primary care, eHealth, and implementation science. The experts discussed the selection of articles and helped troubleshoot the issues that occurred during the review process. This exercise helped to validate and shape the study outcomes of this review.
Quality assessment of the studies was done using the Mixed Methods Appraisal Tool (MMAT). MMAT was used as it is a validated tool to quickly assess qualitative, quantitative, and mixed methods studies [
The searching databases yielded a total of 16,476 records. After the initial screening and removal of duplicates, 4 studies were included in this review (
Flow diagram of selection of studies.
Most studies were excluded at the title or abstract screening phase because the aim of this review was to identify implementation strategies for web-based apps. Most studies found through the database search evaluated the effectiveness of the web-based apps rather than focusing on their implementation. Only 4 studies fulfilled the inclusion criteria [
Characteristics of included studies.
Authors | Year | Intervention | Setting | Study population | Methods | Implementation |
Webb et al [ |
2018 | Health and lifestyle screening tool (“app”), Check Up GPa for young people in an Australian general practice |
Australia General practice |
Patients: young people 14-25 years of age HCPb: GP, practice manager, reception coordinator, and receptionists |
Qualitative: semistructured interviews and focus groups Quantitative: cross-sectional survey |
2 months |
Diez-Canseco et al [ |
2018 | A web-based mental health screening app |
Peru Primary care |
Patients: adults ≥18 years of age HCP: midwives, nurses, and nurse assistants |
Mixed methods Qualitative: face-to-face structured interviews with HCPs and patients Quantitative: web-based data collection platform |
9 weeks |
Krist et al [ |
2014 | MOHRc, a web-based health risk assessment tool |
United States Primary care |
Patients: adults HCP: clinicians, front desk staff, practice rooming staff, and medical assistants |
Cluster-randomized, mixed methods implementation trial Qualitative: interviews with HCPs Quantitative: data from research networks and the MOHR tool |
10 months |
Scribano et al [ |
2011 | Computerized intimate partner violence screening |
United States Hospital emergency department |
Patients: adults HCP: emergency department staffs |
Qualitative: direct observation of patient use, feedback from patients, one-on-one feedback from emergency department staff, and team meetings Quantitative: questionnaire survey through kiosk. |
15 months |
aGP: general practitioner.
bHCP: health care provider.
cMOHR: My Own Health Report.
The single randomized controlled trial included in this review was of moderate quality [
Only 2 studies used frameworks to guide and evaluate the process and outcomes of the study. However, in both studies, a framework and theory were used to guide the evaluation of the studies and not on the development of the implementation strategies.
Implementation frameworks and strategies used for web-based apps.
Authors | Frameworks | Types of framework [ |
Implementation strategies [ |
Implementation activities |
Webb et al [ |
NPTa | Implementation theories |
|
|
|
|
|
Reminder | Phone call and SMS as reminder to patients to complete the app |
|
|
|
Reminder and facilitation | Receptionist to prompt patient to complete app in the waiting room using tablet |
|
|
|
Continuous quality improvement | Quality improvement meetings with HCPsb |
|
|
|
Educational materials | Provision of educational documents to support HCPs |
Diez-Canseco et al [ |
None | N/Ac |
|
|
|
|
|
Leadership engagement/buy-in | Engaging policy makers for support and buy-in |
|
|
|
Educational meetings | Training for HCPs |
|
|
|
Technical assistance | Telephone and face-to-face support, and supervision for HCPs throughout the implementation period |
|
|
|
Coaching | Regular supervision meetings with HCPs to troubleshoot problems encountered throughout the implementation period |
Krist et al [ |
RE-AIMd | Evaluation framework |
|
|
|
|
|
Clinical champions | Appointment of practice champions |
|
|
|
Educational meetings | Training for HCPs |
|
|
|
Facilitation | Mailed invitations to complete the app |
|
|
|
Facilitation | Kiosk provided in the clinic waiting room to complete the app with help from researcher |
|
|
|
Facilitation | Completion of the app via phone call by researcher |
|
|
|
Facilitation | Completion of the app via tablet with help from either researcher, practice rooming staff, or medical assistant in the clinic waiting room |
Scribano et al [ |
None | N/A |
|
|
|
|
|
Patient education and facilitation | Nurses and receptionists provided instruction forms to patients for the screening kiosks |
|
|
|
Environment | Placing screening kiosks in strategic locations |
|
|
|
Clinical champions | Appointment of practice champions |
aNPT: Normalization Process Theory.
bHCP: health care provider.
cN/A: not applicable.
dRE-AIM: Reach, Effectiveness, Adoption, Implementation, Maintenance.
Facilitation was the most cited strategy used. Other strategies that were cited more than once included reminders, clinical champions, and educational meetings.
The implementation outcomes for the studies included in this review were categorized based on the taxonomy of implementation outcomes by Proctor et al [
Outcome measures.
Authors, Outcomes measured | Implementation outcome | |
|
||
|
Number of times needed to provide support to staff on the use of the app | Feasibility |
|
Location completed Check Up GPa | Feasibility |
|
If patients received an SMS with a link to Check Up GP before attending the practice | Fidelity |
|
If patients felt that they had sufficient privacy when completing the app | Fidelity |
|
Postimplementation staff interviews and focus group discussions based on NPTb | Acceptability, adoption, appropriateness, feasibility, fidelity, sustainability |
|
||
|
Number of screenings | Penetration |
|
Fidelity to screening | Fidelity |
|
Integration of screening into routine clinical service | Sustainability |
|
||
|
Reach: proportion of eligible patients approached who completed a MOHRc assessment | Penetration |
|
Adoption: percentage of practices approached for study participation who agreed to participate | Adoption |
|
Implementation: how practices integrated MOHR into their workflow and the time and staff needed to carry out implementation steps | Fidelity |
|
Maintenance: whether early intervention practices continued to use MOHR after the study | Sustainability |
|
||
|
Number of screenings completed by patients | Penetration |
|
Computer technology failure rate | Feasibility |
aGP: general practitioner.
bNPT: Normalization Process Theory.
cMOHR: My Own Health Report.
Different studies revealed different implementation strategies that were reported to be effective in their settings (
For Webb et al [
Quality improvement meetings
Facilitation
Educational meetings
Clinical champions
Environment
Facilitation
In the study conducted by Diez-Canseco et al [
For Krist et al [
In the study by Scribano et al [
This review revealed that there is a very small set of literature reporting on the implementation of web-based apps for screening. Many studies evaluated the effectiveness or application of web-based screening apps, but very few described how to implement them [
Integrating an implementation study with an effectiveness study may be one way forward. Hybrid study designs may yield more information, as both the effectiveness of the intervention and the implementation can be assessed in a single study [
In this review, the most common implementation strategies used in implementing web-based apps were facilitation, education, and clinical champions. Most of these strategies targeted only patients and health care providers. Only 1 study mentioned strategies related to leadership engagement [
There was also no clear discussion on how strategies were developed or designed for implementation. Using a systematic process and determinant frameworks to help guide the development of strategies can be important for successful implementation and evaluation [
In this review, different outcome measures were used in each study. However, there was no clear explanation of how the authors of the studies determined which outcome to measure. Which implementation strategies contributed to the outcomes were not clear because several different strategies were used in most studies, and their individual effects could not be evaluated. Therefore, we were unable to conclude which implementation strategies were useful. The lack of implementation frameworks or theories used in the studies may have contributed to this issue. The process of developing implementation strategies should be done systematically, ideally to link each strategy to different outcomes. If this is feasible, it could allow us to assess which strategies work. This may also allow researchers to further explore why a particular strategy works for a specific outcome. Lack of effect may lie with the implementation strategy itself or may be related to how the implementation strategy was implemented. Therefore, it is crucial that researchers give adequate thought to the implementation outcomes they measure and how an implementation strategy might affect each outcome. In addition, the field of implementation research is shifting from an exclusive focus on the effectiveness of implementation strategies to a focus on how strategies achieve their effect, if any, and on the factors that make implementation, across a spectrum of evidence-based practices, including ICT approaches, either more or less likely to be successful. Future studies will hopefully include this broader focus, as the current literature does not go beyond a simplistic examination of the implementation strategies’ effectiveness.
This review may be limited because only English language studies were included. Another limitation could have been related to screening titles and abstracts. This may have limited the inclusion of studies if implementation strategies were not discussed in the abstract. However, based on our review, we found that most of the implementation strategies could be identified from abstracts alone. To this end, this scoping review revealed that only a few studies addressed the implementation of web-based health screening apps, and therefore, more research is needed in this area.
This scoping review shows that the study of implementation of web-based apps for screening is still in its infancy. Many studies have assessed the effectiveness of web-based apps, but only a few have focused on how to implement them. We were able to identify different implementation strategies for implementing web-based apps for screening. However, there is little evidence that the strategies reported were systematically developed using theories or frameworks. The lack of frameworks and theories used in these studies was also evident. There is a need to study not only the effectiveness of implementation strategies but the process of implementation and how this affects outcomes. This review shows that more work is still needed to study implementation of web-based apps for screening in a systematic process based on implementation theories and frameworks.
Search terms and search strategy for scoping review.
Mixed Method Appraisal Tool Table.
Cumulated Index to Nursing and Allied Health Literature
colorectal cancer
cardiovascular disease
evidence-based intervention
electronic health
information and communication technology
International Standard Randomised Controlled Trials Number
Mixed Methods Appraisal Tool
International Prospective Register of Systematic Reviews
This study is funded by University of Malaya Specialist Centre C.A.R.E research grant PV-063-2018. The authors would also like to thank Universiti Malaysia Sarawak for financial support.
None declared.