This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
Digital health innovations can improve health system performance, yet previous experience has shown that many innovations do not advance beyond the pilot stage to achieve scale. Vietnam’s National Immunization Information System (NIIS) began as a series of digital health pilots, first initiated in 2010, and was officially launched nationwide in 2017. The NIIS is one of the few examples of an electronic immunization registry (EIR) at national scale in low- and middle-income countries.
The aim of this study was to understand the determinants of scale-up of the national EIR in Vietnam.
This qualitative study explored the facilitators and barriers to national scale-up of the EIR in Vietnam. Qualitative data were collected from October to December 2019 through in-depth key informant interviews and desk review. The mHealth Assessment and Planning for Scale (MAPS) Toolkit guided the development of the study design, interview guides, and analytic framework. MAPS defines the key determinants of success, or the “axes of scale,” to be groundwork, partnerships, financial health, technology and architecture, operations, and monitoring and evaluation.
The partnership and operations axes were critical to the successful scale-up of the EIR in Vietnam, while the groundwork and monitoring and the evaluation axes were considered to be strong contributors in the success of all the other axes. The partnership model leveraged complementary strengths of the technical working group partners: the Ministry of Health General Department of Preventive Medicine, the National Expanded Program on Immunization, Viettel (the mobile network operator), and PATH. The operational approach to introducing the NIIS with lean, iterative, and integrated training and supervision was also a key facilitator to successful scale-up. The financial health, technology and architecture, and operations axes were identified as barriers to successful deployment and scale-up. Key barriers to scale-up included insufficient estimates of operational costs, unanticipated volume of data storage and transmission, lack of a national ID to support interoperability, and operational challenges among end users. Overall, the multiple phases of EIR deployment and scale-up from 2010 to 2017 allowed for continuous learning and improvement that strengthened all the axes and contributed to successful scale-up.
The results highlight the importance of the measured, iterative approach that was taken to gradually expand a series of small pilots to nationwide scale. The findings from this study can be used to inform other countries considering, introducing, or in the process of scaling an EIR or other digital health innovations.
Digital health innovations are changing the way health is delivered worldwide. The World Health Organization (WHO) defines digital health as “the field of knowledge and practice associated with the development and use of digital technologies to improve health” [
Vietnam’s National Immunization Information System (NIIS) began as a series of digital health pilots, first initiated in 2010 [
Scale-up refers to “deliberate efforts to increase the impact of innovations successfully tested in pilot or experimental projects so as to benefit more people and to foster policy and program development on a lasting basis” [
The digital health community has increasingly recognized the importance of scale and has initiated efforts to support scaling digital health interventions [
The evidence base to understand factors that influence scaling digital health interventions is limited [
The aim of this study was to understand the determinants of scale-up of the national EIR in Vietnam, thereby contributing to the evidence on how and why digital interventions can successfully achieve scale.
This study explored the facilitators and barriers to national scale-up of the EIR in Vietnam through qualitative methods, using the mHealth Assessment and Planning for Scale (MAPS) Toolkit as a conceptual framework [
The MAPS Toolkit served as the conceptual framework for this study to ground Vietnam’s experience in scaling up an EIR within known factors that influence successful scale-up and sustainability of digital health products. Although the toolkit was designed to prospectively guide iterative program implementation, it has also been used for retrospective program evaluation [
The mHealth Assessment and Planning for Scale Toolkit axes of scale [
The mHealth Assessment and Planning for Scale Toolkit drivers of success within each axis of scale [
Vietnam has a 5-level health system, with administrative levels at the national, regional, provincial, district, and commune levels [
Structure of the health system in Vietnam. EPI: Expanded Program on Immunization; IHE: Institute of Hygiene and Epidemiology.
The evolution of the current national EIR in Vietnam began with a small pilot (
Timeline of the electronic immunization registry introduction and scale-up in Vietnam. MOH: Ministry of Health; NIIS: National Immunization Information System.
These two systems were combined, upgraded, and then deployed by NEPI with support from PATH in all districts of Ben Tre province from 2014 to 2015. This resulted in increased full immunization coverage and improved on-time vaccination rates [
In 2016, this combined system was integrated into the NIIS, which was being developed by the MOH in partnership with Viettel, the largest mobile network operator (MNO) in Vietnam. The NIIS was to be an EIR system in which health workers could register and track the immunization records of pregnant women and newborns, as well as inform vaccine stock management. The NIIS technical working group includes GDPM, NEPI, Viettel, and PATH, who partnered to pilot the NIIS in 5 provinces. The NIIS was further upgraded based on learnings from this experience and in preparation for national scale-up. In June 2017, the NIIS was officially deployed nationally. Leading up to nationwide scale-up, the MOH requested all communes to back enter the full vaccination history for all children born from January 2015 through June 2017.
From 2018 to 2019, the NIIS technical working group focused on increasing NIIS uptake, improving data quality and data use, and transitioning completely to paperless immunization records. As of January 2020, over 20 million records have been registered into the system (including back-entered data, immunization data for pregnant women, and childhood immunizations delivered among the annual birth cohort of approximately 1.7 million [
Project documents were selected for inclusion in the document review based on project and study relevance. A total of 1 project proposal and 8 project evaluation reports detailing the project objectives, processes, successes, and challenges were included in the document review. Documents included end users’ perspectives on the acceptability and feasibility of the system through surveys of Expanded Program on Immunization (EPI) staff at provincial, district, and commune health centers conducted in 2012 and 2015.
Critical case purposive sampling [
PATH staff conducted 6 in-depth semistructured interviews in person with key informants from October to December 2019. The semistructured interview guide was developed based on the evaluation questions and the MAPS framework. Interviews were audiorecorded with permission from the informant, and the interviewer documented summary notes during the interview. Audiorecordings were transcribed into Microsoft Word and translated into English.
An a priori codebook was developed, guided by the MAPS axes. The codebook was then refined by coding data from project documents and in-depth interviews in Microsoft Excel and Atlas.ti (version 8; ATLAS.ti Scientific Software Development GmbH). Coding was completed independently and concurrently by 2 PATH staff and an external evaluator. Codes were reviewed by the evaluation team and any changes or disagreements were discussed until resolved. Once coding was finalized, codes were grouped into key themes guided by the evaluation questions and MAPS axes. Exemplary quotations were identified to present the themes and were validated with key stakeholders.
The study procedures were reviewed and received nonresearch determination by PATH. Prior to the interviews, participants were informed of the study’s objectives, advantages and disadvantages of participating, and rights of participants. Importance of maintaining confidentiality was emphasized during training of data collectors and the start of interviews. Written consent was obtained from all participants. Study data were stored in an access-restricted server, only available to study staff for the purpose of data analysis.
Key facilitators and barriers to scale-up.
Axes of scale | Facilitators | Barriers |
Groundwork and monitoring and evaluation | - Learnings from multiple pilots and phases contributed to optimization of all other axes |
N/Aa |
Partnerships | - Each member of the partnership plays a critical role |
N/A |
Financial health | - Estimated time and cost of system |
- Unanticipated operational costs, particularly for refresher trainings, supportive supervision, and regular system upgrades |
Technology and architecture | - User-friendly system |
- Enormity of data affects system capacity and network connectivity |
Operations | - Availability of pretested and enhanced training materials and standard operating procedures |
- Dual reporting systems (paper and digital) |
aN/A: not applicable.
bMOH: Ministry of Health.
cMNO: mobile network operator.
The partnership and operations axes were most commonly perceived to be critical to the successful deployment and scale-up of the EIR in Vietnam. Though key informants identified elements in the financial health axis and technology and architecture axis that were facilitators to successful scale-up, these axes were not as commonly or strongly mentioned in comparison to the partnership and operations axes. The groundwork and monitoring and evaluation axes were considered to be strong contributors in the success of all the other axes.
The NIIS is a product of a hybrid model, with the public, private, and civil society sectors contributing to the successful scale-up and sustainability of the NIIS. The NIIS partnership has enabled the government to own and run the system alongside supportive partners working together as a team.
The NIIS TWG was formed in 2016 with the aim to contribute to the technical implementation and sustainability of the NIIS. Membership comprises MOH’s GDPM as the management authority, NEPI as the immunization expert, Viettel as the technology expert, and PATH as the liaison, serving as the connection across stakeholders. Initially the TWG met weekly, then over time it shifted to once or twice per month, depending on whether NIIS operationalization challenges emerged that required a timely response.
The most commonly expressed theme among key informants was the importance of the partnership in the success of the EIR implementation and scale-up. Key informants repeatedly noted the importance of the role that each member held in the TWG. One informant commented:
Each member of the TWG plays an important role. It could not be as it is without each member.
Key informants described the importance of the MOH’s role, both to sustainably lead and to oversee the national system. As key informants observed:
Political commitment is key. First you need to have political commitment, to invest in resources and provide guidance from the government for implementation.
Only the MOH can manage such a large system that includes so much of the population.
NEPI provided immunization expertise and acted as the technical lead to define and develop immunization workflows, user requirements, and reporting systems and mechanisms. NEPI also oversaw and strengthened the implementation of the NIIS.
Given the expansive technological requirements of a national EIR, the MOH sought to partner with an MNO that already had an extensive network and was capable of high-capacity data storage. In addition to these technical specifications, key informants noted the importance of selecting a well-established MNO with a history of success:
If you want to implement a system at this scale, it is imperative to work with a big company like Viettel. The MNO of choice must have large capacity and a large presence in the country, be financially secure, and be able to provide necessary human resources in both quality and quantity. Choosing an MNO, it would be best to select the largest MNO with the longest history of success in the country of implementation.
Additionally, some informants observed that the nature of the partnership between MOH and Viettel was a driver for successful scale-up and sustainability. That is, Viettel provided key development, storage, and maintenance services rather than only providing software developer services that would eventually be transitioned to the government to maintain and upgrade. As one informant explained:
Viettel provides a service, not just the software, so there is always a partnership and not just a handover. So, whenever the system needs to be updated, the MOH and Viettel are working together. NIIS belongs to the MOH and Viettel provides a service. This is a good model for other countries: outsource the service to a mobile network operator.
PATH has played a critical role throughout the pilot–to–national scale-up process. Having both local and global experience in piloting, scaling up, and evaluating digital health products, PATH was able to transfer all technology, such as data flow and database structure design, to Viettel and share lessons learned with the TWG. PATH could also act as a liaison between the MOH, a health system–focused entity, and Viettel, an information technology entity, given its experience in global health and digital technologies. One informant explained:
PATH complements and supplements the gaps in MOH and Viettel. PATH’s experience in the field, piloting and demonstrating and disseminating that information globally, has been critical.
This partnership, in which each partner contributed a critical role, was built on a foundation of trust, which key informants identified as an important driver of successful scale-up and sustainability:
Viettel is government owned, so they trust the government, and the government trusts them. So when MOH trusts PATH, then there is a big circle of trust… An MNO that is trusted by the government is very important. Because the health information is very sensitive. So the partnership can only be established with an MNO that is an old and trusted one, not a new or just established MNO. This helps with scaling-up.
Another recurrent theme across the interviews was the strength of the end user training and support approach that was developed for successful scalability and sustainability.
For NIIS national deployment, training materials and standard operating procedures, which had been developed, tested, and enhanced based on learnings from a landscape assessment and monitoring and evaluation of multiple phases of the EIR, were readily available for use. Additionally, the cascade approach to conduct training-of-trainer trainings had been developed to quickly establish and expand a network to train end users across the country. As one informant explained:
You want to conduct a cascade training because you don’t have the resources to go everywhere. Instead, conduct the training-of-trainer training to provincial and district health workers, then they can provide training and technical support to end-users in hospitals, fee-based facilities, and commune health centers. This is the best method due to limited resources and the trainers will be the supervisors, so this informs them about who is the strongest and who is weak so they can provide feedback to them to improve.
Through these trainings, a multilevel technical support network was established across the country. The trained provincial and district health staff acted as mentors who served as focal points for technical support at each health level and provided direct support to end users. Supportive supervision was integrated into existing structures of immunization supervision. In addition, Viettel trained district-level staff on the NIIS, who are now able to provide technical support to end users. This approach is working to sustain country ownership and sustainability of the NIIS in Vietnam.
Estimating the cost of the system and its implementation to appropriately allocate resources to scale up was critical to the success of the NIIS. After the first pilot from 2010 to 2012, a business model framework was developed to provide an overview of the proposed key partners, resources, and activities for sustaining and scaling an EIR in Vietnam. A costing model was also created to determine the financial resources needed to implement the system across multiple provinces in Vietnam.
Identified costs included software development and maintenance, training, supportive supervision and monitoring, and the cost for technical meetings after the first months of implementation. In the case of the NIIS, the MOH has not incurred the cost of software development and maintenance, as the MNO has been providing these services in kind, currently with the expectation that the pricing model may shift in the long-term.
In addition to estimating costs of the system, as a necessary component to successful EIR scale-up, it was essential to have the government commitment to allocate a budget to this work. As one informant explained:
The government needs to commit and support the implementation and [provide] direction and leadership to the local government to allocate the budget for implementation and maintenance.
Key informants noted 3 main technology and architecture facilitators to successful scale-up of the EIR related to the technology itself: its user-centered design, adaptability, and strong data security and privacy. Initial development of the EIR and subsequent upgrades prioritized end user needs. As such,
The system is designed with high-rate of acceptability and is user-friendly. It meets the basic requirements of end-users and it operates smoothly. These are factors to excite provinces to implement and use the system.
The NIIS supports facility health workers to document and track immunization records and can easily print electronic records for storage in compliance with current regulations. Individual immunization records can be updated with vaccine doses delivered in different facilities, which is particularly important in large cities where many children receive vaccines from public and private facilities and there is a large migrant population moving between facilities.
To maintain system acceptability by ensuring smooth operation of the system, the NIIS has a mechanism to monitor system operations so that issues that appear can quickly be resolved. The system is updated regularly, and adaptations can be made to fit evolving needs. Relatedly, the system has strong data security and privacy measures to maintain public trust of the NIIS.
In addition to the NIIS design, the importance of an already existing technology infrastructure at health facilities cannot be overstated as a facilitator to the successful deployment and scale of the EIR. That is, because almost all health facilities in Vietnam already had computers and internet, and health care workers had access to computer systems, the NIIS could be developed with this starting point in mind. Had computers and internet not already been available at health facilities, the NIIS would likely have been designed differently or would have required additional resources at the outset for equipment and connectivity.
The steps taken in the early phases of the EIR and the M&E learnings from multiple EIR implementation phases set the groundwork for successful national scale-up, especially informing the planning, implementation, and optimization of the other axes of scale. Given this interconnected relationship between groundwork and M&E throughout the many implementation phases, we merged these two axes for our analysis.
Key groundwork activities included a landscape analysis and the development of a business model framework. The main aim of the landscape analysis was to evaluate how a digital registry might improve the ability to track children due for vaccinations and how it might shorten the time required for recording and reporting immunizations compared to a paper-based registry. This analysis provided a snapshot of the current policies, technical capacities, and health information systems already in place or under development in Vietnam. On the importance of conducting a landscape analysis, one informant emphasized:
It is a necessary step to look at two sides: the policy and technical sides. Before you scale-up, you must look at the policy environment and see what are the advantages, what is available, and what are the gaps. To understand what is financially necessary and what are some foreseen challenges.
The analysis was particularly critical in informing the development of the pilot EIR system in 2010 but also continued to guide decisions about the operations and the technology and architecture of the EIR phases. One major takeaway from the analysis was the importance of understanding user needs to build a system that supports end users at all levels. The analysis also informed the development of the end user training plan, as it identified potential issues such as high turnover and low computer literacy of health care workers that could affect successful implementation and scale-up of the EIR.
Learnings gathered from both groundwork activities and M&E facilitated effective and efficient national scale-up:
We only had 8 months to work on the NIIS together [in 2016], but at that time we already had the business model and understood well about the business and the technical, so we could just transfer the knowledge and the technology to develop the NIIS. This shortened the time for NIIS development.
The multiple phases of the EIR that preceded NIIS national scale-up created many opportunities for learning and optimization. In total, the Vietnam EIR system phased through 4 pilots and deployments from 2010 to 2017, starting with a pilot at the district level, then scaled and piloted at the provincial level, scaled again and piloted in multiple provinces, and then finally, scaled nationally. As one informant noted:
The piloting of the system is so important to get feedback from the end-users, before scale-up. And it is important to collect lessons learned as you go, to determine when you can and how you can scale-up.
This continuous test-learn-optimize and scale-learn-optimize improvement cycle allowed for improvements in the operations and technology and architecture of the EIR, such that by the time it was scaled nationally, many things had been pretested and improved.
Furthermore, evaluation activities from these phases contributed to the growing evidence of the benefits of an EIR, as well as local validation that such a system was both feasible and effective in the Vietnam context. It was these quantifiable successes that catalyzed the support for nationwide scale-up. As one key informant recommended:
The pilot should be conducted in two years, to get the feedback and upgrade the system, and have enough time to evaluate the system. Two years is enough to evaluate the impact of the system rather than just evaluate the acceptability and feasibility of the system. In the short-term, end-users just want to say the good things.
The financial health, technology and architecture, and operations axes were most commonly perceived to be barriers to the successful deployment and scale-up of the EIR in Vietnam.
Although the cost of EIR deployment was estimated during various EIR phases, there were some costs to national scale-up that were underestimated or not estimated. The majority of the initial financial model was based on costs associated with software development and maintenance. Budgets for operational costs of training, supportive supervision, and monitoring and evaluation were sometimes insufficient. For example, there was insufficient funding and human resources for EIR’s supportive supervision visits:
A low budget for supportive supervision visits made it difficult for higher level supervisors to conduct timely supervision of end-users.
Local governments do not allocate budget for research and some don’t have supervision budgets, so they had to integrate [NIIS supervision] with supervision of other health areas.
Another unanticipated cost to scale-up was that, because of the enormity of the data, the server has needed to be frequently updated and expanded. Through this experience, the TWG has identified the importance of accurately estimating the operational aspects of scaling up an EIR.
Two key barriers to successful scale-up and sustainability were identified within the technology and architecture axis, namely issues with data transmission and storage and interoperability.
Although a sizing and infrastructure assessment was conducted to inform the server capacity and connectivity bandwidth, the substantial increase in the demand for data storage and transmission for national scale-up were not appropriately anticipated. There are nearly 20 million clients in the NIIS, and this number will continue to increase as the population grows. Informants explained:
We did not foresee just how large the data would be, how large of a number of people would be registered into the system, and how to store this data without slowing down the system. The number of people registered is growing rapidly… During times of high influx, the NIIS system is overloaded. The server needs to be updated and expanded frequently to accommodate the enormity of the data.
We did not estimate the increase of data, therefore could not optimize the algorithm as well as the database infrastructure from the beginning.
This, along with a large number of people using the system, has meant that the system can sometimes run slowly. The TWG is working to find a solution, which includes upgrading the server, separating the data storage and data query servers, and developing standard operating procedures in cases where the system is down. The MNO has also been working to address this issue by identifying and indexing frequently queried data files to optimize the speed of queries.
Another barrier to successful scale-up has been the lack of a national ID to develop an integrated, interoperable system. Viettel is currently working to develop a health information system that integrates all data from existing systems, including immunization, infectious diseases, and noncommunicable diseases, all of which have their own software without a common unique identifier for each client. This has been an issue even within the NIIS, with duplication of individual records. (Based on a January 2019 GDPM report, 1.2% of NIIS records were duplicates [
There were 3 key barriers to successful scale-up and sustainability identified within the operations axis: dual reporting, low computer literacy of end users, and high turnover of health care workers.
From the beginning, there have been persistent concerns among health sector staff about whether an EIR would increase the workload of health care workers. Evaluations of the early phases of the system found that it successfully reduced the time burden of reporting among health workers; however, the current dual-reporting scenario is burdening the health workforce workload. Because a completely digital system is not mandated by the government, end users are tasked with using both the paper-based system and the NIIS to ensure data accuracy in both systems. During planning for national scale-up, a clear transition plan from paper to digital would reduce the workload for end users.
During the landscape analysis conducted in 2010, low computer literacy of end users was identified as a possible barrier to effective implementation and scale-up of an EIR. Some health care workers were not well versed in computer usage, including basic computer functions, such as typing and Microsoft Excel. Low computer literacy also meant that staff would not know how to use the data to calculate indicators and interpret results for decision making. As one key informant described:
One of the difficulties in the pilot phase was the computer skills of the health care workers. Some needed a lot of support, even in typing. We trained on everything. This was a big challenge for implementing the system.
Given that the issue was identified during the landscape analysis, the project team was able to use various strategies in the development of the EIR and training approach to increase the usability of the EIR for end users:
Due to different levels of digital literacy of commune health workers, each commune must assign two people to take part in the training sessions. After the training, this peer support is very important to ensure success in implementation.
A lot has changed since these early years of EIR implementation, with an increasingly computer-literate health workforce and increased computer and internet infrastructure at the commune level. This change has helped increase the accessibility and usability of the NIIS.
Another barrier to scale-up and sustainability identified during the landscape analysis was the frequent staff rotation and high staff turnover rate. Again, because this was identified during the landscape analysis, it was possible to address this during the development of the training approach. Multiple staff members at every level of the health system, including the health facility, district, provincial, and national levels, are trained. The cascading training-of-trainers approach (described in “Operations” under the “Key Facilitators to Scale-up” section) also ensures a sustainable peer-training system is in place to train new staff. Vietnam is also exploring e-learning options to train new staff.
While there were facilitators across all axes within the MAPS framework, the partnership and operations axes were most commonly perceived to be critical to the successful scale-up of the EIR in Vietnam. The EIR scale-up has been facilitated by a partnership comprised of public, private, and civil society actors working together toward shared goals and leveraging each partner’s expertise. It was operationalized through a cascading training-of-trainer approach, which was integrated within existing supervision structures and used training materials that were iteratively refined during each phase of implementation.
The most commonly perceived barriers to the scale-up of the EIR in Vietnam were related to the financial health, technology and architecture, and operations axes. Related to financial health, there were challenges in sufficiently estimating operational costs associated with training, supportive supervision, and monitoring and evaluation. Technology and architecture barriers included issues with the unanticipated volume of data storage and transmission demands and the lack of a national ID to support interoperability with other health information systems. Key operational barriers were mainly among end users and included challenges of dual-reporting systems, low computer literacy, and high staff turnover.
Overall, the phased approach of multiple small pilots allowed for iterative assessment and planning that strengthened all the axes and contributed to the successful scale-up.
Though the MAPS Toolkit was not designed as an evaluation framework, overall, the axes of scale and their specified definitions helped ground the analysis for this evaluation. The MAPS conceptual model acknowledges that scaling up is an “iterative cyclical process of thorough assessment, careful planning and targeted improvements” [
Using the MAPS Toolkit allowed for comparison between the experience in Vietnam with that of Tanzania and Zambia, where a recent study explored the factors influencing the introduction and adoption of EIRs in low-resource settings using the MAPS Toolkit [
The central role of the partnership model in Vietnam cannot be overstated. There were initial challenges as partners developed shared language and understanding across their areas of expertise, which included public health and immunization, financial health, and information technology. It was important to allow time for this learning curve and to formalize roles as part of the NIIS TWG. Lessons learned from implementing subnational EIRs in the United States also highlighted the importance of a steering committee or coalition to guide EIR development and garner support [
In terms of operations, Tanzania initially used an on-the-job training approach that relied on PATH staff to visit each health facility, but later Tanzania and Zambia implemented a training-of-trainers approach similar to Vietnam’s model. As mentioned, nongovernmental partners were more heavily involved in the initial subnational deployment and trainings in Tanzania and Zambia, which was also a large cost driver in these countries [
Whereas in Vietnam, the existing technology infrastructure was in place at health facilities, this was not the case in Tanzania and Zambia, where lack of hardware and electricity was a barrier to initial introduction of the EIR [
As more countries aim to introduce EIRs, these findings can inform efforts to plan for scale-up. Seth Berkley, CEO of Gavi, the Vaccine Alliance, wrote about the importance of harnessing digital innovations to support immunization, noting that “one of the biggest needs is for affordable, secure digital identification systems that can store a child’s medical history, and that can be accessed even in places without reliable electricity” [
This study was based on the experience scaling the EIR in Vietnam, and care should be taken when generalizing these findings beyond Vietnam. This limitation was in part addressed by identifying determinants guided by the MAPS Toolkit, a conceptual framework that can be applied to digital health implementations in varied contexts.
This study purposively selected key informants to represent the range of partners on the NIIS TWG; each was very involved in the EIR scale-up and could speak to the details of the implementation. The study focused on national-level stakeholders and did not include key informants representing subnational or community perspectives. Although end user perspectives were represented in the documents reviewed, for a more comprehensive understanding of the factors influencing the EIR scale-up in Vietnam, future studies could also include perceptions of the barriers and facilitators from EIR end users at the district and health facility levels, as well as caregiver or community perceptions of immunization service delivery. Different determinants may emerge at each level. For example, a systemic literature review (with the majority of studies from the United States) identified health care providers’ perceptions of usefulness and ease of use of digital health innovations as influential factors to their acceptance of the innovation [
This study described the key facilitators and barriers that influenced the scale-up of the NIIS in Vietnam using a comprehensive digital health framework. The results highlight the importance of the measured, iterative approach that was taken to gradually expand a series of small pilots to nationwide scale. Key facilitators included the partnership model, which leveraged complementary strengths of the MOH and GDPM, NEPI, Viettel, and PATH, and the operational approach to introducing the NIIS with lean, iterative, and integrated training and supervision. Key barriers to scale-up included insufficient estimates of operational costs, unanticipated volume of data storage and transmission, lack of a national ID to support interoperability, and operational challenges among end users. The findings from this study can be used to inform other countries considering, introducing, or in the process of scaling an EIR or other digital health innovations.
Bill and Melinda Gates Foundation
electronic immunization registry
General Department of Preventive Medicine
mHealth Assessment and Planning for Scale
monitoring and evaluation
mobile network operator
Ministry of Health
The Vietnam National Expanded Program on Immunization
Vietnam’s National Immunization Information System
technical working group
World Health Organization
The authors thank the Bill and Melinda Gates Foundation (BMGF) for providing support for this study. The authors also thank the technical working group team members involved in implementing the EIR scale-up in Vietnam, including representatives from GDPM, NEPI, Northern EPI, Hanoi Center for Disease Control, and PATH. Thank you to the PATH Vietnam team: Hung Pham, Thi Hoang, and Hai Le. The authors thank Tove Ryman (BMGF) for her support and guidance. This study was funded by BMGF (Investment ID OPP1198325).
None declared.