Ningbo is a city in the Zhejiang province of China (28°51’ – 30°33’ N, 120°55’ – 122°16’ E). It is located in the south wing of the Yangtze River Delta and consists of 4 counties (Yuyao, Cixi, Ninghai, and Xiangshan) and 6 districts (Yinzhou, Haishu, Jiangbei, Zhenhai, Beilun, and Fenghua), with a total land area of 9365 km². In 2017, the city population was >8 million people, and nearly 6 million are part of the hukou (ie, registration of an individual) of Ningbo. The resident population is relatively stable, with a 4.1% out-migration rate and 11.1% in-migration rate.

The regional health care information platform in Ningbo was initially devised and launched in 2011 by the Health Commission of Ningbo to construct a united and standardized medical information network and facilitate health care services. By 2015, the database covered almost all health-related activities of residents within this area, and more than 87% of the residents were registered in the database with a valid health care identifier. In 2016, the database was approved and awarded a Four Grade Class A, which at the time was the top achievement level by the Standardization and Maturity Measurement of Regional Health Information Interconnection by the National Health Commission of China.

Based on information in previous studies, we developed the questionnaire and open-ended interview guidelines to evaluate the feasibility of active surveillance following HPV vaccination using the regional health care information platform in Ningbo. To finalize the questionnaire and guidelines, four experts from pharmacoepidemiology, vaccine safety, medical informatics, and database administration independently reviewed the documents and provided feedback. Then, a 2-step survey was conducted to collect database information from the data owners, management, and other staff. First, we sent the questionnaire, its instructions, a comprehensive description of the study purpose, and the completion deadline to the database staff via email. After receiving the questionnaire responses, we confirmed any potential discrepancies with Ningbo through telephone communication. Second, we conducted face-to-face interviews, using the open-ended interview guidelines, with relevant staff during field visits.

The questionnaire about the database consisted of 7 parts: (1) basic information in the database; (2) information in electronic medical records (EMR) in the database; (3) vaccination registration and information about maternal and child health care in the database; (4) method and possibility of linking data between different data sources; (5) age distribution of HPV vaccine–protected women in 2017; (6) completeness of core variables relevant to active surveillance of vaccine safety in 2017; and (7) crucial outcome information in EMR in 2017.

The interview guideline was mainly composed of 3 parts: (1) general characteristics of the database; (2) general characteristics of the essential information system for active surveillance, including vaccination registration, EMR, maternal and child health care information, cancer registration, death registration; and (3) other several specific questions.

The study was approved by the Peking University Institutional Review Board (IRB 00001052-18016). Moreover, all the tasks were performed by staff of the data owner, and investigators could not access the raw data throughout the study period.

The Ningbo database was constructed and is owned by the Ningbo Health Commission (NHC), and the Ningbo Centers for Disease Control and Prevention (CDC) can operate and develop this database with permission from the NHC. There are two primary data sources in the database, including the digital CDC platform and digital hospital platform. The CDC controls the former, which mainly focuses on public health and provides a unified information system for the whole city, including integrated primary health care, chronic or infectious disease surveillance, vaccine registration (children and adults), and death registration. Data in the hospital digital platform are collected during regular clinical services provided in hospitals, such as from EMR, and the information system usually differs by hospital, but the NHC has developed harmonized standards to transform and upload these data, which are in a nearly structured format in the Ningbo database. Moreover, the NHC and CDC have endeavored to improve the data quality in the database through official regulations, upgrading of standards, and regular quality checks. The local CDC has also established an institutional review board to review ethical issues and guarantee privacy protection during the process of scientific research.

Table 1 summarizes the characteristics of the five data sources related to active surveillance for HPV vaccine safety in the Ningbo database; data from these sources were updated and uploaded into the database daily. First, residents’ electronic health records were created primarily by general physicians when the residents visited medical and health institutions for services; these records currently cover >87% of the permanent residents in Ningbo. Second, all vaccination records, including free vaccines provided by the Expanded Programme on Immunization or vaccines paid for out-of-pocket, for infants and children aged <6 years are registered in the immunization program system. It was a compulsory requirement from the Ningbo CDC beginning in May 2017 that vaccination records for adults be registered. Third, starting in 2015, the platform integrated new and historical EMR data from all public hospitals and most of the large private hospitals in the city. Fourth, the maternal and child health care information system includes data recorded during health care services provided for prenatal examinations, maternal deliveries, and newborn follow-ups (until the infant was 1 year old), which resulted in an increase in the local pregnancy and maternal registration rate to 97% by 2016. Finally, the death surveillance information system primarily includes identity information and medical certificates of the dead and can trace death records outside of the city of Ningbo but inside Zhejiang province.

From 2017 to 2018, there were two types of HPV vaccines: bivalent and quadrivalent. Both were only available for women in this district. Table 2 describes the characteristics of the population who received the HPV vaccination in Ningbo. During this period, 19,328 women received at least one dose of the HPV vaccine. Women aged 30-40 years represented the largest proportion. Quadrivalent vaccination was administered more often than bivalent vaccination. Moreover, 60 (60/19,328, 0.31%) first doses seemingly occurred outside Ningbo, and these data were not traced back to the database.

The completeness of critical variables relevant to HPV vaccination (eg, recipient identity card [ID], name, gender, birth date, address, vaccine name, vaccination date, vaccination manufacturer, batch number) was also investigated, and for these variables, there were no missing data. The system did not have information for education, occupation, height, weight, smoking habits, and alcohol consumption.

Table 3 shows the ability of the 10th revision of the International Statistical Classification of Diseases and Related Health Problems (ICD-10) codes to identify the safety outcomes of interest in EMR as well as the incident cases of diseases 90 days after HPV vaccination. All visit records of the cases were retrieved from the EMR using standard query language and the ICD-10 code, and duplicate records were removed. As a result, for data from 2017 and 2018, ICD-10 codes can be used to identify all the outcomes of interest in women aged 9-45 years. Among these diseases, there were 1867 cases of systemic lupus erythematosus, which was the largest proportion of cases. In contrast, there were only 2 cases of acute disseminated encephalomyelitis, which was the smallest proportion of cases. Except for that of Henoch-Schonlein purpura (HSP; 118/1606, 7.35%), the rate of missing IDs for other diseases was near or <5%, and the total rate of missing IDs was 1.88% (173/9180).

Moreover, we monitored the risk of specific adverse events during the 90 days after HPV vaccination. For all the doses, we found 4 incident cases with no history of the disease at least 1 year before HPV vaccination. Two new cases, one each of rheumatoid arthritis (RA) and optic neuritis (ON), were diagnosed within 30 days of bivalent HPV vaccination, and one new case of HSP was diagnosed within 90 days of bivalent HPV vaccination. One new case of RA emerged within 90 days after quadrivalent HPV vaccination.

Hospitals not included in the database could not diagnose nor treat the outcomes of interest. Regarding case validation, structured data from imaging examinations, laboratory tests, histological examinations, and other medical examinations could be found directly in the database, while nonstructured or free-text information, such as the chief complaint, may have needed to be retrieved from the hospital.

A feasibility assessment is the first step to improve the quality of study protocols and accelerate regulatory approvals and, in turn, the start of an actual study [17]. The regional health care information platform in Ningbo covers the entire city and has integrated the essential data sources for active surveillance of the HPV vaccine, including immunization program registration, EMR, resident health records, and death surveillance; these data can be linked using the unique national ID or encoded identifier. More importantly, this database can sensitively identify the outcome of interest through ICD-10 coding, so we can actively monitor risks following HPV vaccination by linking HPV vaccination records with adverse events from EMR, death, or other data sources. Therefore, this database could be available and feasible in research for active surveillance of adverse events following HPV vaccination.

The Chinese CDC established the AEFI Surveillance System, which is a passive national surveillance system covering more than 29 provinces, in 2015 [18]. However, regarding active surveillance, the relevant studies have used traditional methods for timely collection of adverse events, such as by telephone or daily cards, that were designed and performed during the H1N1 pandemic [19]. Plus, we have not found a study that attempted to construct a sustained active surveillance system for AEFI despite the large population and vast geographical area in China [7]. Internationally, some countries have developed sustainable active surveillance systems for adverse events using population-based databases that link vaccine history with outcomes, such as the VSD [9] and Post-Licensure Rapid Immunization Safety Monitoring program [20]. In this study, considering that the crucial data components in the Ningbo database are similar to those in the VSD and Post-Licensure Rapid Immunization Safety Monitoring program, we consider this to be pioneering work in China – to construct a continuous method for actively monitoring AEFI during the post-marketing phase of vaccines. Moreover, we can also link vaccination history with death records in the Ningbo database, which might reduce potential bias from right-censored data.

In this study, of the specific list of outcomes, we identified 4 incident cases within the 90 days following HPV vaccination. The combined incident rate of RA, ON, and HSP was 8.84/100,000 doses of bivalent HPV, and the incident rate of RA was 3.75/100,000 doses of quadrivalent HPV. There was no cluster of these adverse events. Considering the prevalence of RA is 0.46% [21] and the lack of large epidemiology studies of ON [22] and HSP in Chinese women, the rate of adverse events was not high enough for a safety signal. Furthermore, an updated systematic review concluded that the risk of RA, ON, or HSP is not increased with any type of HPV vaccine [23]. However, due to the limited sample size and different backgrounds of Chinese women, a larger sample or longer observational period is required to comprehensively review the safety of HPV vaccines in China.

To monitor the long-term safety of HPV vaccines, a continuous, sustainable surveillance system is necessary. There are several ways of setting controls to promptly identify and alert of vaccine safety signals. First, comparing with historical controls, or background rates, is an approach to identify whether the incidence rate of adverse events following vaccination is higher than the background rate, which is not due to chance [24]. However, it may not be feasible to perform this comparison in the Ningbo database due to the short premarketing period of the HPV vaccine. Four study designs were recently recognized and have become widespread for vaccine safety surveillance, including the self-controlled case series (SCCS), self-controlled risk interval (SCRI), cohort, and case-control study designs [25], and SCRI and SCCS have been proven to be an efficient, rational alternative to the cohort in a simulation study [25,26]. In a real-world study, self-controlled analyses, such as those used in SCRI and SCCS, are used because each case acts as its own control within a short period, thereby inherently adjusting for all time-unvarying potential confounders, such as sex, nationality, and genetic predisposition [27,28]. In our study, given that the database can only capture the visit records of safety outcomes and HPV vaccines that occurred in the city, it is likely that we will misclassify these crucial variables in a subsequent study; for example, some local young women can be vaccinated while attending college outside the city. In addition, these records cannot currently be traced back to the Ningbo database unless the women return to Ningbo and utilize the relevant local services.

Therefore, using the SCRI as the primary design to evaluate the risk after vaccination seems most appropriate, minimizing misclassification bias due to incomplete vaccine exposure. A cohort study as the secondary design may improve statistical power [28]. Also, additional regional databases need to be incorporated to increase the robustness of risk estimation, especially for different subpopulations or districts.

Some potential barriers need to be overcome in additional studies. First, because the HPV vaccination rate is so low in Ningbo, the observation period should be lengthened, or additional sites should be incorporated. Second, identification algorithms for safety outcomes should be developed to improve performance, with a positive predictive value >70% [29]. Third, migration may cause potential bias in the representative population or misclassification of exposure and outcomes, so we should consider its influence in future research. Finally, missing links between data should be addressed and balanced with privacy concerns [30].

The study presents an available approach to initiate an active surveillance system for adverse events after HPV vaccination, based on a regional health care information platform in China. Utilizing a longer observation period or including additional functional sites is warranted to conduct future hypothesis-generating and hypothesis-confirming studies to address vaccine safety concerns.