Emerging and re-emerging infectious diseases such as HIV and AIDS, severe acute respiratory syndrome (SARS), and influenza continue to pose considerable national and global public health concerns [1] based on reported cases of these viruses leading to outbreaks [2]. They remain a severe public health problem in the 21st century and a leading cause of death worldwide [3]. According to the World Health Organization (WHO), infectious diseases are the second leading cause of death in humans, with approximately 15 million fatalities annually [3,4]. Infectious diseases are the most common health threats associated with religious, sporting, and festival mass gatherings (MGs) [5]. A leading global concern regarding MGs is the spread of infectious diseases and their importation and exportation as they spread from attendees to the general population. This spread causes substantial problems for the host country’s health system [6]. International travel, which has increased and become faster, facilitates the transmission of infectious diseases during MG events from attendees to the host country’s local population and vice versa. Examples of outbreaks include the 2002/2003 SARS pandemic, the 2009 swine influenza pandemic, the 2012 Middle East respiratory syndrome outbreak, the 2014 Ebola epidemic, and the recent COVID-19 pandemic in 2020 [7].

The transmission of an infectious disease poses a high risk for attendees at MGs because of crowd size, density, and the duration of the gathering, especially high-profile events such as sporting events (eg, the Olympic Games and the Federation International Football Association [FIFA] World Cup), religious events (eg, the Hajj and World Youth Day), cultural events (eg, Glastonbury Music Festival), and MGs involving national security (eg, political conventions). Respiratory, zoonotic, vector-borne, fecal, oral, sexual, and blood-borne transmission have been reported to be leading causes of morbidity and mortality worldwide [8].

Countries prepare for MG events by following the WHO 2005 International Health Regulations for public health planning, surveillance, and response during MGs [9]. They are required to develop, strengthen, and maintain the capacity of their health care systems to detect, assess, notify, and report risk events to international public health authorities. Public health authorities ensure preparedness for infectious disease hazards that can spread internationally [10]. They use risk assessments as an essential step in the preparation for MGs along with MG stakeholders to strengthen the capacity of management during an MG, mitigate the risk of infectious disease transmission, and decrease pressure on the health system. Preparedness begins with the identification, evaluation, and prioritization of public health threats [11]. There are several factors event planners consider when using risk assessment tools that contribute to the health and safety of participants: weather, attendance, duration of the event, location of the event, event type, crowd mood, alcohol or drug use, crowd density, and age of attendees [9]. Examples of health risk assessment frameworks and tools are the Sendai Framework 2015-2030 developed by the United Nations, the Jeddah Tool developed in 2016 by the Global Centre for Mass Gathering Medicine, and the WHO COVID-19 Mass Gathering Risk Assessment [11-13].

Public health surveillance is essential for assessing, predicting, and mitigating infectious disease outbreaks [14]. Surveillance plays a substantial role in preparedness for MGs. It is necessary to provide health decision makers with timely and accurate information about infectious disease events to set priorities, identify the need for interventions, and evaluate their effects [15].

During the last 2 decades, experts have emphasized the importance of strengthening public health surveillance and consider it vital to recognize emerging infectious diseases and track the prevalence of those that are more established [1]. Public health surveillance is defined as “the ongoing systematic collection, analysis, and interpretation of data, closely integrated with the timely dissemination of the resulting information to those responsible for preventing and controlling disease and injury” [15].

An important function of surveillance systems is to monitor infectious diseases that have pandemic potential (eg, SARS and influenza), with timely detection of health events such as outbreaks, especially at MGs [1]. Traditional and disease-specific surveillance relies on passive routine reporting by health care facilities and diagnostic laboratories for structured predefined information regarding infectious disease events. However, this indicator-based surveillance (IBS) is inefficient because of limited resources, time, and reporting systems, which results in the incomplete reporting of data on emerging infectious diseases [16,17].

Other methods of surveillance have emerged with advancements in computational sciences to complement the shortcomings of IBS and improve the timeliness and sensitivity of surveillance systems [17]. Digital surveillance uses the internet, other computer-based systems, and emerging technologies for communications and diagnosis [18]. A description of each type is presented in Textbox 1.

Multiple public health interventions have been used to mitigate the spread of infectious diseases through early detection, prediction, and management (Table 1). A web-based real-time surveillance system (Google Flu) was used to identify regional infectious disease activity using search queries [17]. Monitoring devices and sensors are used for crowd control and tracking, such as wireless sensor networks and radiofrequency identification devices. Epidemic modeling and simulation of mixing patterns and disease transmission use computational modeling tools such as Susceptible-Infected-Recovered and agent-based modeling to represent the dynamics of the disease and simulate the behavior of individuals [19]. Event-based surveillance (EBS) systems and sites such as Health Map, BioCaster, EpiSPIDER, ProMED-mail, and the Global Public Health Intelligence Network are used to detect outbreaks and emerging public health threats [17].

Digital surveillance conducted using either passive or active approaches for data collection has become essential for infectious disease surveillance [16]. Passive surveillance is inexpensive and provides critical information for monitoring community health compared with active surveillance as critical data are reported by multiple health institutions such as hospitals, clinics, and public health units. In contrast, active surveillance is an expensive means of providing timely and accurate information on health conditions collected directly by staff members [20]. Technological advances in communication and unofficial mechanisms such as websites and social media simplify detection and monitoring and improve the response to health problems, thus reducing the potential damage caused by them [21].

Infectious disease prevention and control have been the focus of attention in research for many years, especially with the re-emergence of global pandemics. Studies have investigated different technologies and surveillance system applications that have the potential to detect outbreaks. Some studies have examined big data opportunities for the early detection of disease outbreaks in global MGs by applying different approaches (syndromic surveillance systems, internet data, monitoring devices, and epidemic modeling and simulation) for real-time aggregation and analysis of data from relevant sources [19]. Infectious disease surveillance and control and electronic surveillance network systems used for religious MGs such as the Hajj have also been investigated [6]. A variety of digital technologies used in public health responses to pandemics have been examined, such as social media and web-based searches, wearables and sensors, computer vision, machine learning, digital diagnostic genomics, and visualization tools [22]. They include more advanced tools that provide wireless connectivity in the manufacturing and service sectors to enhance automation, such as Industry 4.0 technologies (known as the Fourth Industrial Revolution) and their applications for fighting global pandemics (eg, COVID-19) [10].

Disease surveillance during MGs has existed for years and has been performed using traditional passive methods. Traditional surveillance systems have been an essential component of public health strategies for many decades, but they are ineffective against emerging infectious diseases. However, digital surveillance can improve the sensitivity and timeliness of health event detection [14,18].

Effective surveillance is achieved through the early identification of public health threats such as emerging infectious diseases. Monitoring and forecasting of emerging and re-emerging infectious diseases is of particular interest [23]. The timely collection, analysis, and interpretation of health data are conducted using appropriate surveillance systems. Thus, an effective and immediate response for disease control at MGs requires enhanced surveillance that informs the health outcomes and exposures to prioritize using risk assessment tools. It also requires the capacity for adequate surveillance to receive and analyze information rapidly [9].

Public health surveillance is critical for managing public health threats during international MGs [24], and it ensures that abnormal events and potential hazards that may threaten attendees’ health are addressed early in the event. Effective planning and responses to infectious disease threats can be achieved using early warning signals provided by an appropriate surveillance system during an MG [25].

This review aimed to identify public health digital surveillance systems that have been implemented for infectious disease prevention and control at MG events and to review evidence of their effectiveness for the prevention and control of infectious diseases at MG events.

A systematic review was conducted in accordance with the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines [26]. The author systematically searched 4 electronic databases (Ovid MEDLINE, Embase, CINAHL, and Scopus) in January 2022 for relevant articles published in English up to January 2022. A combination of search terms was used based on a framework developed by the author that divided the topic into 4 areas of interest (context: MGs, phenomenon of interest: public health, focus: infectious disease prevention and control, and intervention: digital technologies).

The authors combined Medical Subject Heading terms (“Mass Gatherings,” “Public Health,” “Infectious Disease,” and “Digital Technology”) derived from the Ovid MEDLINE and Embase databases with keywords from the relevant literature. The CINAHL and Scopus databases were searched without using subject terms as the data yielded by the search of Ovid MEDLINE and Embase had already been maximized. The authors minimized the search terms used in Scopus owing to differences in how the queries were processed in Scopus compared with the way they were processed in the other databases. The first search in Scopus using the same search terms as those used in Ovid MEDLINE, Embase, and CINAHL yielded numerous results because of the confusion between some of the search terms, such as “Large events,” “Large Gatherings,” “Pandemic,” and “Outbreak.” A description of the search strategies used in the Ovid MEDLINE, Embase, CINAHL, and Scopus databases is presented in Multimedia Appendix 1.

One author (NM) conducted the database search and independently screened the titles of all articles retrieved from the 4 databases to identify eligible studies for full-text review. If the title met the eligibility criteria, the abstract was screened by an additional author (MA) to check whether the study fulfilled the inclusion criteria. Both NM and MA reviewed the full text of the studies with the potential to meet the eligibility criteria and examined their relevance to the topic of this review. The included studies consisted of journal articles published in English up to 2022 that described or evaluated an MG or large religious, sporting, or musical event; implemented an intervention such as disease surveillance during the event using electronic means; and described the effects and outcomes of the intervention. Studies that were published in book chapters or periodic articles were excluded from the analysis, and studies in which the disease surveillance system was implemented in a non-MG setting or the surveillance system in the MG setting was not implemented using electronic means were excluded from the analysis. The author used an additional assessment framework to evaluate the included studies by assessing the characteristics of (1) the MG event and (2) the surveillance system or digital intervention that was implemented. A description of the inclusion and exclusion criteria of the studies included in this review is presented in Table 2. Details of the inclusion and exclusion criteria are shown in Figure 1.

The authors used EndNote (Clarivate Analytics), a reference manager, after the search to record potential studies retrieved and eliminate duplicates of articles across the 4 databases.

Owing to the absence of appraisal tools and criteria appropriate for interventional studies on public health surveillance systems for infectious disease prevention and control in MG settings, the primary researcher (NM) and the research team (JA and AV) developed an appraisal tool to assess the quality of the studies. The assessment tool was developed based on published frameworks and public health assessment tools. The questions included in the appraisal tool were divided into 3 categories: study, event, and intervention. The appraisal tool that was developed for this study is presented in Multimedia Appendix 2. A single author (NM) appraised the included studies.

The primary researcher (NM) extracted the data using a form developed by the authors to collect relevant data from each article. The form was developed to ascertain the basic features of each study (ie, author and year, study objective, event type, location, duration and year of the event, intervention, aim of the intervention, outcomes, limitations, and recommendations).

The database search in Ovid MEDLINE and Embase using the predefined combination of Medical Subject Heading terms yielded 1609 studies from Ovid MEDLINE; 61 (3.79%) were selected for abstract screening, of which 2 (3%) met the eligibility criteria and were selected for inclusion in the review. A total of 82 studies were retrieved from Embase, and 24 (29%) were selected for abstract screening, of which 2 (8%) were included in the analysis after meeting the eligibility criteria.

The database search in CINAHL yielded 71 studies; however, none of the studies met the eligibility criteria after their titles and abstracts were screened, so none were included in the review. The Scopus search (using the advanced search option) yielded 2160 studies; 74 (3.43%) were chosen for title and abstract screening, of which 12 (16%) underwent a full review for potential inclusion and 4 (33%) met the eligibility criteria for inclusion. A draft of the retrieved results is presented in Multimedia Appendix 3.

The characteristics of the included studies are summarized in Table 3. The 8 included studies in this review reported MG events in different countries for different purposes where an infectious disease surveillance system was implemented, described, or evaluated. A total of 2 studies on religious gatherings were included: 1 (50%) from Saudi Arabia (the Hajj in 2015 and 2019) [25,27] and 1 (50%) from India (the Prayagraj Kumbh in 2019) [24]. In total, 38% (3/8) of studies on sporting events were included [21,28,29] (the 2014 FIFA World Cup in Brazil, the 2012 Olympic and Paralympic Games in London, and the 2014 Micronesian Games in Pohnpei in the Federated States of Micronesia). A total of 12% (1/8) of studies on a cultural event were included as follows: the 11th Festival of Pacific Arts (FOPA) in the Solomon Islands in 2012 [30]. Only 12% (1/8) of the studies with more than one type of MG event (ie, religious, sporting, and other events) described the surveillance systems used in each event [31]. All the included studies (8/8, 100%) were interventional studies with descriptions of a public health intervention (ie, a surveillance system that was piloted [21], implemented [24,25,27,31], enhanced [29,30], or evaluated [28]).

The quality of the 8 included studies is presented in Multimedia Appendix 4 [21,24,25,27-31]. On the basis of the critical appraisal tool, 50% (4/8) of the studies [21,24,25,31] received low ratings as they scored 10 to 19 points, 38% (3/8) of the studies [27,29,30] received moderate ratings as they scored 20 to 29 points, and 12% (1/8) of the studies [28] received a high rating as they scored 30 to 39 points. We chose not to exclude studies with low ratings from the final synthesis because of the exploratory nature of the review.

The results of the included studies were synthesized under 2 themes: type of MG and implementation of the surveillance systems and effectiveness of the digital surveillance systems implemented for infectious disease prevention and control at MG events.

Owing to the different types of MG events included in this review and the variety of surveillance systems identified, the results are reported in 2 tables. Table 3 shows the characteristics of each type of MG event included in the review, the location and duration of the event, and the number and characteristics of the attendees. Table 4 presents the type of surveillance system implemented at each MG event, the aim of the intervention, the outcomes, and further recommendations.

A narrative synthesis was used to report each type of MG event; implementation of the surveillance system; outcomes of the implementation; and the effectiveness, if reported, to summarize the evidence reported in the included studies.

The characteristics of the surveillance systems implemented at each MG event (religious, sporting, and cultural) are shown in Table 4.

The objectives of this review were to (1) identify the different public health digital surveillance systems implemented at MG events for infectious disease prevention and control and (2) assess the evidence on the effectiveness of digital surveillance systems implemented for infectious disease prevention and control at MG events.

We found that the most common types of surveillance systems used in MG events, either independently or with other systems, varied among IBS; EBS; syndromic surveillance; or the use of EI, which consists of IBS and EBS. Of the 8 studies included in this review, 7 described different surveillance systems and web-based tools that were implemented at MG events for infectious disease prevention and control. They reported beneficial outcomes of implementation [21,24,25,27,29-31] (Table 4). Only one study described and evaluated an implemented surveillance system and reported its effectiveness based on attributes relevant to the context of MGs that were measured (sensitivity, positive predictive value, timeliness, acceptability, stability, simplicity, and usefulness) [28]. However, these 7 attributes were not fully reported or measured in 88% (7/8) of the studies [21,24,25,27,29-31] (Table 5). Each attribute is defined in Multimedia Appendix 5 [32]. With only 1 comprehensive report [28] of the evaluation of the implemented surveillance system, there is limited evidence of the effectiveness of public health digital surveillance systems for infectious disease prevention and control at MG events.

The real-time on-site surveillance systems that operated during the MG events illustrated their usefulness in generating valid surveillance data for public health actions that were based on early detection [24]. Although the ability to detect infectious hazards was important during the MGs, the real-time surveillance systems also improved the detection and timeliness of reporting noninfectious health risks during the 2019 Hajj [27].

The enhancement of traditional passive surveillance systems with real-time active surveillance, consisting of IBS with EBS or syndromic surveillance with EBS, proved useful. Early warning signs are essential in MGs for the early detection of disease and timely response to this warning to mitigate the risk of disease spread and outbreaks. These warnings are provided with real-time reporting of both syndromic surveillance and EBS data. The combination of syndromic and EBS complements case-based surveillance systems in MGs. Syndromic surveillance is characterized by the rapid detection of health threats and the response to them, with improvements in emerging disease surveillance. In contrast, EBS triggers public health alerts by collecting unstructured data from various settings through a prediagnostic data analysis. Health care facilities, the media, and the community are examples of settings where event data are collected [27].

Syndromic surveillance data were found to be a powerful tool that, if used efficiently, can better inform health planning and decision-making for purposes other than outbreak detection [29]. In addition, ESS can contribute to the early detection of diseases [30]. Compared with traditional EBS, EBS as a component of EI is different and was considered an adequate safety net that was reliable, reassuring, timely, simple, and stable for the London 2012 Olympic Games [28]. Participatory surveillance provides comprehensive access to a range of data from users worldwide. It involves benefits at a lower cost, timely data acquisition, information collection and sharing, platform scalability, and the capacity for integration between the population being served and public health services [21].

Owing to data extraction and response limitations, the HEWS operated as a near–real-time surveillance system. In a congested MG such as the Hajj where attendees (pilgrims) are mobile, it is crucial to have immediate case identification for better decision-making [27]. In the case of Kumbh Mela, the absence of baseline data did not permit the generation of outbreak thresholds from surveillance data [24].

Mistakes in classification and missing surveillance data because of the closure of health facilities on weekends during the art festival led to potential bias in the surveillance system. Untrained laboratory staff led to miscommunication with the clinical staff at the sentinel sites responsible for responding to outbreaks [30].

For participatory surveillance, apps such as the Healthy Cup might be an effective tool to identify potential threats of outbreaks associated with MGs. However, the attendees’ engagement during the event was low because of exposure bias to the app depending on the users’ age and language barriers restricting app use [21].

Surveillance systems vary in their methods, scope, and objectives [33]. Evaluation of surveillance systems is essential to assess whether the system is meeting its objectives and effectively using available resources [34]. There is limited information on the usefulness of surveillance systems for infectious disease control and outbreak detection [32]. Guidelines for evaluating public health surveillance systems have existed since 1988, with updates by the Centers for Disease Control and Prevention and a framework for evaluating public health surveillance systems for the early detection of outbreaks [32]. Nevertheless, there are no international standards or guidelines for evaluating digital public health surveillance systems for infectious disease prevention and control with a focus on the MG context.

An evaluation of the EBS system implemented at the 2012 London Olympic and Paralympic Games [28] was conducted using the Updated Guidelines for Evaluating Public Health Surveillance Systems from the US Centers for Disease Control and Prevention. A selection of attributes relevant to sporting MG events was reported in a study [28] to assess the system that was implemented based on the sporting context. According to one study [34], some of the systems’ attributes were more critical for selected surveillance purposes and health outcomes than others. This finding indicates that the system-chosen attributes used in the London Games [28] were important and relevant to the nature of the MG event.

In this review, the surveillance systems for the MG events were implemented for similar reasons. The primary and essential aim of the implemented surveillance systems during the MG events was to provide early warning of health events and to detect, identify, and respond to disease outbreaks rapidly. Several studies included in this review (5/8, 62%) reported a vital attribute achieved after implementing or enhancing the existing surveillance system, namely, an improvement in the timeliness of reporting and data sharing [21,25,27,29,30]. However, a precise measurement of this attribute and its assessment were not reported or explained. Timeliness is a key performance measure of public health surveillance systems [35]. Periodic evaluation of such attributes is crucial as it reflects time delays between all the response steps in the public health surveillance process. Severi et al [28] measured the effectiveness of the reported timelines in terms of the time between the entry of a new event into the health protection agency “HP” zone (an electronic public health case management tool used by a local health protection unit) and the time of its report to the EBS. Timeliness is critical when data are to be used to implement immediate disease control and prevention activities for infectious diseases that are acute, severe, and highly transmissible [35].

Disease-specific surveillance, which is the traditional type of surveillance, has existed for many years and was the chief component of public health strategies [14]. This type of surveillance data is considered accurate [36]; however, they do not identify emerging infectious diseases because of limited resources, time, and reporting systems [16]. Timeliness and sensitivity are the most essential features of surveillance systems to detect outbreaks and identify health threats early. The WHO 2005 International Health Regulations [9] emphasize the importance of enhancing the existing web-based tools of surveillance systems to complement traditional passive surveillance systems with active surveillance systems to improve the timeliness and accuracy of the existing surveillance capacity. The enhancement of existing surveillance systems with nontraditional systems, such as EBS, syndromic surveillance, or digital surveillance, can improve the timeliness of reporting. These systems are high in sensitivity but low in specificity compared with traditional surveillance systems [3], and these new systems cannot be considered alternatives or replacements for traditional surveillance systems. However, they are an extension that complements and reinforces the capacity of traditional systems [18]. This characteristic was illustrated in the study by Aggrawal et al [24], where a combination of the 2 types of surveillance was used to complement each other: EI (IBS and EBS).

The review and evaluation of surveillance systems are critical for improving their performance and cost-effectiveness [33]. They enable the continuous improvement of data and system quality. However, with only 12% (1/8) of the studies reporting an evaluation of public health digital surveillance systems for infectious disease prevention and control at MGs, the effectiveness of these systems cannot be determined.

This review highlights the need to develop a comprehensive approach to evaluating digital public health surveillance systems for infectious disease prevention and control at MG events. A comprehensive evaluation approach can be achieved by targeting multiple aspects of digital surveillance systems’ characteristics. This process can begin before deployment by applying a systems engineering approach to identify the challenges of surveillance system implementation and effectiveness. The assessment should cover the social and economic aspects of the intervention or system features in addition to the epidemiological aspects. Simulations could be used to identify the key performance characteristics of the components of the system that contribute the most to overall effectiveness. This approach will provide a complete understanding of the multiple factors and constraints that stakeholders need to consider when implementing a digital surveillance system for an MG event. It will highlight the key determinants for the successful implementation of disease surveillance systems, including effectiveness and cost-effectiveness. It is recommended that the design of such an evaluation be flexible to the context of the surveillance systems as well as operational.

This review’s findings revealed limited evidence for the availability of standard international guidelines for evaluating public health digital surveillance systems for infectious disease prevention and control in the MG context. Moreover, most studies (7/8, 88%) did not report evaluations of digital public health surveillance systems for infectious disease prevention and control that were implemented at MG events.

Another standardized guideline or framework is needed to evaluate digital surveillance systems in MGs for infectious disease prevention and control.

Additional reporting guidelines for interventional studies should provide a minimum amount of information needed to report the implementation of digital public health surveillance systems at MG events. This will facilitate the reporting by surveillance systems at MGs and increase the number of evaluation studies.

A quality assessment tool is necessary for interventional studies that evaluate digital public health surveillance systems for infectious disease prevention and control at MGs. This tool will facilitate the assessment of interventional studies’ reports of surveillance systems at MG events.

The perspectives of stakeholders of MGs on the performance of existing surveillance systems and their intentions to evaluate digital public health surveillance systems should be investigated.

This research is the first systematic review that highlights the gap in the literature on evidence related to the effectiveness of digital public health surveillance systems implemented at MGs for infectious disease prevention and control and the need to establish a standard guideline for such evaluations. Given the increase in emerging infectious diseases such as COVID-19, which affected countries and MG events worldwide, new technologies and approaches to detection have been developed. Therefore, evaluating surveillance systems and new digital technologies implemented at MG events for infectious disease prevention and control is crucial for measuring their effectiveness. We developed a new appraisal tool for this review, but it has not been validated.

This study has several limitations. First, only 4 databases were searched. Although these 4 databases are the most relevant sources of health informatics publications, it is possible that relevant studies were missed. Although gray literature would have retrieved studies that might have met the eligibility criteria, it was not included because of the absence of peer-reviewed studies. Second, the screening of the titles and abstracts of the studies retrieved from the databases was performed by a single author. However, we believe that this did not affect the quality of this study. Furthermore, ratings of the quality of the included studies based on the developed appraisal tool varied between low and moderate because of the absence of studies reporting evaluations of the surveillance systems implemented at MGs. The number of studies included in this review might be considered low because of the limited number of studies that met the eligibility criteria of studies evaluating digital surveillance systems for infectious disease prevention and control at MG events to measure effectiveness. However, we believe that these limitations do not affect the validity of our findings.

Effective and timely communicable disease control relies on effective and timely disease surveillance. The nature of emerging infectious diseases often limits the effectiveness of traditional surveillance systems. Digital surveillance could improve the sensitivity and timeliness of the detectors of health events and facilitate responses to emerging diseases [18]. Hence, the risk of transmission of infectious diseases during MG events remains.

There is sparse evidence of public health digital surveillance systems as an effective method for infectious disease prevention and control at MG events owing to the limited number of evaluation studies.

This finding indicates a methodological challenge in evaluating surveillance systems at MGs because of a lack of available guidelines for evaluating digital surveillance systems for infectious disease prevention and control at MG events. Hence, further research on the effectiveness of public health digital surveillance systems for infectious disease prevention and control is necessary.