This paper is in the following e-collection/theme issue:

Clinical Information and Decision Making (1373) Influenza and Influenza-Like-Illness (ILI) Surveillance (214) Machine Learning (1433) Artificial Intelligence (1323) Infectious Diseases (non-STD/STI) (1555)

Published on in Vol 25 (2023)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/44238, first published .
Deep-Learning Model for Influenza Prediction From Multisource Heterogeneous Data in a Megacity: Model Development and Evaluation

Deep-Learning Model for Influenza Prediction From Multisource Heterogeneous Data in a Megacity: Model Development and Evaluation

Deep-Learning Model for Influenza Prediction From Multisource Heterogeneous Data in a Megacity: Model Development and Evaluation

Authors of this article:

Liuyang Yang1, 2 Author Orcid Image ;   Gang Li3 Author Orcid Image ;   Jin Yang2 Author Orcid Image ;   Ting Zhang2 Author Orcid Image ;   Jing Du3 Author Orcid Image ;   Tian Liu4 Author Orcid Image ;   Xingxing Zhang2 Author Orcid Image ;   Xuan Han2 Author Orcid Image ;   Wei Li5 Author Orcid Image ;   Libing Ma2, 6 Author Orcid Image ;   Luzhao Feng2 Author Orcid Image ;   Weizhong Yang2 Author Orcid Image
Article Authors Cited by (7) Tweetations (5) Metrics

    Original Paper

    1Department of Management Science and Information System, Faculty of Management and Economics, Kunming University of Science and Technology, Kunming, China

    2School of Population Medicine and Public Health, Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing, China

    3Beijing Centre for Disease Prevention and Control, Beijing, China

    4Jingzhou Center for Disease Control and Prevention, Jingzhou, China

    5The First People’s Hospital of Yunnan Province, The Affiliated Hospital of Kunming University of Science and Technology, Kunming, China

    6Department of Respiratory and Critical Care Medicine, Affiliated Hospital of Guilin Medical University, Guilin, China

    *these authors contributed equally

    Corresponding Author:

    Weizhong Yang, MM

    School of Population Medicine and Public Health

    Chinese Academy of Medical Sciences & Peking Union Medical College

    No. 9, Dongdan Santiao, Dongcheng District, Beijing

    Beijing, 100730

    China

    Phone: 86 1 391 181 9068

    Email: yangweizhong@cams.cn


    Background: In megacities, there is an urgent need to establish more sensitive forecasting and early warning methods for acute respiratory infectious diseases. Existing prediction and early warning models for influenza and other acute respiratory infectious diseases have limitations and therefore there is room for improvement.

    Objective: The aim of this study was to explore a new and better-performing deep-learning model to predict influenza trends from multisource heterogeneous data in a megacity.

    Methods: We collected multisource heterogeneous data from the 26th week of 2012 to the 25th week of 2019, including influenza-like illness (ILI) cases and virological surveillance, data of climate and demography, and search engines data. To avoid collinearity, we selected the best predictor according to the weight and correlation of each factor. We established a new multiattention-long short-term memory (LSTM) deep-learning model (MAL model), which was used to predict the percentage of ILI (ILI%) cases and the product of ILI% and the influenza-positive rate (ILI%×positive%), respectively. We also combined the data in different forms and added several machine-learning and deep-learning models commonly used in the past to predict influenza trends for comparison. The R2 value, explained variance scores, mean absolute error, and mean square error were used to evaluate the quality of the models.

    Results: The highest correlation coefficients were found for the Baidu search data for ILI% and for air quality for ILI%×positive%. We first used the MAL model to calculate the ILI%, and then combined ILI% with climate, demographic, and Baidu data in different forms. The ILI%+climate+demography+Baidu model had the best prediction effect, with the explained variance score reaching 0.78, R2 reaching 0.76, mean absolute error of 0.08, and mean squared error of 0.01. Similarly, we used the MAL model to calculate the ILI%×positive% and combined this prediction with different data forms. The ILI%×positive%+climate+demography+Baidu model had the best prediction effect, with an explained variance score reaching 0.74, R2 reaching 0.70, mean absolute error of 0.02, and mean squared error of 0.02. Comparisons with random forest, extreme gradient boosting, LSTM, and gated current unit models showed that the MAL model had the best prediction effect.

    Conclusions: The newly established MAL model outperformed existing models. Natural factors and search engine query data were more helpful in forecasting ILI patterns in megacities. With more timely and effective prediction of influenza and other respiratory infectious diseases and the epidemic intensity, early and better preparedness can be achieved to reduce the health damage to the population.

    J Med Internet Res 2023;25:e44238

    doi:10.2196/44238

    Keywords

    influenza; ILI; multisource heterogeneous data; deep learning; MAL model; megacity 


    Acute respiratory infectious diseases have long represented a threat to human health. Every large-scale outbreak of acute infectious diseases will have a significant impact on human life, health, and social development [1]. On March 11, 2020, the World Health Organization (WHO) officially declared COVID-19 as a global pandemic. To date, there have been 613,410,796 confirmed COVID-19 cases and 6,518,749 deaths worldwide [2]. In addition, an influenza pandemic may occur every 10-50 years [3]. Influenza viruses readily mutate and spread quickly. Influenza epidemics, outbreaks, and even pandemics have always threatened human health and socioeconomic development. The Spanish influenza (H1N1) outbreak in 1918 caused approximately 50 million deaths worldwide, the Asian influenza (H2N2) outbreak in 1957 caused approximately 1-2 million deaths, Hong Kong influenza (H3N2) in 1968 caused approximately 1 million deaths, and influenza A (H1N1) in 2009 caused approximately 200,000 deaths [4-7].

    Owing to the constant variation in viruses and increased global connectivity, new respiratory infectious diseases have caused a heavy disease burden and economic losses to humanity. Especially in megacities, with the growth of the population and the expansion of urbanization, the contradiction between humans and animals has become increasingly significant. In addition, aging issues in megacities are of concern, with older individuals being more likely to become infected with serious acute respiratory infectious diseases. The rapid urbanization process in megacities leads to population aggregation and residential congestion. Moreover, air pollution, climate warming, and other issues in megacities have created new opportunities for the emergence of infectious diseases [8]. Therefore, in megacities, there is an urgent need to establish more sensitive forecasting and early warning methods for acute respiratory infectious diseases to monitor their epidemic trends and minimize losses.

    Seasonal influenza has long been one of the major problems of public health [9]. The spread and prevalence of influenza are affected by many factors, including climate, population mobility, population immunity, and social economy. In past research, multisource data have been used for improved modeling of disease outbreaks. Ginsberg et al [10] proposed using Google search engine data to estimate the trend of influenza outbreaks, finding a high correlation between the disease search trend and the actual number of influenza infections. Subsequently, researchers began using similar data sources, including Yahoo, Wikipedia, Baidu search, and other platforms, to establish prediction models for influenza and other acute respiratory infectious diseases [11]. Some studies also found that environmental factors such as absolute humidity, temperature, and sunlight were helpful for the prediction of influenza [12,13]. China has established a national influenza surveillance system, which plays an important role in influenza surveillance. However, the national influenza surveillance system mainly analyzes confirmed influenza cases, excluding pharmacy, meteorological, and other data that may affect influenza. Therefore, the current system may have a lag in the detection of an influenza epidemic.

    Surveillance approaches of influenza usually include time-series analysis, time-space analysis, infectious disease dynamics, and machine-learning and deep-learning methods [14]. Different approaches have their own advantages and disadvantages. For example, the time-series method is effective in the short-term prediction of an influenza trend, but may not be suitable for the long-term prediction of influenza [15]. The dynamic model of infectious disease can better predict the trend of an influenza epidemic from the aspect of the transmission mechanism; however, the process of model establishment is complex, and population mobility and other factors need to be considered [16]. Machine learning can process a large amount of data in a short time, which is a fast and accurate approach. Cheng et al [17] used machine-learning methods (autoregressive integrated moving average, random forest, support vector regression, and extreme gradient boosting [XGBoost]) to accurately predict the trends of influenza-like illness (ILI) in Taiwan. Choo et al [18] established a more sensitive influenza screening model by using the gated current unit (GRU) model. Jang et al [19] used news data to predict influenza, and the results showed that the long short-term memory (LSTM) model could accurately predict influenza. However, most traditional machine-learning methods may not be equipped to use multisource heterogeneous data to effectively complete prediction tasks. Therefore, although the modeling process of deep learning is also complex, it may be able to solve the problem of incorporating multisource heterogeneous data.

    In this study, we integrated heterogeneous data from different sources such as ILI case data from hospitals, climate data, search engine data, and social economy data, because data from different sources may capture different influenza incidence signals in the population to varying degrees. The aim was to establish a new and better-performing deep-learning model to predict influenza trends from multisource heterogeneous data in a megacity.


    Study Design

    In this study, we chose Beijing as the research object to represent a megacity, because Beijing is the capital of the People’s Republic of China, a central national city. By the end of 2021, Beijing had more than 20 million permanent residents. Furthermore, the center of Beijing is located at a longitude of 116°20 east and a latitude of 39°56 north. Influenza is more prevalent in winter with significant peaks in this season. We first collected multisource heterogeneous data from the 26th week of 2012 to the 25th week of 2019, including ILI cases, data of virological surveillance, climate and demography data, and data from search engines. We then aligned these multisource heterogeneous data with time as the label. Simultaneously, since we observed that the absolute values of some dimensional data were not on the same order of magnitude, we normalized all data to be between 0 and 1 for further analysis and training. Because some variables may be collinear or strongly correlated, we performed correlation analysis and weight analysis to select the best data set before using the data for modeling. Finally, we performed deep-learning modeling for the filtered data of the percentage of ILI cases (ILI%) and the product of ILI% and the influenza positive rate (ILI%×positive%). We used data from the 26th week of 2012 to the 45th week at the end of 2018 to train the model, predicted the data from the 46th week of 2018 to the 25th week of 2019, and compared the obtained data with the actual data to observe the model’s fitting effect. The specific modeling process is described in detail in the Procedure section below.

    Data Collection

    ILI Cases

    The Beijing Municipal Influenza Monitoring System conducts case screening for patients in the fever clinic, internal medicine clinic, internal medicine emergency department, children’s internal medicine clinic, and children’s internal medicine emergency department of sentinel hospitals; registers those who meet the definition of influenza-like cases; and summarizes the number of influenza-like cases in each age group (0-4, 5-14, 15-24, 25-59, and ≥60 years) every day. An ILI case is defined as a patient with fever (axillary temperature≥38 ℃), cough, or sore throat. In this study, the monitoring data of ILI cases reported by more than 140 medical institutions above the third level in the Beijing Medical Institutions Infectious Disease Monitoring and Early Warning System were used for analysis. We collected ILI data from the 26th week of 2012 to the 25th week of 2019. The ILI% was calculated as the proportion of ILI case reports relative to the total number of outpatient and emergency visits in the same period.

    Virological Surveillance

    The national influenza surveillance network in China is led by the Chinese Center for Disease Control and Prevention (CDC), with 554 sentinel hospitals and 407 network laboratories. The network laboratory conducts pathogenic monitoring of influenza virus on the respiratory specimens of ILI cases and monitors the activity level and variation of influenza virus. We also obtained influenza virological surveillance data from the 26th week of 2012 to the 25th week of 2019. The average weekly positive influenza test rate was obtained by dividing the number of influenza-positive samples by the total number of samples tested during the week. We used ILI%×positive% to express the intensity of influenza activity during the study period.

    Climate and Demography

    Climate data were obtained from the China Meteorological Data Sharing Service System [20]. We collected meteorological data from Beijing from the 26th week of 2012 to the 25th week of 2019. The data comprised the mean temperature per week (Tmean), maximum temperature per week (Tmax), minimum temperature per week (Tmin), mean relative humidity (RH) per week, mean air pressure per week (Apmean), mean wind speed per week, mean precipitation per week, and mean sunlight hours per week. The absolute humidity (AH) calculation formula was as follows: AH={6.112×e[(17.67×T)/(T±243.5)]×RH×2.1674}/(273.15+T) [21,22]. We defined the weekly temperature difference as Tmax–Tmin. Air quality data were obtained from the Beijing Municipal Environmental Monitoring Center [23]. We also collected the mean weekly air quality data for Beijing from the 26th week of 2012 to the 25th week of 2019. Air quality was divided into six grades according to the air quality index (AQI): Grade I (excellent, AQI=0-50), Grade II (good, AQI=51-100), Grade III (slight pollution, AQI=101-150), Grade IV (moderate pollution, AQI=151-200), Grade V (severe pollution, AQI=201-300), and Grade VI (severe pollution, AQI>300). We collected the gross domestic product (GDP) data of Beijing from 2012 to 2019 and information on statutory holidays from the 26th week of 2012 to the 25th week of 2019 [24]. According to the Chinese lunar calendar, we set a holiday week at 1 and a week without a holiday at 0.

    Search Engine

    The Baidu search engine is one of the most widely used search engines in China. The Baidu search index represents the keyword search trend of many Baidu internet users, which can be used to understand internet users’ concerns and monitor public opinion trends [25]. We selected “influenza” as the keyword and the time range from the 26th week of 2012 to the 25th week of 2019. The search data were based on personal computer and personal mobile phone data.

    Procedure

    Normalization, Correlation, and Weight Analysis

    We normalized all data to fit in the range of 0-1. Because some dimensional data may have collinearity or strong correlations, to avoid the impact of collinearity on the model, we performed correlation analysis and weight analysis on all variables to select the best data set for training. The Pearson correlation coefficient (r) was used to measure the linear correlation in which a greater absolute value of the correlation coefficient indicates a stronger correlation; that is, a correlation coefficient closer to 1 or –1 indicates a stronger correlation and a correlation coefficient closer to 0 indicates a weaker correlation. We calculated the Pearson correlation coefficient between each variable to observe the correlations between the variables. As a traditional machine-learning model, the random forest model can score the importance of different variables and dimensional characteristics, and further evaluates the contribution of each independent variable to the dependent variable results. We constructed a random forest model to explore the influence weight of independent variables on ILI% and ILI%×positive%.

    Model Construction

    In this study, we aimed to fully explore the inherent characteristics of multisource heterogeneous data and establish the mapping from characteristics to results through building and training models to effectively predict ILI% and ILI%×positive%.

    Based on LSTM, we innovatively built a new multiattention deep-learning model (MAL) (Figure 1). We first set up an LSTM layer. Subsequently, three different attention modules were connected in parallel: Channel Attention, Spatial Attention, and the concatenation of Channel Attention and Spatial Attention. Different attention modules not only focuses the model on key nodes in the time series but also focuses the model on features with high weight, which can promote the improvement of model performance. We then added the flatten layers to fully obtain the information after passing the different attention modules and concatenating the three branches. To prevent the gradient from disappearing, the second concatenate was conducted after the first concatenate, with the two branches passing through the global pooling layer. Finally, a full connection was made through the dense layer, and the predicted values of ILI% and ILI%×positive% were obtained as output.

    ‎
    Figure 1. Structure of the multiattention deep-learning model (MAL) based on long short-term memory (LSTM). Schematic diagram representing multisource heterogeneous data input to the MAL model, including climate, Baidu searrch, demography, and other data. The curves of five independent variables are displayed: maximum temperature per week (Tmax), mean relative humidity per week (Hmean), mean precipitation per week, the mean temperature difference per week (dT; Tmax–Tmin), and air quality index (AQI).

    We used the data from the 26th week of 2012 to the 45th week at the end of 2018 to conduct the training model, predicted the data from the 46th week of 2018 to the 25th week of 2019, and finally compared the predicted data with the actual data to observe the model fitting effect. The following parameters were adopted for model evaluation: R2, explained variance score, mean absolute error, and mean squared error. An R2 and explained variance score close to 1 indicate a good prediction effect of the model. A mean absolute error and mean squared error close to 0 further indicate the good prediction effect of the model.

    Model Comparison

    To evaluate the scientific validity and robustness of the selected data set and model constructed, we compared the performance of the MAL models built with the data from ILI%+climate, ILI%+demography, ILI%+Baidu, ILI%+climate+Baidu, ILI%+climate+demography, ILI%+demography+Baidu, and ILI%+climate+demography+Baidu to verify whether the prediction effect was improved after data from different sources were fused into a model. Similarly, we compared the performance of the MAL models for the data from ILI%×positive%+climate, ILI%×positive%+demography, ILI%×positive%+Baidu, ILI%×positive%+climate+Baidu, ILI%×positive%+climate+demography, ILI%×positive%+demography+Baidu, and ILI%×positive%+climate+demography+Baidu.

    Several common models such as random forest, XGBoost, GRU, and LSTM were also used for comparison with our MAL model. Random forest refers to a classifier that uses multiple trees to train and predict samples. For many data forms, random forest can produce a classifier with high accuracy. XGBoost is an integrated machine-learning algorithm based on a decision tree, which is suitable for classification, regression, sorting, and other problems. This method is fast and effective, can handle large-scale data, and supports user-defined loss functions. GRU is a gating mechanism in recurrent neural networks and can be applied to short-term or long-term prediction. LSTM is a time-recurrent neural network of a deep-learning model that can effectively solve the problem of gradient explosion or disappearance of a simple circular neural network, and is suitable for processing and predicting notable events with long intervals and delays in time series.

    Statistical Analysis

    We performed all statistical analyses with Python (version 3.6.0) and Tensorflow (version 2.0.0) software.

    Ethics Considerations

    This study was approved by the Ethics Committee of the Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing, China (review number CAMS&PUMC-IEC-2021-032).


    Description of Data From Various Sources

    We calculated the weekly average of data collected from various sources at different times. The average weekly number of ILI cases was the highest from the 26th week in 2018 to the 25th week in 2019, with 21,142 cases. We also calculated other variables as shown in Table 1.

    We built random forest models to explore the impact weight of different source variables on ILI% and ILI%×positive%. We found that Baidu’s “influenza” search index had the highest correlation with ILI% and AQI had the highest impact weight with ILI%×positive% (Table 2). To avoid collinearity between variables, we performed a correlation analysis of different variables. Tmean, Tmax, Tmin, and Apmean were highly correlated (absolute value of r>0.9; Figure 2). Therefore, we used Apmean with the highest weight in the modeling process for predicting ILI% and removed Tmean, Tmax, and Tmin, and used Tmean with the highest weight in the modeling process for predicting ILI%×positive% and removed Tmax, Tmin, and Apmean.

    Table 1. Average value of different source variables for different time periods.a
    Variables2012-20132013-20142014-20152015-20162016-20172017-20182018-2019
    ILIb cases, n13,74812,30012,35113,88714,31819,10021,142
    Influenza-positive cases, n26.0037.0047.0039.0030.0045.0045.00
    Maximum temperature (℃)16.3218.5518.4218.2018.6417.9718.35
    Minimum temperature (℃)5.456.936.586.857.066.326.17
    Temperature difference (℃)10.8711.6211.8411.3511.5811.6512.17
    Mean temperature (℃)10.5312.2912.1212.2212.4611.7711.90
    Relative humidity (g/m3)58.4850.0550.8354.9455.4753.7351.30
    Atmosphere pressure (hPa)993.18992.97993.46993.75993.77993.48994.01
    Wind speed (m/s)1.8590.171.761.771.661.691.70
    Precipitation (cm)1.671.661.221.541.561.381.27
    Sunlight (weeks)6.516.676.636.526.676.977.02
    AQIc70.1964.4270.19100.38159.13108.65115.38
    Baidu search index for “influenza”117.00205.00218.00215.00284.00428.00443.00

    aIn each time period, data were collected from the 26th week of the first year to the 25th week of the subsequent year.

    bILI: influenza-like illness.

    cAQI: air quality index.

    Table 2. Impact weight of variables from different data sources on the proportion of influenza-like illness cases (ILI%) and the product of influenza-like illness cases and the influenza positive rate (ILI%×positive%).
    VariablesWeight

    		ILIILI×positive
    Search index0.0850.081
    Weekly temperature difference0.0830.072
    Sunlight0.0810.066
    Mean air pressure per week0.0770.074
    Mean relative humidity per week0.0760.067
    Mean wind speed per week0.0740.069
    Mean temperature per week0.0730.083
    Air quality index0.0720.088
    Minimum temperature per week0.0700.080
    Maximum temperature per week0.0690.073
    Week0.0660.067
    Precipitation0.0600.052
    Gross domestic product0.0570.061
    Year0.0460.057
    Holiday0.0110.010
    ‎
    Figure 2. Pearson correlation analysis of different variables. Apmean: mean air pressure per week; Aqi: air quality index; dT: the weekly temperature difference; GDP: gross domestic product; Hmean: mean relative humidity per week; Spmean: mean wind speed per week; Tmax: maximum temperature per week; Tmean: mean temperature per week; Tmin: minimum temperature per week.

    MAL of ILI% and Other Source Data

    We first used the MAL to model ILI% and then combined and modeled ILI% with climate, demography, and Baidu data in different forms (ILI%+climate, ILI%+demography, ILI%+Baidu, ILI%+climate+demography, ILI%+climate+Baidu, ILI%+demography+Baidu, and ILI%+climate+demography+Baidu) to compare the effects of the different combinations. The explained variance score of ILI%+climate+demography+Baidu reached 0.78 and the R2 reached 0.76, which were higher than the corresponding values obtained for the other combinations. Similarly, the mean absolute error of ILI%+climate+demography+Baidu was 0.08 and the mean squared error was 0.01, which were lower than the corresponding values of the other combinations (Table 3). This showed that the ILI%+climate+demography+Baidu model had the best prediction effect. We then drew a prediction diagram for the ILI%+climate+demography+Baidu model. These results also showed that the model can accurately predict the two peaks from the 46th week in 2018 to the 25th week in 2019 (Figure 3).

    Table 3. Multiattention-long short-term memory (MAL) deep-learning model of influenza-like illness case proportion (ILI%) in various combinations with other source data.
    Data source combinationsR2Explained variance scoreMean absolute errorMean squared error
    ILI%0.63650.73310.11070.0228
    ILI%+climate0.67020.69960.08960.0207
    ILI%+demography0.68300.75040.09550.0199
    ILI%+Baidu0.70220.74550.09360.0187
    ILI%+climate+demography0.69520.70370.09160.0191
    ILI%+climate+Baidu0.69980.74380.07570.0188
    ILI%+demography+Baidu0.68010.76600.08930.0201
    Alla0.76380.78010.07650.0148

    aAll data sources included: ILI%+climate+demography+Baidu.

    ‎
    Figure 3. Prediction diagram for the ILI%+climate+demography+Baidu multiattention long short-term memory (MAL) deep-learning model. ILI%: percentage of influenza-like illness cases.

    MAL of ILI%×Positive% and Other Data Sources

    We first used the MAL to model ILI%×positive% alone, and then combined and modeled ILI%×positive% with climate, demography, and Baidu data in different forms (ILI%×positive%+climate, ILI%×positive%+demography, ILI%×positive%+Baidu, ILI%×positive%+climate+demography, ILI%×positive%+climate+Baidu, ILI%×positive%+demography+Baidu, and ILI%×positive%+climate+demography+Baidu) to compare the effects of the different combinations. The explained variance score of ILI%×positive%+climate+demography+Baidu reached 0.74 and the R2 reached 0.70, which were higher than the corresponding values of the other combinations. Similarly, the mean absolute error of ILI%×positive%+climate+demography+Baidu was 0.02 and the mean squared error was 0.02, which were lower than the corresponding values of the other combinations (Table 4). This showed that the ILI%×positive%+climate+demography+Baidu model had the best prediction effect. We then drew a prediction diagram for the ILI%×positive%+climate+demography+Baidu model. The results also showed that the model can accurately predict the two peaks from the 46th week in 2018 to the 25th week in 2019 (Figure 4).

    Table 4. Multiattention-long short-term memory (MAL) deep-learning model of the product of the percentage of influenza-like illness cases and influenza-positive rate (ILI%×positive%) in combination with other source data.
    Data source combinationsR2Explained variance scoreMean absolute errorMean squared error
    ILI%×positive%0.55930.57190.11280.0234
    ILI%×positive%+climate0.66090.66100.08860.0180
    ILI%×positive%+demography0.65560.69420.09500.0183
    ILI%×positive%+Baidu0.63750.65390.09820.0192
    ILI%×positive%+climate+demography0.66130.66900.09110.0179
    ILI%×positive%+climate+Baidu0.64490.70710.09410.0188
    ILI%×positive%+demography+Baidu0.67880.68290.08980.0170
    Alla0.70250.73960.01580.0161

    aAll data sources included: ILI%×positive%+climate+demography+Baidu.

    ‎
    Figure 4. Prediction diagram for ILI%*positive%+climate+demography+Baidu multiattention long short-term memory (MAL) deep-learning model. ILI%*positive%: product of the percentage of influenza-like illness cases and the influenza-positive rate.

    Comparison of ILI% and Other Source Data Between Different Models

    We modeled different combinations of ILI% and other data sources, and compared the performance of traditional machine-learning models (random forest and XGBoost) and other deep-learning models (LSTM and GRU) with that of our MAL model. The results showed that the MAL model with the ILI%+climate+demography+Baidu combination had the best prediction effect. The R2 was 0.76 and the explained variance score was 0.78, which were higher than the corresponding values of the other models; the mean absolute error was 0.08 and the mean squared error was 0.01, which were lower than the corresponding values of the other models (Table 5).

    Table 5. Comparison of the percentage of influenza-like illness cases (ILI%) and combinations of other source data between different models.
    Model performanceILI%ILI%+climateILI%+demographyILI%+BaiduILI%+climate+demographyILI%+climate+BaiduILI%+demography+BaiduAll
    R2

    		RFa–0.05640.01210.0657–0.02190.13450.10620.08350.0531

    		XGBb0.38410.34250.38270.31970.37430.32760.31530.3327

    		LSTMc0.58910.61020.59190.56210.56610.57180.57050.6116

    		GRUd0.47750.53870.52320.52630.52060.57680.57810.6461

    		MALe0.63650.67020.68310.70220.69520.69980.68010.7638
    Explained variance score

    		RF0.33490.37350.42710.31780.47780.46030.45240.3449

    		XGB0.45670.41800.45450.41620.43720.41080.41570.4102

    		LSTM0.72830.73530.68330.69740.73360.71310.69550.6901

    		GRU0.60270.61950.62290.68760.62920.66860.68210.7006

    		MAL0.73310.69960.75040.74550.70370.74380.76600.7801
    Mean absolute error

    		RF6.40626.21876.21886.46876.06256.21876.34376.4062

    		XGB4.85224.97814.89805.04734.94314.99635.10844.9209

    		LSTM0.11560.10620.10480.10930.11380.10660.10430.0947

    		GRU0.12050.11010.11310.11710.10680.10340.10820.1037

    		MAL0.11070.08960.09550.09360.09160.07570.08930.0765
    Mean squared error

    		RF110.7812103.593797.9687107.156290.750193.718796.093799.2812

    		XGB64.592768.939664.729671.325765.606970.505471.793669.9706

    		LSTM0.02580.02440.02560.02750.02720.02680.02690.0243

    		GRU0.03280.02890.02990.02970.03010.02650.02650.0222

    		MAL0.02280.02070.01990.01870.01910.01880.02010.0148

    aRF: random forest.

    bXGB: extreme gradient boosting.

    cLSTM: long short-term memory.

    dGRU: gated current unit.

    eMAL: multiattention-long short-term memory.

    Comparison of ILI%×Positive% and Other Source Data Between Different Models

    We modeled different combinations of ILI%×positive% and other data sources, and compared the performance of traditional machine-learning models (random forest and XGBoost) and other deep-learning models (LSTM and GRU) with that of our MAL model. The results showed that the MAL with the ILI%×positive%+climate+demography+Baidu combination had the best prediction effect. The R2 was 0.70 and the explained variance score was 0.74, which were higher than the values of the other models; the mean absolute error was 0.02 and the mean squared error was 0.02, which were lower than the values of the other models (Table 6).

    Table 6. Comparison of the product of the proportion of influenza-like illness cases and influenza-positive rate (ILI%×positive%) in combination with other data sources between different models.
    Model performanceILI%×positive%ILI%×positive%+climateILI%×positive%+demographyILI%×positive%+BaiduILI%×positive%+climate+demographyILI%×positive%+climate+BaiduILI%×positive%+demography+BaiduAll
    R2

    		RFa0.01740.0351–0.09750.0548–0.06620.10660.05630.2482

    		XGBb0.45590.46740.48430.62990.45780.58560.61310.6241

    		LSTMc0.47960.49330.50370.47810.59660.59760.60260.6321

    		GRUd0.44020.51780.50820.48170.54010.53960.52260.6322

    		MALe0.55930.66090.65560.63750.66130.64490.67880.7025
    Explained variance score

    		RF0.32680.37210.29850.32730.28790.42130.37110.3615

    		XGB0.46170.46780.48860.63570.46170.58590.62590.6305

    		LSTM0.49960.49380.65010.62250.68010.65680.60380.6679

    		GRU0.56650.63040.61910.59040.56070.61620.60460.6381

    		MAL0.57190.66100.69420.65390.66900.70710.68290.7396
    Mean absolute error

    		RF4.06254.15624.46874.21874.25003.96874.03123.4687

    		XGB2.84463.10222.83042.50673.04272.78292.52612.5808

    		LSTM0.11460.11070.11720.12210.09510.10450.10220.0868

    		GRU0.11340.10010.10150.11640.10810.11850.10970.0877

    		MAL0.11280.08860.09500.09820.09110.09410.08980.0158
    Mean squared error

    		RF40.312539.593745.031238.781243.750036.656238.718730.8437

    		XGB22.324321.850021.157915.183922.242516.999115.874115.4241

    		LSTM0.02750.02680.02630.02760.02130.02130.02110.0195

    		GRU0.02960.02550.02600.02740.02430.02440.02530.0194

    		MAL0.02340.01800.01830.01920.01790.01880.01700.0161

    aRF: random forest.

    bXGB: extreme gradient boosting.

    cLSTM: long short-term memory.

    dGRU: gated current unit.

    eMAL: multiattention-long short-term memory.


    Principal Findings

    Our research purpose was to explore a new, better-performing deep learning model to predict influenza trends from multisource heterogeneous data in a megacity. The highest correlation coefficients were found for Baidu query data for ILI% and air quality for ILI%×positive%. In addition to these variables, temperature, sunlight, air pressure, humidity, and wind speed had high correlation coefficients for ILI%, whereas temperature, Baidu query data, air pressure, wind speed, and humidity had high correlations for ILI%×positive%. We established the MAL model and predicted the ILI% and ILI%×positive%. We also combined the data in different forms and evaluated the performance of several common machine-learning and deep-learning models to predict influenza trends for comparison. We first used the MAL to model ILI% and then combined and modeled ILI% with climate, demography, and Baidu data in different forms. We found that the ILI%+climate+demography+Baidu combination had the best prediction effect. Similarly, we used the MAL to model ILI%×positive% and combined this with different data forms. We found that the ILI%×positive%+climate+demography+Baidu combination had the best prediction effect. The comparison with random forest, XGBoost, LSTM, and GRU models showed that the newly established MAL model had the best prediction effect using multisource data.

    The combination of multisource heterogeneous data and multichannel surveillance could reduce forecasting errors introduced from a prediction based purely on internet-derived data or climatic data. The fact that the models combining ILI% or ILI%×positive% with climate, demography, and Baidu search data had the best prediction effect indicated that the emergent risk of seasonal influenza could be assessed using multiple forms of surveillance data. Both internet-based query and climate data have been previously suggested in developing predictive models for climate-sensitive infectious diseases based on spatiotemporal models [26]; traditional surveillance data are necessary and the moderate symptomatic surveillance system could be further exploited. The WHO has developed “pandemic and epidemic intelligence” as a new model for the surveillance of emerging threats, which is expected to expand for increased anticipation and preparation for future threats. Traditional surveillance is inadequate when only data for confirmed cases are available [27]. Risk factors and the absence of symptomatic data lead to reduced detection of multipoint triggering mechanisms for infectious diseases. Therefore, moderate surveillance channels must be created to increase detection sensitivity and accuracy.

    In our results, the search engine data had one of the highest correlation coefficients for ILI% and proved to be a powerful variable for estimating epidemic status. During the current COVID-19 pandemic, Twitter could estimate the prevalence of COVID-19 in the United States [28]. Baidu data could also be used to forecast dengue fever. However, studies have also shown that some search engines may overestimate peak amplitude [29]. Therefore, investigating the regulations of various combined keywords and search policies in search engine data by using machine-learning methods will be necessary in future studies. Based on the findings of this study, we also encourage the use of search engine query surveillance, particularly in developing countries and regions with the greatest number of internet users, such as Asia (53.6%) and Africa (11.9%) [30].

    Natural factors may substantially influence the epidemic of respiratory infections more than social factors. Our study showed that natural factors had a greater impact weight on ILI cases, whereas economic factors such as GDP, holidays, and social factors had a lower impact weight on ILI cases. Several studies have indicated that climate is the key factor in forecasting climate-sensitive infectious diseases such as dengue and other vector-borne diseases. Seasonal influenza is also a climate-sensitive disease. However, social measures have remained stable. Therefore, we have not found a significant impact of social factors, because the COVID-19 pandemic response provided an opportunity to identify the social public health measures that have a significant impact on influenza protection [31-33].

    Deep learning is a new aspect of machine learning. In recent years, deep learning has made unprecedented achievements in classification, detection, recognition, prediction, and other tasks, and has attracted extensive attention from all levels of society. Deep learning can solve problems that are difficult to solve by traditional machine learning, such as high-dimensional and jumbled data. Owing to the COVID-19 epidemic, there has been a gradual increase in exploration and research on the prediction and early warning of respiratory infectious diseases using deep learning. Yi et al [34] improved pneumonia surveillance ability using a revolutionary, scalable, and interpretable deep neural network. Jung et al [35] used a deep-learning model based on self-attention to predict influenza in a region, and the results showed that the model was effective in predicting the trend of influenza. An overview study summarized the application of existing deep-learning and medical image analysis methods, systematically discussed the problems associated with deep-learning methods and the COVID-19 imaging mode, and reported several promising research results [36].

    In the past, the influenza surveillance of the CDC in China was mainly conducted using only confirmed case data; hence, there was a time lag in influenza surveillance. In the future, we should prepare for respiratory infectious disease epidemics, especially emerging acute respiratory infectious diseases. Therefore, it is important to improve and optimize the prediction and early warning system of respiratory infectious diseases. Researchers began to use machine learning, deep learning, artificial intelligence, and other methods in combination with big data to establish multichannel and multisource prediction and early warning models. Furthermore, it is essential to establish a trinity working system of detection, decision-making, and early warning. In this study, we first used ILI cases, search engine, and other data, which occurred before the case was confirmed, to predict the outbreak of influenza in advance. In addition, owing to the diversity of data sources, it is difficult for traditional models to simultaneously input multiple variables for prediction. Therefore, we innovatively established a deep-learning model (MAL), which can model ILI cases and other monitoring data from different sources at the same time. The results also showed that this model has a good effect on predicting the trend of seasonal influenza. Therefore, we believe that this study can advance the time of influenza outbreak prediction and provide a reference for future technologies to predict influenza trends from different sources. Thus, our research can help establish an auxiliary decision-making system for public health emergencies.

    Study Limitations

    This study had a few limitations. First, owing to the limited availability of data, many influencing factors and symptom surveillance data were not available or sufficiently detailed. Second, circulating virus strains, specimen collection rates, case selection bias, and health care–seeking behaviors could affect virological surveillance. Finally, we innovatively used a deep-learning model to predict the trend of influenza in a megacity. Although this method is applicable to seasonal influenza, it may not be applicable to the prediction of chronic respiratory infectious diseases.

    Conclusions

    In summary, our findings demonstrated that the MAL model performs better in predicting influenza than traditional machine-learning and deep-learning models. The prediction and early warning model for influenza and other acute respiratory infectious diseases still needs to be improved. We believe that the model developed in this study can advance the time of influenza outbreak prediction and provide a reference for future technologies to predict influenza trends from different sources. Based on further timely and effective prediction of influenza epidemic intensity, early and better preparedness can be implemented to reduce the health damage to the population.

    Acknowledgments

    We thank the staff members at Beijing Center for Disease Prevention and Control and the Chinese Center for Disease Control and Prevention. This study was funded by the Chinese Academy of Medical Sciences (CAMS) Innovation Fund for Medical Sciences (2021-I2M-1-044), The Special Fund for Health Development Research of Beijing (2021-1G-3013), and Beijing Municipal Natural Science Foundation (M21024).

    Data Availability

    Climate data were obtained from China Meteorological Data Sharing Service System [20]. The air quality data were from Beijing Municipal Ecological and Environmental Monitoring Center [23]. The search index came from the Baidu search index engine [25]. GDP data were from Beijing Municipal Bureau of Statistics [24]. The data of influenza-like illness cases and influenza etiology in this study were from Beijing Center for Disease Prevention and Control and the Chinese Center for Disease Control and Prevention. The data sets analyzed during the current study are available from the corresponding author on reasonable request.

    Authors' Contributions

    LM, LF, and WY contributed to the study design. GL, JD, and TL collected the data. LY analyzed and interpreted the data. LY, JY, and TZ wrote the manuscript. XZ, XH, and WL checked the data and results. All authors read and approved the final manuscript. Authors LM (malibing@sph.pumc.edu.cn), LF (fengluzhao@cams.cn), and WY (yangweizhong@cams.cn) are co-corresponding authors for this article. LY, GL, and JY contributed equally as coauthors. LM, LF, and WY contributed equally as senior coauthors.

    Conflicts of Interest

    None declared.

    1. Petersen E, Koopmans M, Go U, Hamer DH, Petrosillo N, Castelli F, et al. Comparing SARS-CoV-2 with SARS-CoV and influenza pandemics. Lancet Infect Dis 2020 Sep;20(9):e238-e244 [FREE Full text] [CrossRef] [Medline]
    2. Coronavirus disease (COVID-19) pandemic. World Health Organization.   URL: https://www.who.int/emergencies/diseases/novel-coronavirus-2019 [accessed 2023-01-15]
    3. Krammer F, Smith GJD, Fouchier RAM, Peiris M, Kedzierska K, Doherty PC, et al. Influenza. Nat Rev Dis Primers 2018 Jun 28;4(1):3 [FREE Full text] [CrossRef] [Medline]
    4. Taubenberger JK, Morens DM. The 1918 influenza pandemic and its legacy. Cold Spring Harb Perspect Med 2020 Oct 01;10(10):a038695 [FREE Full text] [CrossRef] [Medline]
    5. Viboud C, Simonsen L, Fuentes R, Flores J, Miller MA, Chowell G. Global mortality impact of the 1957-1959 influenza pandemic. J Infect Dis 2016 Mar 01;213(5):738-745 [FREE Full text] [CrossRef] [Medline]
    6. Reperant LA, Moesker FM, Osterhaus AD. Influenza: from zoonosis to pandemic. ERJ Open Res 2016 Jan 11;2(1):00013-2016 [FREE Full text] [CrossRef] [Medline]
    7. Dawood FS, Iuliano AD, Reed C, Meltzer MI, Shay DK, Cheng P, et al. Estimated global mortality associated with the first 12 months of 2009 pandemic influenza A H1N1 virus circulation: a modelling study. Lancet Infect Dis 2012 Sep;12(9):687-695. [CrossRef] [Medline]
    8. Baker RE, Mahmud AS, Miller IF, Rajeev M, Rasambainarivo F, Rice BL, et al. Infectious disease in an era of global change. Nat Rev Microbiol 2022 Apr 13;20(4):193-205 [FREE Full text] [CrossRef] [Medline]
    9. Uyeki TM, Hui DS, Zambon M, Wentworth DE, Monto AS. Influenza. Lancet 2022 Aug 27;400(10353):693-706 [FREE Full text] [CrossRef] [Medline]
    10. Ginsberg J, Mohebbi MH, Patel RS, Brammer L, Smolinski MS, Brilliant L. Detecting influenza epidemics using search engine query data. Nature 2009 Feb 19;457(7232):1012-1014. [CrossRef] [Medline]
    11. Santillana M, Nguyen AT, Dredze M, Paul MJ, Nsoesie EO, Brownstein JS. Combining search, social media, and traditional data sources to improve influenza surveillance. PLoS Comput Biol 2015 Oct 29;11(10):e1004513 [FREE Full text] [CrossRef] [Medline]
    12. Ali ST, Cowling BJ, Wong JY, Chen D, Shan S, Lau EH, et al. Influenza seasonality and its environmental driving factors in mainland China and Hong Kong. Sci Total Environ 2022 Apr 20;818:151724 [FREE Full text] [CrossRef] [Medline]
    13. Yu H, Alonso WJ, Feng L, Tan Y, Shu Y, Yang W, et al. Characterization of regional influenza seasonality patterns in China and implications for vaccination strategies: spatio-temporal modeling of surveillance data. PLoS Med 2013 Nov 19;10(11):e1001552 [FREE Full text] [CrossRef] [Medline]
    14. Yang W. Early warning of infectious disease theory and practice. Cambridge, MA: Academic Press; 2017.
    15. Tsan Y, Chen D, Liu P, Kristiani E, Nguyen KLP, Yang C. The prediction of influenza-like illness and respiratory disease using LSTM and ARIMA. Int J Environ Res Public Health 2022 Feb 07;19(3):1858 [FREE Full text] [CrossRef] [Medline]
    16. Zhang X, Feng L, Lai S, Ma L, Zhang T, Yang J, et al. Research progress on early warning model of influenza. Zhonghua Yu Fang Yi Xue Za Zhi 2022 Nov 06;56(11):1576-1583. [CrossRef] [Medline]
    17. Cheng H, Wu Y, Lin M, Liu Y, Tsai Y, Wu J, et al. Applying machine learning models with an ensemble approach for accurate real-time influenza forecasting in Taiwan: development and validation study. J Med Internet Res 2020 Aug 05;22(8):e15394 [FREE Full text] [CrossRef] [Medline]
    18. Choo H, Kim M, Choi J, Shin J, Shin S. Influenza screening via deep learning using a combination of epidemiological and patient-generated health data: development and validation study. J Med Internet Res 2020 Oct 29;22(10):e21369 [FREE Full text] [CrossRef] [Medline]
    19. Jang B, Kim I, Kim JW. Effective training data extraction method to improve influenza outbreak prediction from online news articles: deep learning model study. JMIR Med Inform 2021 May 25;9(5):e23305 [FREE Full text] [CrossRef] [Medline]
    20. China Meteorological Data Service Centre.   URL: https://data.cma.cn/en [accessed 2023-01-15]
    21. Qi L, Liu T, Gao Y, Li Q, Tang W, Tian D, et al. Effect of absolute humidity on influenza activity across different climate regions in China. Environ Sci Pollut Res Int 2022 Jul 26;29(32):49373-49384. [CrossRef] [Medline]
    22. Peci A, Winter A, Li Y, Gnaneshan S, Liu J, Mubareka S, et al. Effects of absolute humidity, relative humidity, temperature, and wind speed on influenza activity in Toronto, Ontario, Canada. Appl Environ Microbiol 2019 Mar 15;85(6):e02426-18 [FREE Full text] [CrossRef] [Medline]
    23. Beijing Municipal Environmental Monitoring Center.   URL: http://www.bjmemc.com.cn/ [accessed 2023-01-15]
    24. Beijing Municipal Bureau of Statistics.   URL: https://nj.tjj.beijing.gov.cn/nj/main/2021-tjnj/zk/indexch.htm [accessed 2023-01-15]
    25. Baidu search index engine.   URL: https://index.baidu.com/v2/index.html#/ [accessed 2023-01-15]
    26. Zhang Y, Bambrick H, Mengersen K, Tong S, Hu W. Using internet-based query and climate data to predict climate-sensitive infectious disease risks: a systematic review of epidemiological evidence. Int J Biometeorol 2021 Dec 01;65(12):2203-2214. [CrossRef] [Medline]
    27. Morgan OW, Abdelmalik P, Perez-Gutierrez E, Fall IS, Kato M, Hamblion E, et al. How better pandemic and epidemic intelligence will prepare the world for future threats. Nat Med 2022 Aug 28;28(8):1526-1528 [FREE Full text] [CrossRef] [Medline]
    28. Zhang Y, Chen K, Weng Y, Chen Z, Zhang J, Hubbard R. An intelligent early warning system of analyzing Twitter data using machine learning on COVID-19 surveillance in the US. Expert Syst Appl 2022 Jul 15;198:116882 [FREE Full text] [CrossRef] [Medline]
    29. Google's Flu Project shows the failings of big data. Time.   URL: https://time.com/23782/google-flu-trends-big-data-problems/ [accessed 2023-01-15]
    30. World Internet Users Statistics. World Population Stats.   URL: https://www.internetworldstats.com/stats.htm [accessed 2023-01-15]
    31. Feng L, Zhang T, Wang Q, Xie Y, Peng Z, Zheng J, et al. Impact of COVID-19 outbreaks and interventions on influenza in China and the United States. Nat Commun 2021 May 31;12(1):3249. [CrossRef] [Medline]
    32. Flaxman S, Mishra S, Gandy A, Unwin HJT, Mellan TA, Coupland H, Imperial College COVID-19 Response Team, et al. Estimating the effects of non-pharmaceutical interventions on COVID-19 in Europe. Nature 2020 Aug 08;584(7820):257-261. [CrossRef] [Medline]
    33. Fricke LM, Glöckner S, Dreier M, Lange B. Impact of non-pharmaceutical interventions targeted at COVID-19 pandemic on influenza burden - a systematic review. J Infect 2021 Jan;82(1):1-35 [FREE Full text] [CrossRef] [Medline]
    34. Yi R, Tang L, Tian Y, Liu J, Wu Z. Identification and classification of pneumonia disease using a deep learning-based intelligent computational framework. Neural Comput Appl 2021 May 20:1-14 [FREE Full text] [CrossRef] [Medline]
    35. Jung S, Moon J, Park S, Hwang E. Self-attention-based deep learning network for regional influenza forecasting. IEEE J Biomed Health Inform 2022 Feb;26(2):922-933. [CrossRef] [Medline]
    36. Liu T, Siegel E, Shen D. Deep learning and medical image analysis for COVID-19 diagnosis and prediction. Annu Rev Biomed Eng 2022 Jun 06;24(1):179-201 [FREE Full text] [CrossRef] [Medline]

    ‎
    AH: absolute humidity
    Apmean: mean air pressure per week
    AQI: air quality index
    CDC: Center for Disease Control and Prevention
    GDP: gross domestic product
    GRU: gated recurrent units
    ILI: influenza-like illness
    LSTM: long short-term memory
    MAL: multiattention-long short-term memory
    RH: relative humidity
    Tmax: maximum temperature per week
    Tmean: mean temperature per week
    Tmin: minimum temperature per week
    WHO: World Health Organization
    XGBoost: extreme gradient boosting

    Edited by A Mavragani; submitted 11.11.22; peer-reviewed by YL Cheong, Y Wang, YS Lo; comments to author 06.12.22; revised version received 21.12.22; accepted 10.01.23; published 13.02.23

    Copyright

    ©Liuyang Yang, Gang Li, Jin Yang, Ting Zhang, Jing Du, Tian Liu, Xingxing Zhang, Xuan Han, Wei Li, Libing Ma, Luzhao Feng, Weizhong Yang. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 13.02.2023.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.