We present a novel cardiac arrest prediction model called the ECG-Image-Aware Network (EIANet) that was designed to predict IHCA with high accuracy. Our study adhered to the Guidelines for Developing and Reporting Machine Learning Predictive Models in Biomedical Research [24]. EIANet operated in 3 key stages, as outlined in Figure 1. In the first stage, ECG waveform images were extracted from triage ECG reports obtained from the National Taiwan University Hospital (NTUH). The NTUH is a tertiary academic medical center with approximately 2400 beds and 100,000 ED visits per year. We obtained 12-lead ECG images at ED triage for adult patients with EDCA (cases) from 2011 to 2019. We defined EDCA as patients who arrived in the ED with vital signs and later developed cardiac arrest in the ED. OHCAs with or without return of spontaneous circulation on ED arrival were excluded. We focused on treated EDCA, so patients with a do-not-resuscitate order were also excluded. The EDCA cases were from the same pool as in our previous studies on EDCA [16,25]. Control ECGs were randomly selected from adult ED patients without cardiac arrest during the same study period.

The ECG images were subjected to a thorough preprocessing step, where extraneous information, artifacts, and noise were removed, ensuring that the data fed into the model were clean and of high quality for more accurate analysis. In the second stage, the preprocessed ECG images were passed through a deep learning model, which was enhanced using a spatial attention module. This module allowed the model to focus on more relevant areas of the ECG image, identifying subtle patterns and features critical for predicting cardiac arrest. Finally, in the third stage, the model’s output was combined with the recall metric to calculate the binary recall loss (BRLoss). This custom loss function helps address slight class imbalance by prioritizing recall during training, leading to more balanced predictions, particularly in rare IHCA cases.

The NTUH study was approved by the NTUH Institutional Review Board (reference number: 202304129RINC), which waived the requirement for patient informed consent. The FEMH study was approved by the FEMH Institutional Review Board (reference number: 112193-F), which also waived the requirement for patient informed consent.

Although deep learning models can automatically extract relevant features from input data, their performance can be significantly improved when noisy or unclean data are addressed through careful preprocessing. In our approach, we began by converting the RGB ECG image into a binary format using a thresholding technique. Specifically, we first used Gaussian blur and thresholding to remove unnecessary elements such as thin grid lines. By applying a threshold, pixels exceeding the threshold value were turned black (0), while those below the threshold were converted to white (255). This step ensured that distracting elements were removed, enhancing the clarity of the ECG waveform for further analysis.

Next, we used a connected component algorithm to identify and remove specific unwanted components such as lead labels and other extraneous noise in the image. These elements, if left unaddressed, can interfere with the accuracy of the model’s predictions. To further refine the image, we applied a morphological opening operation, which consists of erosion followed by dilation, using a 3 × 3 structuring element. This process smoothed the ECG lines and reduced minor noise, ensuring that the primary features of the ECG waveform were preserved while removing small, irrelevant artifacts. This enhanced the overall quality of the image, making it more suitable for deep learning analysis.

To complete the preprocessing, we applied a Gaussian blur to the image, which helped minimize any remaining noise that may have been left after previous steps. The ECG image was then transformed to gray scale, and we used histogram equalization to improve the contrast between the ECG waveform and any residual background noise. This step ensured that the critical features of the waveform were more distinct, allowing the deep learning model to focus on the most relevant data points. In the final stage of preprocessing, the image was reconverted to binary format, accentuating the essential features necessary for more accurate and effective analysis by the model, ultimately improving its ability to predict IHCA.

In our model, we used the ResNet50 [26] architecture as the foundational framework due to its robust performance in handling complex image data. ResNet50’s use of residual connections enhances the learning capacity of deep networks by facilitating the flow of information and mitigating issues such as vanishing gradients. We integrated an attention mechanism into the network. This integration ensured that the model can better identify subtle, critical signals in the ECG data, ultimately enhancing its predictive performance and improving patient outcomes.

Following the concept of the Convolutional Block Attention Module [27], we strategically positioned spatial attention blocks after each of the first 4 layers, as Figure 2. This design choice was motivated by 2 primary considerations. First, the nature of ECG waveforms, which display both fine and coarse-grained patterns, necessitates a multiscale approach to feature extraction. By placing spatial attention blocks at the front, middle, and end of the network, we enhanced the model’s ability to capture and optimize features across varying scales. This is crucial for accurately interpreting the diverse characteristics present in ECG signals, which can vary significantly in amplitude and duration due to different physiological conditions. Second, we intentionally limited the number of attention blocks to mitigate the risk of overfitting during the training process. By judiciously inserting these blocks after each layer, we balanced the complexity of the model with its generalization capabilities.

The incorporation of the spatial attention mechanism allowed the model to learn more representative features, leading to improved performance. Visualization results further demonstrated the model’s ability to concentrate on more relevant patterns. Detailed findings are discussed in the following section.

Our training data set exhibited a slight imbalance of positive and negative samples, which posed a challenge for accurately predicting IHCA. Given the serious threat that IHCA presents to patient safety, maximizing the model’s recall is of paramount importance, as it measures the system’s ability to correctly identify true positive cases. A high recall ensures that most actual cardiac arrest occurrences are detected, allowing health care professionals to respond swiftly and effectively. However, this focus on recall can inadvertently lead to an increase in false positives, overwhelming the health care system with unnecessary alarms and interventions. Such a scenario can strain resources and divert attention from genuine emergencies, potentially leading to alarm fatigue among medical staff. Therefore, it is equally critical to maintain a reasonable level of precision alongside recall. Striking a balance between these 2 metrics is essential to ensure that we effectively identify critical cases of IHCA while also minimizing the burden on health care providers, ultimately safeguarding patient safety and enhancing the efficiency of emergency care.

To address this, rather than using the traditional binary cross-entropy loss, we used BRLoss by drawing on the concept of performance-based loss from [28]. BRLoss used recall-related weights for each class, as shown in Equation 1:

where N is the number of samples in a batch, y_i is the ground truth label of i-th data in the batch, y_i ∈ [0, 1], p_i is the predicted probability, 0 ≤ p_i ≤ 1 and Recall_c means recall of class c, Recall_c [0, 1], c is 0 or 1.

Unlike many other statistics-based weighted loss [29-31] functions that may cause unnecessary false positives, sacrificing precision to gain recall, BRLoss effectively balanced recall and precision. We used the same analytical method as [26] to perform a partial derivative of the loss function. Assuming the i – th data sample in the batch has a final output z_i before applying the sigmoid function, the gradient for BRLoss can be calculated as shown in Equation 2:

In the case where the ground truth is 1, an increase in false negatives leads to a larger negative gradient in the term related to Recall₁in Equation 2. This drives z to increase during gradient descent, reducing false negatives and improving Recall₁. Similarly, when the ground truth is 0, more false positives result in a larger positive gradient in the term related to Recall₀ (specificity), causing z to decrease and reduce false positives. BRLoss thus balanced recall and precision, leading to more reliable model performance.

We implemented EIANet using the Python and Pytorch [35] framework. The EIANet training was accelerated using an RTX 3090 GPU. The AdamW [36] optimizer was used to train the model over 200 epochs, with a learning rate initialized at 2 × 10^-5.

In our binary classification task aimed at predicting IHCA using ECG images, we used a comprehensive evaluation strategy to assess model performance. This involved using several metrics derived from the confusion matrix, such as accuracy, precision, recall (sensitivity), and specificity. These metrics provided a detailed breakdown of how well the model distinguished between positive (IHCA) and negative (non-IHCA) cases, helping us understand not only the overall accuracy but also how well the model balanced false positives and false negatives. We also calculated the area under the receiver operating characteristic curve (AUROC), which measured the trade-off between true positive rates and false positive rates across different decision thresholds. A higher AUROC value indicates that the model is more capable of distinguishing between the 2 classes, even in the presence of class imbalance. Moreover, we incorporated the area under the precision-recall curve (AUPRC) to evaluate performance, particularly in handling imbalanced data sets like ours, where the positive class is underrepresented. The AUPRC focuses on precision (positive predictive value) and recall, offering a more sensitive measure of the model’s performance when identifying IHCA cases without being skewed by the larger number of negative cases. Together, these metrics provided a robust, multifaceted assessment of our model’s effectiveness at predicting IHCA, ensuring that it performs well across a range of critical evaluation criteria.

Figure 3 and Figure 4 provide feature map [37] visualizations of positive samples from the NTUH-ECG data set. The visualization shows the Grad-CAM output, highlighting regions with Grad-CAM values greater than 0.4. Figure 3 shows the output of a model incorporating spatial attention, while Figure 4 presents the results of a model devoid of spatial attention. Alterations in the ST segment serve as pivotal indicators of myocardial ischemia. Notwithstanding the presence of a bundle branch block in this ECG, EIANet, as depicted in Figure 3, could still accurately pinpoint the ST segment. This underscores EIANet’s capacity to discern patterns associated with EDCA within ECG images. Furthermore, tachycardia is a prevalent ECG manifestation preceding EDCA. The regular bright spots in the figures suggest that the algorithm is adept at counting the heart rate. In contrast, although the model without spatial attention, as illustrated in Figure 4, also counted the heart rate, its focus on the ST segment was less precise and more dispersed.

Figure 5 and Figure 6 present visualizations from the FEMH-ECG data set, which further illustrate the impact of spatial attention by focusing on heart rate–related features. In Figure 5 (with spatial attention), the model effectively identifies regular bright spots corresponding to heartbeats, indicating its ability to count the heart rate accurately. Conversely, Figure 6 (without spatial attention) shows a less distinct attention pattern, reducing interpretability.

The first ablation study focused on evaluating the impact of different types of input data on model performance. As detailed in Table 5, using raw ECG images directly extracted from PDF reports for training demonstrated a certain level of predictive capability, indicating that even unprocessed images contain valuable information that the model can learn from. However, the study revealed that, when these raw images undergo a series of image preprocessing steps to remove irrelevant or distracting elements, the model’s performance saw marked improvement because the model could focus on crucial visual patterns and structures essential for predicting EDCA. This performance boost underscores the importance of preprocessing for enhancing the quality of the input data.

By applying preprocessing techniques, such as noise reduction, removal of grid lines, and extraction of key features, the model was better able to focus on the most relevant aspects of the ECG images, allowing it to learn more effectively. Preprocessing not only reduced the amount of noise and irrelevant information present in the images but also helped to clarify the visual patterns and structures that are crucial for predicting EDCA. This led to more accurate predictions and overall model performance.

Table 6 presents an ablation study evaluating the influence of spatial attention and BRLoss on the model’s performance and demonstrates the impact of incorporating the spatial attention mechanism and BRLoss on model performance. We trained our model on the NTUH-ECG training set and validated it using both the NTUH-ECG testing set and the FEMH-ECG data set. As expected, due to differences in patient populations, there was a slight decrease in model performance in the FEMH-ECG data set. The results indicate that the spatial attention module and BRLoss elevated the model’s performance. The spatial attention mechanism empowered the model to extract more effective ECG features, such as the ST segment and heart rate patterns, thereby enhancing feature representation and overall model accuracy. This mechanism ensured that the model concentrated on critical segments of the ECG image, leading to improved interpretability and reliability. Moreover, BRLoss ameliorated the model’s recall without compromising precision to a significant degree. BRLoss focuses on reducing false predictions and improving F₁-score and AUROC. However, its effectiveness also depends on the model’s ability to sufficiently represent the data. When combined with spatial attention, the performance of the model can be further enhanced by leveraging the strengths of both modules. Although BRLoss prioritizes recall, decision thresholds can be adjusted in clinical applications to achieve a suitable balance between recall and precision, addressing specific requirements in emergency scenarios.

To evaluate the contributions of the spatial attention module and BRLoss under varying class imbalances, we performed random resampling of the NTUH-ECG and FEMH-ECG data sets to adjust the proportion of positive samples to 0.1 and 0.2. Each resampling experiment was repeated 30 times, and the results were averaged to ensure robustness. The ablation study results under positive sample ratios of 0.1 and 0.2 are presented in Table 7. These results clearly demonstrate the contributions of spatial attention and BRLoss to the model’s performance. In addition, for metrics that are not dependent on the prevalence of positive cases (eg, AUROC), the model performance remained.

To our knowledge, this is the first study to develop and validate a deep learning algorithm (EIANet) specifically for predicting EDCA using ECG image data. The findings demonstrate that deep learning, a powerful artificial intelligence tool, can detect subtle ECG changes hours before cardiac arrest in the ED. Deployment of this tool at ED triage has the potential to gain lead time to identify high-risk patients for timely interventions and reduce EDCAs.

Traditionally, ECG prediction of cardiac arrest has mainly focused on OHCA (ie, using ECG to predict sudden cardiac death at the population level) [38]. For example, an electrical risk score that comprises 6 features (heart rate, prolonged corrected QT interval, Tpeak-Tend interval, QRS-T angle, left ventricular hypertrophy, and delayed QRS transition zone) can accurately predict sudden cardiac death within a community-based cohort [39]. A refined deep learning–based ECG model can further improve its predictive ability using the same 6 features [40]. Notably, tachycardia has been included as one of the features in the electrical risk score and was also documented in our feature map analysis and in a previous IPCA ECG study [23], suggesting the importance of this feature in predicting OHCA, IPCA, and EDCA.

One of our previous studies on EDCA focused on the development and validation of a simple 8-item, rule-based prediction tool using structural data at ED triage [25]. Another study of ours further advanced EDCA prediction by using deep learning with static triage data and dynamic vital signs measured during the ED stay [16]. None of these studies used ECG as an input for deep learning, as we did in this study. Given the promising results of using triage ECG to predict EDCA, the next logical step would be to combine ECG and structural data as a multimodal model for EDCA prediction.

The EIANet has several technical innovations. Before model training, we used image processing techniques to eliminate manual labels and noise from the ECG images. To enhance model performance, we integrated an attention mechanism and BRLoss. These components had a positive impact on EIANet’s predictive capabilities. Our research used 2 ECG image data sets: NTUH-ECG and FEMH-ECG. The NTUH-ECG data set, with a positive sample ratio of 40%, yielded impressive results, with EIANet achieving an F₁-score of 0.805 and AUPRC of 0.842. The FEMH-ECG data set, characterized by a lower positive sample ratio of 34.6%, still demonstrated strong performance, with EIANet attaining an F₁-score of 0.650 and AUPRC of 0.678. To validate the effectiveness of the proposed modules, we conducted ablation studies, further confirming their importance. Additionally, Grad-CAM visualization was used to provide interpretability for EIANet’s predictions using ECG images.

In the past, ECG prediction models were typically designed for diseases with relatively obvious features, such as myocardial infarction or hyperkalemia [41,42]. However, precardiac arrest ECGs lack absolutely distinct characteristics, making EDCA significantly more challenging to predict. Additionally, most existing image-based ECG models have remained in the proof-of-concept stage, often relying on general purpose framework models like ResNet without substantial customization for this specific task. These approaches, although promising, have struggled with critical limitations, such as high false-negative rates and limited interpretability. To address these challenges, we enhanced ResNet50 by incorporating a spatial attention mechanism, enabling the model to focus on subtle but clinically significant ECG patterns. Furthermore, we introduced a BRLoss function to prioritize recall and reduce false negatives, a critical improvement for predicting rare but life-threatening events like EDCA. These innovations aimed to overcome the inherent limitations of prior approaches and establish a practical, interpretable, and effective prediction framework.

Although our results demonstrate a certain level of predictive capability on both the NTUH-ECG and FEMH-ECG data sets, the performance on the FEMH-ECG data set declined. The observed drop in recall when evaluating the EIANet model on the FEMH-ECG data set can be attributed to multiple factors. First, patient population differences between the data sets likely played a significant role. Variations in patient populations could influence the manifestations of ECGs, leading to differing model performance. Second, differences in instrumentation between the 2 data sets may have introduced some biases. These discrepancies, combined with slight differences in image sizes formats and preprocessing process, might have impacted the model’s ability to generalize effectively. Addressing these challenges in future work by incorporating more diverse data sets could enhance the model’s robustness and applicability across different clinical settings. In the future, we hope to use EIANet as an encoder for ECG data, integrating it with other modalities of patient information to establish a more comprehensive multimodal prediction method for EDCA.

This study has some potential limitations. First, the model’s performance was dependent on the quality and variety of the training data. Although we included ECGs from 2 independent medical centers, further external validation studies are needed to evaluate the model’s robustness in different patient populations. Second, this was a retrospective study. Even with external validation, as in our retrospective study, models could have poor performance in the prospective setting, such as the Rothman Index by PeraHealth [43,44] and the Epic Sepsis Model [45,46]. Future implementation of the EIANet model in the ED would be warranted to test its real-world effectiveness.

In conclusion, we presented a comprehensive study aimed at understanding the relationship between ECG images and EDCA. To the best of our knowledge, this is the first in the literature that uses readily available 12-lead ECG images at ED triage to predict imminent ED cardiac arrest. Using 2 independent ED data sets, we developed and externally validated the novel deep learning EIANet model that can be used to identify high-risk patients and potentially reduce devastating ED cardiac arrest events. Future implementation of the EIANet in real time is warranted to test whether this tool could flag high-risk patients at ED triage for timely interventions, thereby improving patient outcomes.