Currently submitted to: Journal of Medical Internet Research
Date Submitted: Jun 11, 2019
(closed for review but you can still tweet)
Exploiting Machine Learning Algorithms and Methods for the Prediction of Agitated Delirium after Cardiac Surgery
Delirium is a temporary mental disorder that occurs frequently among patients undergoing cardiac surgery. It is strongly associated with major adverse events, which in turn leads to increased cost and poor outcomes (e.g.: need for nursing home due to cognitive impairment, stroke, and death). The ability to foresee patients at risk of delirium will guide the timely initiation of multi-modal preventive interventions, which will aid in reducing the burden and negative consequences associated with delirium. Several studies have focused on the prediction of delirium. However, the number of studies in cardiac surgical patients that have used machine learning methods is very limited.
Hence, we sought to explore the application of several machine learning predictive models that can pre-emptively predict delirium in patients undergoing cardiac surgery and compare their performance.
We investigated a number of machine learning methods to develop models that can predict delirium after cardiac surgery. A clinical dataset comprising over 5000 actual patients who underwent cardiac surgery in a single center was used to develop the models using: Logistic Regression, Artificial Neural Networks, Support Vector Machines, Bayesian Belief Networks, Naïve Bayesian, Random Forrest, and Decision Trees
Since only 11.4% patients developed delirium, we addressed the underlying class imbalance, using random under-sampling, in the training dataset. The final prediction performance was validated on a separate test dataset. Due to the target class imbalance, several measures were used to evaluate algorithms performance for the “Delirium” class on the test dataset. Out of the selected algorithms, the Support Vector Machines algorithm had the best F1-Score for positive cases, Kappa, and positive predictive value (40.2%, 29.3%, 29.7%; respectively). The Artificial Neural Networks had the best receiver-operator area-under the curve (78.2%). The Bayesian Belief Networks had the best precision-recall area-under the curve for detecting positive cases (30.4%).
Although, delirium is inherently complex, preventive measures to mitigate its negative effect can be applied proactively if patients at risk are prospectively identified. Our results highlight two important points: 1) addressing class imbalance on the training dataset will enhance machine learning model’s performance in identifying patients likely to develop post-operative delirium, 2) When it comes to complex medical problems, like delirium, increasing the complexity by using machine learning methods will improve the prediction, which will lead to reduction of cost by prevention of complications and better patients outcomes.