Two authors (WYY and XY) independently searched PubMed, Cochrane, Embase, and Web of Science databases for relevant articles using the keywords “Parkinson’s disease,” “Machine Learning,” and “cognitive decline” in different combinations (Table S1 in Multimedia Appendix 1). The included studies were from the start of the database until March 2, 2024. The review was reported according to the PRISMA (Preferred Reporting Items for Systematic Review and Meta-Analyses) statement. It is registered in PROSPERO (International Prospective Register of Systematic Reviews; number CRD42023480196).

Studies were eligible if they aimed to diagnose or predict the cognitive impairment in PD. The target condition was the normal cognition (PD-NC), PD-MCI, and PD-D. ML models were carried out to predict the target condition. Studies were eligible if these reported data on the following: true positive (TP), true negative (TN), false positive (FP), false negative (FN), sensitivity, specificity, accuracy, positive predictive value, or negative predictive value, and others. In case of not reporting TP, TN, FP, or FN, these were calculated from known variables (sensitivity and specificity).

The articles retrieved from the electronic databases were imported into EndNote (Clarivate) for further analysis. Duplicates, reviews, dissertations, cases, and conference abstracts were systematically eliminated through an automated process. Two authors (WYY and XY) independently screened the titles and abstracts of the rest of the records. Any disagreements were decided by the third author (CYF). Following this step, the full text of the remaining studies was downloaded and carefully reviewed, resulting in the final selection of relevant articles.

The data collected from the chosen studies included publication details such as title, first author, and year of publication, as well as study information such as study design, whether it was conducted at single or multiple centers, diagnostic criteria for PD and cognitive impairment, ML models employed, predictors used, and value index assessed.

There is no widely accepted checklist for assessing the quality of diagnostic ML papers. We chose combined criteria that had been used in a previous report [14]. Two review authors (WYY and XY) independently assessed the risk of bias based on the Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) [15], and any disagreements were resolved through the third author (CYF). If one of the questions was scored at a high risk of bias, the domain was scored at a high risk of bias. At least one domain at high risk of bias resulted in an overall score of high risk of bias, and only one domain scored as unclear risk of bias resulting in an overall score of unclear risk of bias for that paper.

Due to the diversity of models and predictors under study, models were categorized by type (random forest, neural network, etc) irrespective of variable selection procedures in the study. Similarly, predictor variables were grouped for analysis. For example, demographic features (such as age, gender, family history, and education), clinical features (such as motor symptoms, nonmotor symptoms, and various scale tests, such as Mini-Mental Status Examination [MMSE] and Montreal Cognitive Assessment [MoCA]), neuroimaging features (such as magnetic resonance imaging [MRI] and single photon emission computed tomography features), biofluid features (such as -synuclein [-syn], amyloid β [Aβ], and phospho-microtubule-associated protein tau [p-tau] in cerebrospinal fluid and blood plasma), genetic features (such as GBA, LRRK2, and single-nucleotide polymorphism [SNP] variants), and quantitative electroencephalography.

Given that classification algorithms in ML can indeed be considered as a diagnostic test, and that the results included in the article, ultimately, we used bivariate meta-analysis [16] to estimate the pooled sensitivity and specificity results, along with their corresponding 95% CIs. Estimated sensitivity and specificity were represented as forest plots. A summary receiver operator characteristic (SROC) curve was generated to evaluate the accuracy of ML for the prediction of cognitive impairment in patients with PD [17]. The test of heterogeneity was assessed using the Cochrane Q-test and I² test [18]. Heterogeneity was defined as significant if the Q-test results showed P<.05 or I²>50%. Potential publication bias was estimated using Deeks’ funnel plot. In the event of bias, it may be evidenced by the visual asymmetry observed in conventional funnel plots. A Fagan nomogram was used to identify the maximum pretest and posttest likelihood. We also conducted sensitivity analysis by removing studies with a high risk of bias to identify potential sources of heterogeneity. Considering not all studies reported the results of both the train and validation sets, we also analyzed the single set. Statistical analyses were performed using GraphPad Prism 9.5 (GraphPad Software, Inc) and Stata (version 17; StataCorp). The level of significance was set at P<.05.

Figure 1 shows the literature selection and filtering process following the PRISMA 2020 guidelines [19]. A total of 1674 records were identified from 4 databases. After removing 199 duplicates, 1475 records were screened, with 785 excluded for being unrelated (reviews, editorials, conference abstracts, case reports, etc). Of the 690 records sought for retrieval, 3 were not obtained. Finally, 687 reports were assessed for eligibility, with 137 excluded for not being related to PD and 511 excluded for being irrelevant. Finally, 38 studies [20-57] published between 2013 and 2024 were included in the qualitative analysis, from which 12 studies [20-31] were included in the meta-analysis. A total of 9698 participants were involved in the current study, comprising 8564 patients with PD and 1134 healthy controls in qualitative analysis. Additional information can be found in Table S2 in Multimedia Appendix 1 [20-57].

Table S3 in Multimedia Appendix 1 shows the risk of bias status of each included article. A total of 18 [24,27,29,32,33,39,42-47,51,52,54-57] out of 38 (47%) studies [20-57] were scored as high risk of bias, 9 [30,31,37,38,40,41,49,50,53] out of 38 (23%) studies were scored as unclear risk of bias, and the rest of the studies were low risk of bias. Studies scored a high risk of bias for failing to describe their study population (patient selection), losing the gold standard for diagnosing cognitive impairment (reference standard).

In the 38 studies [20-57], 172 models were constructed with different ML models and predictors. However, 142 models were subjected to analysis, with accuracy being the primary outcome. These included 45 SVM models, 29 random forest models, 9 K-nearest neighbors models, 18 naive Bayes models, and other supervised learnings. Also, we included 3 semisupervised models that used SVM with principal component analysis PCA and 2 neural networks. The overall mean accuracy of 142 models was 74.32% (SD 10.25%). Of the 89 models that reported area under the receiver operating characteristic curve, the mean accuracy was 0.74 (SD 0.12). Also, 120 models reported sensitivity and specificity, with mean values of 71.07% (SD 13.72%) and 77.01% (SD 14.31%), respectively.

This study discovered the predictors used to diagnose and predict cognitive impairment in PD (Figure 2A). Clinical features have the highest percentage, followed by neuroimaging and demographic features. Other predictors such as biofluid markers, genetic variants, electroencephalography, and gait features show a more balanced or lower percentage representation. The top-used predictors were summarized in 142 models (Figure 2B). The demographic and clinical characteristics that emerged as the most significant predictors included age, gender, and education level, as well as the Unified Parkinson’s Disease Rating Scale (UPDRS), rapid eye movement sleep behavior disorder, MMSE, and MoCA. Furthermore, additional neuroimaging and biomarkers have been identified as potentially contributing to the prediction process. Overall, 116 out of 142 models used a single type of feature to diagnose, with a mean accuracy of 73.42% (SD 10.24%). Nearly 18% of models used multiple types of features combined, with a higher accuracy of 78.57% (SD 9.2%). In the single type of model, the predictors with the highest accuracy were neuroimaging features, achieving an accuracy of 78.78% (SD 11.48%). Furthermore, 13 articles used longitudinal data to predict cognitive impairment or conversion, with the majority of predictors being neuroimaging features and biofluid markers.

Due to limited reporting of TP, TN, FP, FN, sensitivity, and specificity, only 12 studies were conducted for meta-analysis [20-31]. For studies with multiple models, we prioritized the one with the highest accuracy, reported in 63% (24/38) of the articles, to represent optimal model performance. A total of 7 studies [25-31] reported outcomes in the train set or validation set, while 5 studies [20-24] reported outcomes in both the train set and validation set. Forest plots for sensitivity and specificity are shown in Figure 3 for the train and validation set, respectively, and Figure S1 in Multimedia Appendix 1 for the single set (train or validation set). The training set sensitivity exhibited different results in the Q-test (Q=5.68, P=.22) and I² test (I²=29.53). Other sensitivity and specificity values showed no heterogeneity. Using sensitivity, the combined sensitivity and specificity were 0.76 (95% CI 0.67-0.83) and 0.83 (95% CI 0.76-0.88), respectively. When using specificity, the combined values were 0.77 (95% CI 0.65-0.86) and 0.76 (95% CI 0.63-0.85), respectively. The area under the curves of the SROC were 0.87 (95% CI 0.83-0.89) and 0.83 (95% CI 0.8-0.86), respectively (Figure 4; Figure S2 in Multimedia Appendix 1). Fagan plots were constructed to illustrate the pretest and posttest probability of ML predicting cognitive impairment in patients with PD (Figures S3 and S4 in Multimedia Appendix 1). The Deeks’ test showed no significant publication bias for ML (Figures S5 and S6 in Multimedia Appendix 1).

According to the risk of bias analysis, 3 articles scored high. After excluding these studies, all results exhibited no statistically significant heterogeneity, for the sensitivity of the training set. The results indicated a moderate degree of heterogeneity between studies (I²=45.45). However, the results suggested no statistically significant heterogeneity between studies in the Q-test (Q=5.5, P=.14; Figure S7 in Multimedia Appendix 1). The sensitivity, specificity, and SROC were consistent with the original analysis (Figure S9 in Multimedia Appendix 1). In the single set, there exists high heterogeneity both in sensitivity (Q=14.6, P=.01; I²=72.59) and in specificity (Q=96.07, P<.01; I²=95.84; Figure S8 in Multimedia Appendix 1). The results were consistent with the original analysis, with high sensitivity and specificity, and good overall diagnostic accuracy (Figures S9 and S10 in Multimedia Appendix 1). Fagan plots are shown in Figures S11 and S12 in Multimedia Appendix 1. The Deeks’ test also found no significant publication bias for ML (Figures S13 and S14 in Multimedia Appendix 1).

This meta-analysis provided new insights into the efficacy of ML in diagnosing cognitive impairment in patients with PD. The average accuracy of 142 models was 74.32%, with 120 models reporting mean sensitivity and specificity values of 71.07% (SD 13.72%) and 77.01% (SD 14.31%), respectively. No significant heterogeneity was found among the sensitivities and specificities reported by the analyzed included in our meta-analysis. Nearly 50% of the included studies had a high risk of bias affecting applicability. A total of 12 studies [22,31,36,37-44,53] explored the cognition changes in a longitudinal cohort of patients with PD. Although complete data were not obtained in this study, we could conclude that the ML can predict potential cognitive impairment at baseline and forecast changes in cognitive state.

According to the QUADAS-2, most of the studies had high or unclear risk in the patient selection and reference standard. Part of the studies lacked detailed patient source descriptions. However, these articles were not excluded due to the use of Movement Disorder Society diagnostic criteria for cognitive impairment in patients with PD [7,58]. Many studies faced similar issues in patient selection, and complete exclusion would significantly reduce the available data. The diverse metrics used to assess ML resulted in incomplete reporting of expected metrics, increasing bias risk. These problems may lead to unstable meta-analysis results and make it difficult to provide meaningful conclusions. Furthermore, the low heterogeneity between studies limits the values of sensitivity analyses.

The meta-analysis of 12 studies [20-31] demonstrated that ML models exhibited greater consistency in terms of sensitivity and specificity in the training and validation sets. In addition, no heterogeneity or publication bias was observed, suggesting that the ML model exhibits superior performance in predicting cognitive impairment in patients with PD. However, further studies are essential to validate its clinical applicability.

Studies with a high risk of bias were excluded, and the results were found to be consistent with those of the original analyses. However, the I² in the sensitivity of the train set exhibited a slight increase, while the Q-test suggested no statistically significant heterogeneity. The sensitivity analysis demonstrated the credibility and robustness of the results. The results of the remaining 9 studies [20-23,25,26,28,30,31] remained stable compared to the original analysis and showed high sensitivity and specificity of the ML to diagnose and predict cognitive impairment in patients with PD. While, due to the limited number of studies included in the meta-analysis, further exploration was required.

A recent review divided ML into traditional ML (including supervised and unsupervised ML), and ML methods based on neural networks (such as convolutional neural networks and recurrent neural networks) [59]. The majority of research still focuses on traditional ML in this review, possibly because supervised learning still has advantages in ML algorithms and processed data. First, supervised learning can be further divided into classification and regression based on their tasks. It can differentiate between patients with cognitive impairment and those with mild cognitive normality and also can identify patients with MCI in PD from those who can be diagnosed with dementia. Unsupervised learning is primarily used for clustering, aiming to discover inherent groupings in the data. It is limited to categorizing patients into distinct groups by reducing the dimensionality of their shared characteristics. It cannot offer a definitive diagnosis due to the absence of labeled features. Second, although unsupervised learning is proving valuable in reducing the cost of labeling data and automatically identifying data structures and patterns, the results still require manual interpretation and analysis for interpretability [60].

Determining the superiority of ML methods is challenging. With the development of imaging technology and the increasing costs of data collection, unsupervised learning, deep learning, and neural networks may become more prevalent in the future. Notably, semisupervised learning also showed great potential [61]. Chen et al [32] combined SVM with principal component analysis to successfully extract 6 features from 32 predictors, improving the performance over the simple SVM model. Other models use unsupervised learning to cluster this unlabeled data and then use supervised learning for longitudinal prediction [33]. Further research and discussion are needed to assess the accuracy and effectiveness of these different models.

We found that cognitive tests such as the MMSE and the MoCA were mostly used as predictors. Meanwhile, the UPDRS score as a specific feature that can reflect the patient’s severity of motor and nonmotor symptoms was highly recommended as a potential predictor. In addition, with the advancement of neuroimaging techniques, not only traditional neuroimaging, such as cortical thickness but also functional MRI was widely used to diagnose and predict disease progression, including Fluorine-18 fluorodeoxyglucose positron emission tomography, which can reflect the metabolism in the brain and diffusion tensor imaging which can show the damages to brain connectivity. On the other hand, demographic features such as older age, male sex, and lower education, which were considered as highly risk factors to cause PD, were highly ranked. Biofluid markers, total-tau, p-tau, Aβ, and -syn detected in the cerebrospinal fluid were the most popular approaches and were deemed as the vital factors to show the prognosis. Furthermore, the rise of genomics has become a crucial factor in diagnosis and prediction. The growing popularity of genome-wide association studies research has made SNP variation a focal point of interest, more and more SNP variants that count were found to cause PD. The APOE gene, which is the risk factor in AD, is also a potential predictor to predict cognitive impairment in PD. This review also shows that the performances of ML vary depending on its feature input. Some studies have found inconsistent accuracy when using single or multiple types of features. Some studies have reported greater accuracy in models with a single type of feature model [34], while others have concluded that the models incorporating multiple types of features are more effective [21,24,26]. Despite the remaining discrepancies in the existing research, integrating multiple types of features for predictive models appears to yield better results overall.

This review also summarized the longitudinal studies on predicting cognitive impairment and conversion in PD [22,31,36-39]. We discovered that neuroimaging features and biofluid markers are robust predictors. With the advancement of imaging technology, such as functional MRI and magnetic resonance spectroscopy, and their noninvasive detection capabilities, neuroimaging has become mainstream today. Neural networks and deep learning are especially focused on applications in imaging. Numerous research studies have suggested that biomarkers such as Aβ, -syn, and tau, play an essential role in the pathologic changes in PD. The detection of these biomarkers in fluids such as cerebrospinal fluid and blood becomes a potential approach to improving the accuracy of ML in diagnosing and predicting the status of cognitive impairment and disease progression.

A personal review proposed new diagnostic criteria: a 3-component system (SynNeurGe). It includes the harmful α-synuclein (S) accumulated in tissues or cerebrospinal fluid, evidence of neurodegeneration (N) in imaging, and the disease-causing gene variants (G) for PD. They are associated with clinical symptoms, defined either by a single highly specific clinical feature or by multiple less specific clinical features. A biological classification will aid in both basic and clinical research and bring the field closer to the precision medicine needed to develop disease-modifying therapies [62]. Consequently, a universally applicable ML model that integrates these significant variations can lead to personalized diagnosis and prediction, providing immense value for early detection, intervention, and management.

A large number of studies have investigated the diagnostic accuracy of ML to diagnose cognitive impairment in patients with PD. With the increasing number of deep learning and detection approaches, a more comprehensive exploration of ML was conducted. We used bivariate analysis to show more robust results. However, there are still several limitations in this review. First, we excluded nonpublished data in the meta-analysis. Due to heterogeneity in intervention methods and outcome measures between studies, it was not possible to synthesize all articles included using meta-analysis. Second, several studies used Parkinson’s Disease Progression Markers Initiative datasets, as they focus on the longitudinal cognitive change. However, we did not analyze the predictive power of ML in the follow-up cohort [22,31,36,38-46]. We are also limited to analyzing the optimum models and the predictors. Further investigation is required to determine the most effective model or predictors for identifying patients with PD who have cognitive impairment, predicting the disease progression (including conversion to dementia), and providing personalized treatment options.

ML algorithms have been proven to be highly effective in diagnosing cognitive impairment in patients with PD, especially MCI, compared to normal cognitive function. Furthermore, ML holds significant potential for predicting the cognitive decline in patients with risk factors and the transition from MCI to dementia.