This systematic review was registered in PROSPERO (International Prospective Register of Systematic Reviews) in March 2026 (CRD420261333545). No separate review protocol was prepared or made publicly available beyond the PROSPERO registration record. No amendments to the registered methods were made after registration. This systematic review and meta-analysis was conducted and reported in accordance with PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) 2020 and guidance for systematic reviews and meta-analyses of prediction model performance [11]. The completed PRISMA 2020 checklist is provided in the Checklist 1. We searched the Cochrane Library, PubMed, Web of Science, and Embase from inception to January 23, 2025, using database-specific combinations of controlled vocabulary and free-text terms related to AKI, ML, and prediction models. No additional trial registers, websites, organizational sources, or reference lists were searched. The complete search strategies are provided in Table S1 in Multimedia Appendix 1.

Two reviewers independently screened titles, abstracts, and full texts, with disagreements resolved by discussion or consultation with a third reviewer. Eligible studies developed or validated an AI- or ML-based multivariable prediction model for AKI occurrence or post-AKI mortality and reported AUC. We excluded reviews, editorials, guidelines, studies without full text, nonprediction model studies, studies without eligible outcomes or ML-based prediction models, and studies with fewer than 200 participants. Reference management software (EndNote 20) was used to remove duplicates and manage records.

Two reviewers independently extracted study characteristics, population and outcome definitions, data source, validation strategy, model type, sample size, number of outcome events, and model performance metrics using a standardized form. AKI occurrence and post-AKI mortality time horizons were extracted as reported in the original studies. Disagreements were resolved by discussion or consultation with a third reviewer. Study investigators were not contacted for additional information. When information was missing or unclear, data were recorded as not reported or unclear rather than imputed. AUC and its variance information were extracted when available. Other model performance information, when reported, was recorded descriptively. Because calibration measures, decision-curve analyses, operating thresholds, and clinical utility outcomes were not consistently available and were not included in the primary quantitative synthesis, the meta-analysis focused on AUC as the main discrimination metric. Details of extracted variables are provided in Table S2 in Multimedia Appendix 1. Clinical application scenarios were classified into four mutually exclusive categories. Detailed operational definitions are provided in Table S2 in Multimedia Appendix 1.

Risk of bias and applicability were assessed independently by 2 reviewers using PROBAST+AI (Prediction Model Risk Of Bias Assessment Tool+Artificial Intelligence), covering participants and data sources, predictors, outcomes, and analysis [12]. Disagreements were resolved by discussion or consultation with a third reviewer. Overall risk of bias was determined according to PROBAST+AI guidance. The PROBAST+AI signaling questions are listed in Tables S3 and S4 in Multimedia Appendix 1, and study-level assessments are provided in Table S6 in Multimedia Appendix 1.

We prespecified one representative AUC estimate per study and per outcome for the primary meta-analysis, selected according to the following validation hierarchy: external validation, internal validation, test cohort, training cohort, and other or unclear cohorts. When multiple eligible estimates were available within the same validation category, the estimate with the highest reported AUC was selected. A total of 219 studies were included for qualitative synthesis. Two records assessing AKI incidence were excluded from quantitative pooling owing to missing AUC values, leaving 217 studies with eligible and complete AUC data. As 2 studies reported both end points simultaneously, we ultimately obtained 219 outcome-specific analytical records, consisting of 188 records for AKI occurrence and 31 records for post-AKI mortality, all of which were incorporated into the quantitative meta-analysis. For complementary model-level analyses, the 1881 extracted model performance records were collapsed by article, outcome, and model or modeling approach, yielding 404 AUC records for AKI occurrence and 72 AUC records for post-AKI mortality. We identified 101 distinct modeling approaches to describe algorithmic diversity, but these were not treated as independent units in the primary meta-analysis. The 101 distinct modeling approaches were defined at the extracted algorithm-specific or study-reported model name level rather than at the level of the 4 broad model classes. For example, random forest, XGBoost (Extreme Gradient Boosting), LightGBM (Light Gradient Boosting Machine), support vector machine, multilayer perceptron, convolutional neural network, recurrent neural network, and study-specific hybrid pipelines were counted as distinct modeling approaches when reported as separate models by the original studies. These approaches were then mapped into 4 broad AI model classes—linear/generalized linear models, tree-based and ensemble methods, deep learning models, and hybrid models or others—for subgroup and meta-regression analyses.

Random-effects meta-analyses were performed using logit-transformed AUCs with restricted maximum likelihood estimation of between-study variance. Confidence intervals and statistical tests for pooled estimates, subgroup analyses, sensitivity analyses, and meta-regression were calculated using the Hartung-Knapp-Sidik-Jonkman (HKSJ) adjustment, and results were back-transformed to the AUC scale for presentation [13]. Standard errors were derived preferentially from reported 95% CIs, followed by the Hanley-McNeil approximation based on event and nonevent counts, and then reported SDs when available. AUC estimates without derivable variance information were excluded from quantitative pooling. The distribution of variance information sources used to derive standard errors is summarized in Table S10 in Multimedia Appendix 1. Subgroup differences were assessed using HKSJ-adjusted omnibus tests. In multivariable meta-regression, category-specific coefficients were assessed using HKSJ-adjusted t tests, global moderator effects were assessed using HKSJ-adjusted omnibus F tests, and complete cases were used for all included moderators. Small-study effects and funnel plot asymmetry were examined using funnel plots, Egger regression test, and trim-and-fill analysis when at least 10 effect sizes were available [14,15]. Sensitivity analyses were restricted to studies rated as low overall risk of bias.

Statistical analyses were performed using R (version 4.5.3; R Foundation for Statistical Computing) with the metafor package.

Not applicable because this study is a systematic review and meta-analysis of published studies.

Figure 1 shows the study selection flowchart according to the PRISMA statement. A total of 219 studies involving 7,343,170 patients were included in this review. The basic characteristics of each included study are detailed in Table S5 in Multimedia Appendix 1, with full references provided in Table S8 in Multimedia Appendix 1. Full references for excluded full-text articles and detailed reasons for exclusion are provided in Table S9 in Multimedia Appendix 1.

The characteristics of the included studies and prediction models are summarized in Table 1. This review included 219 studies and 101 distinct prediction models or modeling approaches. Overall, 188 outcome-specific model records addressed AKI occurrence prediction and 31 addressed post-AKI mortality prediction; 2 studies reported both outcomes.

For AKI occurrence prediction, tree-based and ensemble methods were the most commonly used approaches (94/190, 49.5%), followed by linear or generalized linear models (62/190, 32.6%), hybrid or other methods (23/190, 12.1%), and deep learning models (11/190, 5.8%). For post-AKI mortality prediction, tree-based and ensemble methods were also predominant (21/31, 67.7%), followed by linear or generalized linear models (7/31, 22.6%) and hybrid or other methods (3/31, 9.7%).

Most model records were based on retrospective cohorts, including 92.6% (176/190) of AKI occurrence records and 93.5% (29/31) of mortality records. AKI occurrence models mainly used in-hospital data (146/190, 76.8%), whereas mortality models more often used public databases (17/31, 54.8%). External validation was reported in 16.3% (31/190) of AKI occurrence records and 29.0% (9/31) of mortality records.

The cumulative sample size was 7,335,791 for AKI occurrence prediction and 417,999 for post-AKI mortality prediction. The main application scenarios for AKI occurrence prediction were early risk stratification (75/190, 39.5%) and perioperative monitoring (68/190, 35.8%), whereas mortality prediction models were mainly developed for early risk stratification (19/31, 61.3%).

Disease contexts were heterogeneous. AKI occurrence models were most often developed in cardiac or cardiac surgery-related populations, sepsis or infection-related cohorts, and trauma or noncardiac surgery settings, with smaller numbers addressing shock or hypoperfusion, respiratory failure or acute respiratory distress syndrome, liver disease, nephrotoxic exposures, severe acute pancreatitis, hematologic or malignancy-related conditions, cerebrovascular disease, and mixed populations. Mortality prediction models more commonly involved sepsis or infection-related cohorts and mixed clinical populations.

Risk of bias assessment based on the PROBAST+AI tool showed that 58.9% (129/219) of included studies were rated as low overall risk of bias, 38.4% (84/219) as having high risk of bias, and 2.7% (6/219) as having unclear risk of bias. At the domain level, most studies were rated as low risk in participants and data sources, predictors, and outcomes. Potential bias was most frequent in the analysis domain, where 36.5% (80/219) of studies were rated as having high risk.

When stratified by outcome, 56.3% (107/190) of AKI occurrence prediction records were at low overall risk of bias, 40.5% (77/190) were at high risk, and 3.2% (6/190) had unclear or insufficient information. For post-AKI mortality prediction, 71.0% (22/31) of records were at low risk of bias, and 29.0% (9/31) were at high risk. Because 2 studies contributed to both outcome strata, their evaluation results were counted in both the AKI occurrence subgroup and the post-AKI mortality subgroup. Graphical summaries and traffic-light plots are provided in Figure S1 in Multimedia Appendix 1.

Using random-effects meta-analysis with HKSJ-adjusted inference, the pooled AUC for AKI occurrence prediction was 0.834 (95% CI 0.821‐0.846; k=188; I²=100.0%), with a 95% prediction interval (PI) of 0.602 to 0.943. AUC estimates from individual studies ranged from 0.654 to 0.995. The forest plot for the pooled AUC analysis of AKI occurrence prediction models is presented in Figure S2 in Multimedia Appendix 1. For post-AKI mortality prediction, the pooled AUC was 0.830 (95% CI 0.807‐0.851; k=31; I²=99.0%), with a 95% PI of 0.675 to 0.920. Individual study-level AUC estimates ranged from 0.720 to 0.943. The corresponding forest plot is shown in Figure S3 in Multimedia Appendix 1.

Substantial between-study heterogeneity was observed in both analyses. We therefore conducted subgroup analyses to compare pooled AUCs across predefined study-level and model-level characteristics and to explore potential sources of between-study heterogeneity.

We stratified discrimination performance according to AI model application type (Table 2). Using HKSJ-adjusted random-effects models, deep learning models had the highest pooled AUC point estimate for AKI occurrence prediction (0.867; 95% CI 0.797‐0.915), followed by tree-based and ensemble methods (0.851; 95% CI 0.833‐0.868), hybrid models and other approaches (0.827; 95% CI 0.788‐0.860), and linear or generalized linear models (0.797; 95% CI 0.776‐0.816). For post-AKI mortality prediction, hybrid models and other approaches had the highest pooled AUC point estimate (0.858; 95% CI 0.542‐0.969), followed by tree-based and ensemble methods (0.831; 95% CI 0.803‐0.855) and linear or generalized linear models (0.814; 95% CI 0.747‐0.866).

Subgroup analyses of pooled AUC for AKI occurrence and post-AKI mortality prediction models are shown in Figures 2 and 3, respectively. AI model application type was associated with discrimination performance for AKI occurrence prediction (F_3,184=4.6891; P=.004). Nonlinear approaches, particularly deep learning and tree-based and ensemble methods, generally showed higher pooled AUC point estimates than linear or generalized linear models. Subgroup differences by clinical application scenario, disease group, and AKI frequency group were not statistically significant for AKI occurrence prediction (P=.23, P=.12, and P=.20, respectively).

For post-AKI mortality prediction, subgroup differences by AI model application type were not statistically significant (P=.55). In-hospital data showed a numerically higher pooled AUC point estimate than public databases (0.849 vs 0.815), but the between-group difference by data source did not reach statistical significance (P=.11). Subgroup differences by clinical application scenario and disease group were also not statistically significant (P=.19 and P=.07, respectively). Only post-AKI mortality frequency group showed a borderline subgroup-level difference (P=.05). Complementary model-level analyses including all available model records yielded lower pooled AUCs than the representative-estimate analyses for both outcomes and are reported in Table S7 in Multimedia Appendix 1 and Figures S4 and S5 in Multimedia Appendix 1.

In multivariable meta-regression using complete cases for all included moderators (Table 3), AI application group and disease group were associated with logit-transformed AUC for AKI occurrence prediction. The global test for AI application group was statistically significant (P=.02). Compared with deep learning models, linear or generalized linear models had lower logit-transformed AUCs (coefficient=−0.526; 95% CI −0.982 to −0.070; P=.02). Disease group was also associated with logit-transformed AUC in the global test (P=.04), with trauma- or noncardiac surgery–related models showing higher logit-transformed AUCs compared with the reference disease group (coefficient=0.598; 95% CI 0.187 to 1.010; P=.005). Data-splitting strategy showed a borderline global association (P=.05). Data source, dataset type, clinical application scenario, AKI frequency group, risk of bias, and patient population were not significantly associated with logit-transformed AUC in the global tests.

For post-AKI mortality prediction, no moderator showed a statistically significant global association in the multivariable meta-regression. Data-splitting strategy (P=.16), dataset (P=.30), AI application group (P=.21), clinical application scenario (P=.52), disease group (P=.26), AKI frequency group (P=.81), risk of bias (P=.45), and patient population (P=.44) were not significantly associated with logit-transformed AUC in the global tests. A forest plot summarizing all multivariable meta-regression coefficients with 95% CIs is provided in Figure S6 in Multimedia Appendix 1.

In sensitivity analyses restricted to studies at low overall risk of bias, 105 AKI occurrence prediction records and 22 post-AKI mortality prediction records were included. The pooled AUCs were similar to the main analyses: 0.832 (95% CI 0.814‐0.849) for AKI occurrence prediction and 0.819 (95% CI 0.792‐0.843) for post-AKI mortality prediction. Heterogeneity remained high, with I² values of 100.0% and 97.4%, respectively. The corresponding forest plots are presented in Figure S7 in Multimedia Appendix 1.

Small-study effects and funnel plot asymmetry were examined using funnel plots, Egger regression test, and the trim-and-fill method (Figure 4). For AKI occurrence prediction models, visual inspection of the funnel plot suggested asymmetry, and Egger regression test was statistically significant among 188 effect sizes (test statistic=2.814; P=.005). However, the trim-and-fill method did not impute potentially missing studies. These findings suggest small-study effects or funnel plot asymmetry. For post-AKI mortality prediction models, there was no clear evidence of small-study effects: the funnel plot appeared broadly symmetrical, Egger regression test was not significant among 31 effect sizes (test statistic=0.062; P=.95), and trim-and-fill did not identify missing studies.

In this large-scale systematic review and meta-analysis of ML models for AKI prediction, analyses using logit-transformed AUCs, restricted maximum likelihood random-effects models, and HKSJ-adjusted inference showed high average discrimination for both AKI occurrence and post-AKI mortality. The pooled AUCs were 0.834 for AKI occurrence prediction and 0.830 for post-AKI mortality prediction. However, substantial heterogeneity and several sparse subgroup categories mean that between-group findings should be interpreted cautiously and as exploratory rather than confirmatory. These results support the potential value of ML-based approaches for AKI risk stratification and prognostic assessment, while also highlighting the need to interpret performance within specific clinical settings, validation designs, and implementation contexts.

Although the pooled AUCs were high, they should be interpreted as estimates of average discrimination across a clinically and methodologically diverse evidence base, rather than as guaranteed performance in any single institution or patient population. This interpretation is consistent with guidance for meta-analyses and external validation of prediction models, in which differences in case mix, outcome definitions, prediction horizons, and validation designs may strongly influence apparent model performance [16,17]. Future studies should therefore report discrimination together with calibration, clinically meaningful thresholds, decision-analytic measures, and transparent reporting according to contemporary prediction model guidance [18-20].

For AKI prediction, clinical usefulness depends on whether risk estimates are early enough, reliable enough, and actionable enough to change care [4,21]. A clinically useful model should be linked to predefined kidney-protective actions, such as reassessing nephrotoxic medications, optimizing hemodynamics and fluid balance, increasing creatinine and urine output monitoring, reviewing contrast exposure, or prompting earlier nephrology consultation. Without such links to workflow and action, even a well-discriminating model may remain a retrospective ranking tool rather than an AKI clinical decision-support intervention.

For AKI occurrence prediction, AI model class was associated with discrimination performance in both subgroup analysis and multivariable meta-regression using HKSJ-adjusted inference. Linear or generalized linear models showed lower logit-transformed AUCs than deep learning models in multivariable meta-regression, while nonlinear approaches generally showed higher pooled AUC point estimates in subgroup analyses. This pattern is biologically and clinically plausible, given the multifactorial nature of AKI. However, wide PIs, uneven subgroup sizes, and high residual heterogeneity indicate that these findings should not be interpreted as evidence that any model family is universally superior across clinical settings. For AKI clinical decision support, better discrimination alone is not enough to justify the adoption of more complex models unless they provide calibrated, timely, interpretable, and actionable predictions across institutions.

Subgroup analyses did not show statistically significant differences in pooled AUC across the 4 clinical application scenarios. However, this should not be interpreted as evidence of equivalent clinical usefulness. Early risk stratification, perioperative monitoring, ICU real-time alerting, and specialty-specific prediction differ in prediction timing, data requirements, intervention opportunities, and consequences of false-positive or false-negative alerts. This nonsignificant result may arise because AUC has limitations in evaluating clinical application and model calibration; it cannot effectively indicate performance differences among ML models under different clinical conditions. Thus, clinical application scenario remains important when evaluating implementation readiness.

For post-AKI mortality prediction, moderator findings were less stable than those for AKI occurrence prediction. In subgroup analyses, disease context was not statistically associated with pooled AUC, whereas AKI frequency group showed only a borderline subgroup-level difference. In multivariable meta-regression, no moderator showed a statistically significant global association with logit-transformed AUC for post-AKI mortality prediction. Given the small number of eligible mortality estimates and several sparse categories, these findings should be interpreted cautiously and considered exploratory rather than confirmatory. Mortality models may capture broader illness severity rather than AKI-specific prognosis alone, which may further contribute to heterogeneity in post-AKI mortality prediction.

The complementary model-level analysis yielded lower pooled AUCs than the representative-estimate analysis for both AKI occurrence and post-AKI mortality prediction. This difference may reflect the inclusion of lower-performing models reported within the same studies. Therefore, the representative-estimate analysis should be interpreted as study-level prioritized performance, whereas the model-level analysis provides a broader but more conservative summary of all reported model estimates.

Sensitivity analyses restricted to studies at low overall risk of bias yielded pooled AUCs similar to those in the main analysis, supporting the stability of the average discrimination estimates after excluding high-risk-of-bias studies. However, heterogeneity remained substantial, indicating that risk of bias was not the only source of between-study variability. Differences in model type, study population, data source, validation method, prediction horizon, and AKI definition may still have influenced the pooled results.

Evidence of small-study effects or funnel plot asymmetry was observed for AKI occurrence prediction but not for post-AKI mortality prediction. These findings should not be interpreted as direct evidence of publication bias. Publication bias represents only one possible explanation for small-study effects; other explanations include clinical and methodological heterogeneity, differences in validation strategies, outcome definitions, model-development practices, selective reporting, and chance.

Alert fatigue is an important consideration for the clinical implementation of ML-based AKI prediction models. Patients at AKI risk often have complex clinical profiles that already trigger frequent electronic health record alerts. Models with suboptimal specificity, calibration, or overly sensitive thresholds may produce excessive false-positive alerts, causing clinician desensitization and reduced responses to genuine AKI risks. Therefore, model performance should not be evaluated solely by AUROC-based discrimination. Future studies must additionally assess calibration, false-alert burden, clinical use, and workflow impact. To reduce alert fatigue, AKI prediction models require rigorous external validation, optimized thresholds, and integrated, actionable clinical workflows rather than standalone risk alerts.

This review has several strengths. First, it provides one of the largest quantitative syntheses of ML models for AKI prediction to date [22-24]. Second, it analyzed AKI occurrence and post-AKI mortality separately, avoiding the combination of clinically distinct prediction tasks. Third, it examined discrimination performance according to both clinical application scenario and model class, which improves clinical interpretability. Fourth, it combined quantitative performance synthesis with PROBAST+AI-based risk of bias assessment and validation characteristics.

Several limitations should be acknowledged. First, substantial heterogeneity was present across included studies. This was expected given differences in AKI definitions, prediction horizons, clinical settings, patient populations, data sources, validation strategies, and model classes. Therefore, pooled AUCs should be interpreted as average discrimination across a broad evidence base rather than as setting-specific expected performance.

Second, many models were developed and evaluated using retrospective data, whereas external or prospective validation was relatively limited. This limits confidence in their transportability across different institutions, health care systems, and patient populations. Therefore, further external and prospective validation is needed before these models can be considered for broader clinical implementation.

Third, most studies primarily reported discrimination, commonly using AUC, whereas other important aspects of model performance, such as calibration and clinical usefulness, were less consistently reported. These are not only limitations of this review but also important findings about the current AKI prediction literature. The inconsistent reporting of calibration, thresholds, decision-analytic evaluation, and clinical impact prevented quantitative synthesis of implementation outcomes and highlights the need for future studies to move beyond discrimination-centered reporting.

Fourth, our representative AUC selection rule may have introduced optimistic selection bias. To reduce within-study dependence, we selected one AUC per study and outcome according to a prespecified validation hierarchy. When several eligible AUCs were available within the same validation tier, the highest value was used because many studies did not clearly identify a primary model, prediction horizon, or clinically intended final model. This approach may have favored better-performing models and inflated the pooled AUCs. Accordingly, these estimates should be interpreted as study-level summaries rather than unbiased averages of all reported model results. The complementary model-level analysis, which included all retrievable AUCs, yielded lower pooled estimates but did not change the overall conclusions.

ML models showed high average discrimination for both AKI occurrence and post-AKI mortality in this large-scale systematic review and meta-analysis. Performance patterns differed by model class and clinical prediction task, with nonlinear approaches showing relatively higher pooled AUC point estimates in exploratory analyses for AKI occurrence, although between-group findings should be interpreted cautiously. These findings support the potential value of ML for AKI risk stratification and prognostic assessment. However, heterogeneity across studies and limited reporting of external validation, calibration, thresholds, alert burden, and clinical utility indicate that future research should move from discrimination-centered model development toward validated, calibrated, and clinically actionable AKI decision-support tools.