Investigating Protective and Risk Factors and Predictive Insights for Aboriginal Perinatal Mental Health: Explainable Artificial Intelligence Approach

Introduction

Perinatal depression and anxiety (PNDA) negatively impact the health and well-being of mothers and babies, and disrupt maternal /infant bonding [1]. Recent studies highlighted the significant association between PNDA and adverse outcomes, including suicidal behaviors and self-harm thoughts during and after pregnancy. Roddy Mitchell et al [2] emphasized the increased risk of preterm birth, stillbirth, and suicide associated with PNDA. Furthermore, Hummel et al [3] highlighted the association of PNDA with adverse infant outcomes such as preterm birth, intrauterine growth restriction, and low birth weight. The loss of an infant's mother through suicide profoundly impacts the infant’s social and emotional well-being [4]. Many Aboriginal women experience strong social and emotional well-being and have flourishing infants and families. However, at a population level, too many Aboriginal women face the increased risk of triggering or worsening depression/anxiety directly resulting from the enduring challenges, barriers, and adversities from colonization, cultural disruption, and past and present policies such as the Stolen Generations. These interrelated risks include poverty, racism, intergenerational and complex trauma, racism, cultural bias, loss of cultural identity, and other inequities [5,6]. A systematic review by Owais et al [7] indicated Aboriginal women face a 38% higher chance of experiencing depression, are 79% more susceptible to mental-health problems during pregnancy, and are 30% more likely to endure mental health complexities post giving birth. A Western Australian study revealed that between 1997 and 2013, one in 3 Aboriginal babies were born to mothers who sought hospital care for mental health issues related to substance abuse, depression, or anxiety [8]. Despite routine screening for PNDA and anxiety in Australia for over 20 years, the gap in Aboriginal mothers’ and infants’ health and well-being remains unacceptable across all key indicators. This is evident in disproportionately higher rates of premature births, low birth weight babies, and child removal [5,6]. Conventional health systems’ approach to antenatal/postnatal care and screening are often culturally insensitive and retraumatizing for Aboriginal women [9]. Risk-focused perinatal screens and assessments with Aboriginal women frequently exacerbate feelings of alienation and disengagement from potentially supportive care [10]. There is an urgent need for culturally safe and effective trauma-aware and healing-informed screening for social, emotional, mental-health and well-being that includes relevant supportive and strength-based follow-up care for Aboriginal mothers.

The Baby Coming You Ready (BCYR) program [10], was co-designed to overcome these barriers and challenges faced by Aboriginal parents during their perinatal care. The BCYR program focuses on a digitized, strengths-based, culturally safe, and holistic perinatal assessment that incorporates all 7 elements of Aboriginal peoples’ social and emotional well-being [11]. The assessment uses iPads with touchscreen images depicting common strengths, worries, and past and present occurrences. Aboriginal voice-overs accompany each slide to guide reflective engagement between the mother and her midwife or health professional. Mothers choose images they relate to while engaging in self-reflection, creating their own personalized story, then prioritizing their strengths and concerns, and designing their own solutions. The assessment automatically generates a clinical event summary, which serves as an individualized follow-up management plan for the mother and health professional. Currently, the BCYR program is operationalized as a model of care in all 6 pilot sites in Western Australia (WA) and effectively replaces all currently required screens for mental-health; family and domestic violence; tobacco; and alcohol and other drugs. While the successful pilot demonstrated increased trust, engagement, honest disclosure, and self-directed management plans, it found that some midwives and managers lacked confidence in conducting culturally considered holistic assessments [10]. Traditional perinatal mental health assessments primarily focus on risk factors, which continue to influence clinical practice and often lead clinicians to overemphasize risk scores and prioritize risk-based discussions during consultations. This reliance may limit trauma-aware and healing-informed care, particularly for Aboriginal mothers [7], highlighting the need for approaches that better support culturally responsive and strengths-based assessments [12].

Over the last decade, advances in digital health and computational technology have driven numerous studies on technology-based approaches to supporting perinatal mental health [13,14]. Among these, artificial intelligence (AI)–based models particularly those using machine learning (ML) and deep learning, have been developed to predict perinatal mental health conditions [15-20]. These models demonstrate the potential to enhance clinical practice by enabling early and accurate detection of depression, facilitating better clinical judgment, and identifying patterns that may be overlooked in manual assessments [21]. Despite these advancements, progress in applying AI to improve health outcomes in Aboriginal populations has been limited [22]. ML models typically trained on data from the general population, often lack cultural relevance and fail to account for unique protective and risk factors, as well as the social determinants of health specific to Aboriginal communities. Moreover, many AI models function as “blackbox” due to their lack of interpretability [23]. However, model transparency is especially critical in health care, particularly for underrepresented populations, where trust and clarity are essential. Explainable artificial intelligence (XAI) has emerged as a promising approach that provides clear explanations for AI and ML algorithm predictions and decision-making processes [24,25]. Such techniques have been successfully applied in various health applications and predictive modeling [26-29].

This study aims to explore different ML techniques to develop a culturally informed, strengths-based AI model for predicting perinatal psychological distress in Aboriginal mothers. The model is built using holistic and culturally contextualized assessment data from the BCYR program. To enhance transparency and clinical relevance, XAI techniques are incorporated to provide clear reasoning behind AI-driven decisions. This approach helps identify, prioritize, and quantify both maternal protective and risk factors, as well as their interactions and impact on perinatal mental health outcomes. By offering deeper insights into Aboriginal perinatal mental health, this model may support more holistic and culturally responsive assessments, ultimately improving clinical decision-making.

Methods

Setting and Data Source

The dataset used in this study consists of de-identified data collected from the WA BCYR pilot program [30]. The dataset includes 293 Aboriginal mothers who participated between September 9, 2021, and June 16, 2023, across 6 diverse pilot sites services in metropolitan Perth and regional WA. The BCYR assessment/screen is being offered to all women at pilot sites as part of their routine perinatal care in an additional 30-minute stand-alone appointment [10]. All pregnant women or mothers with infants who accessed participating perinatal services at the pilot sites were eligible to take part in the BCYR assessment. The sampling approach was convenience-based, with the BCYR assessment offered to all eligible Aboriginal mothers attending the pilot sites during the study period.

Ethical Considerations

Ethics approval for this study was obtained from the Human Research Ethics Committee (HREC) - Western Australia Research Governance Service (RGS000000649), Murdoch University (2021/101), and the Western Australian Aboriginal Health Ethics Committee (WAAHEC; HREC553). Access to deidentified data was granted only to participants who provided informed consent. The information sheet, and a consent button, allowing participants to choose whether or not to participate and share their deidentified data for research purposes, were embedded in the digital application.

Data Preparation and Preprocessing

Observational units were individual patients, with response variables (psychological distress risk) and predictor variables (demographic, social, and behavioral factors). The skipped question’s answer by the participants was assigned a value of –1.

Predictors

The original dataset contains 345 variables for each participant, covering a wide range of inquiry domains such as strengths and culturally protective factors, common life events, worries, quality of relationships, childhood experiences, family and domestic violence, and tobacco and alcohol and other drug use. Feature selection was performed using the RF to compute variable importance ranking. The algorithm was configured with 500 trees, and the “mtry” parameter was set to the square root of the total number of variables, rounded down to the nearest integer. Initially, the top 30 most significant variables were selected. These variables were then reviewed by the BCYR research team, which included Aboriginal researchers, both Aboriginal and non-Aboriginal health care professionals, and BCYR digital assessment users. Incorporating their domain knowledge and experience, the final list was narrowed down to 20 predictor variables along with the Kessler-5 item psychological distress scale (K5) output variable, for analysis. The final selected variables are listed in Table 1.

Outcome Variable

The indicator for maternal psychological distress is based on the K5 scale [31]. The K5 scale consists of 5 items, each rated on a 5-point scale from 1 to 5, with all items negatively keyed. The total score ranges from 5 to 25, with a score below 12 indicating low risk (0), and a score of 12 or higher indicating high risk (1) [32]. Five records were excluded from the original dataset due to missing information on the K5 outcome variable. The dataset had a class ratio of 0.65 (low risk) to 0.35 (high risk).

Model Development and Building

We used 7 ML models to train and evaluate the prediction model on the processed dataset. These models were chosen due to their widespread use and proven effectiveness in health care predictive modeling, particularly with relatively small tabular data [33-35]. The models include random forest (RF) [36], CatBoost (CB) [37], light gradient-boosting machine (LightGBM) [38], extreme gradient boosting (XGBoost) [39], k-nearest neighbor (KNN) [40], and support vector machines (SVM) [41] as blackbox models, and one inherently interpretable glassbox model explainable boosting machines (EBMs) [42]. More descriptions of these selected models are provided in Section I in Multimedia Appendix 1.

Prediction Performance Evaluation

We used a 10-fold cross-validation strategy for splitting the training and testing data to ensure a fair model training and evaluation process. Hyperparameters for each model were tuned using grid search with cross-validation. Details for hyperparameter tuning were provided in Section III in Multimedia Appendix 1.

Ten-fold was adopted as it is more suited for relatively small datasets, offering a better balance between bias and variances [43]. We then report the average model performance using a comprehensive set of metrics, including accuracy, precision, recall, F₁-score, and area under the curve (AUC). Precision, recall, F₁-score, and AUC are particularly recognized as robust metrics when dealing with imbalanced class ratios. Additionally, 95% CIs are provided for each performance metric to account for uncertainty.

Model Explanation

Different model explanation techniques were used to investigate how factors influence psychological distress and to compare their outputs. First, EBM, as a glassbox model, is inherently explainable and provides both global explanations for the model’s overall behavior, and local explanations for specific predictions on individual cases [42]. We also applied post-hoc explanation techniques, including Shapley additive explanations (SHAP) [44], local interpretable model-agnostic explanations (LIME) [45,46], and partial dependence plots (PDP) [47], to elucidate the predictions of the best-performing blackbox model. SHAP and LIME can offer both global and local explanations. PDP, as a visualization technique, shows how 1 or 2 selected features impact the predicted outcome while keeping all other features constant. More details on these post-hoc explanation techniques are provided in Section II in Multimedia Appendix 1.

Results

Prediction Model Performance

Table 2 displays the training and evaluation results of all the ML models across various performance metrics, with the best results highlighted in italics. Among black-box models, RF achieved the highest performance, with an accuracy of 0.829, an F₁-score of 0.736, and an AUC of 0.795. The glass-box model EBM outperformed all models, attaining an accuracy of 0.849, an F₁-score of 0.771, and an AUC of 0.821. Ensemble models, including RF, CB, XGBoost, and LightGBM, demonstrated strong predictive performance, all achieving an accuracy above 0.81. KNN and SVM showed slightly lower accuracy (0.798 and 0.794) and comparable AUC values (0.733 and 0.742). In terms of precision and recall, KNN exhibited the highest precision (0.868) but the lowest recall (0.514), leading to a lower F₁-score (0.621). EBM and RF achieved a better balance, with EBM attaining a precision of 0.829, a recall of 0.727, and an F₁-score of 0.771, while RF reached a precision of 0.820, a recall of 0.680, and an F₁-score of 0.736. Figure 1 plots the receiver operating characteristic curve curves of all the models on one set of testing, showing that EBM and RF achieved the highest AUC values.

‎

Explanation Results

Global Interpretation by EBM, SHAP, and RF

As EBM demonstrates high predictive performance and operates as a transparent glassbox model, we generated the explanation from EBM and illustrated the global feature importance over the whole dataset in Figure 2. Longer bars in the figure indicate the higher importance of features in the model’s predictions. It is noteworthy that specific features exert a more significant influence on the decision-making process of the EBM model. For instance, the feature “Feeling Lonely” emerges as the most impactful, suggesting that a participant’s response to a reflection concerning feelings of loneliness, might serve as a robust predictor for perinatal mental health risk. This is followed by “Blaming Herself” and “Makes Family Proud,” indicating that these 2 features have a higher contribution to the overall prediction (risk or protective). Other features, such as “Managing Day-to-Day” and “Strong Mum,” also demonstrate a notable impact in relation to the K5 target outcome.

‎

For the high-performing black-box RF model, we used SHAP to also gain insights into the model’s explanation. We displayed the global feature importance ranking in Figure 3. The most important features, including “Feeling Lonely,” “Blaming Herself,” “Managing Day-to-Day,” “Life Not Worth Living,” and “Makes Family Proud,” exactly overlap with the top features selected from EBM. Additionally, features related to questions concerning “Family Gambles,” “Strong Mum,” “Goal Likely,” and “Trouble Sleeping” were subsequently ranked, aligning with EBM’s ranking tier as well. We also provided the RF feature importance rankings in Figure 4. The top-ranked features including “Feeling Lonely,” “Life Not Worth Living,” “Blaming Herself,” “Managing Day-to-Day,” “Strong Mum,” and “Trouble Sleeping” are largely consistent with the key features identified by SHAP and EBM. There are some variations in ranking. For example, RF assigns higher importance to “Life Not Worth Living” and “Managing Day-to-Day,” while SHAP emphasizes “Family Gambles” and LIME highlights interaction terms such as “Feeling Lonely” & “Life Not Worth Living.”

‎

‎

Local Interpretation by EBM, SHAP, and LIME

EBM can also provide an interpretation of the model’s prediction on an individual instance. Figure 5 shows one Aboriginal woman who is low risk, labeled “Instance I,” it shows the specific contributions of each feature and their interactions in combination toward risk prediction for this individual by EBM. EBM accurately predicts this individual as “low risk” (class=0), with a high probability score of 0.904. The y-axes of the plot show impactful features and their corresponding values in brackets. Specifically, contributing features highlighted in blue, such as “Blaming Herself”=1.00 (hardly ever) and “Feeling Lonely”=1.00 (hardly ever), contribute to shifting the model’s prediction toward the low-risk category. These 2 features stand out, suggesting that “hardly ever feeling lonely” and “hardly ever blaming herself” are strongly protective factors for this Aboriginal mother. In addition, the combination of “Blaming Herself”=1.00 (hardly ever) and “Family Gambles”=0.00 (loved ones do not gamble) shows an additive positive influence, pushing the prediction toward low risk. Conversely, contributing features displayed in orange in the figure, would have pushed the model’s prediction away from low risk. For example, “Children in Her Care”=3 (having 3 children in care) and “Strong Mum”=3 (sometimes), and “Makes Family Proud”=2 (often) and “Partner Moody”=1 (yes), are risk factors that increase this woman’s risk. In our analysis, we noted that “Makes Family Proud” and “Strong Mum” were highly protective only when women selected the top rating: “Always.”

Using SHAP, we generated force plots in Figure 6 to visualize the contributions of important features to the prediction for Instance I. The key features influencing the prediction for this individual are depicted in red and blue. Red indicates features that elevated the model’s score toward high risk. Blue signifies features that reduce the risk. Features having a greater impact on prediction scores are located closer to the dividing boundary between red and blue in the figure. Therefore, the strongly protective factors contributing to this low-risk prediction, including “hardly ever blames herself,” “hardly ever feels lonely,” and “close loved ones do not gamble,” align with the EBM’s individual interpretation. Additionally, “sometimes feels strong about being a mum” and “moody partner” tended to push the prediction toward higher risk, which again is similar to EBM’s interpretation.

‎

‎

We used LIME to further explore individual influencing features for the same woman. Overall, the results were very similar to EBM and SHAP technologies. Figure 7 illustrates that the most influential features in predicting low risk were “hardly ever feels lonely,” “hardly ever blames herself,” “never feels life is not worth living,” and “close loved ones do not gamble.” Interestingly LIME uniquely identified that “needs no help with housing” was a contributor to low risk, which was not highlighted by EBM or SHAP.

Figure 8 provides the EBM’s local explanation for another Aboriginal woman labeled “Instance II,” who was identified as being at higher risk. EBM accurately predicted the outcome with a probability score of 0.655. Several protective factors highlighted in blue contributed to mitigating the risk include “Blaming Herself”=1 (hardly ever blames herself), “Managing Day-to-Day”=0 (manages day-to-day well), “Life Not Worth Living”=1 (never feels life is not worth living), “Strong Mum”=1 (almost always feels strong about being a mum), “Children in Her Care”=1 (having one child in care), and “Previous Births”=1 (one previous birth). Several factors in red significantly influencing the prediction decision toward high risk include “Feeling Lonely”=2 (feels a little lonely), “Trouble Sleeping”=2 (having trouble sleeping), “Family Gambles”=2 (close loved ones gamble), “Makes Family Proud”=3 (sometimes makes her family proud), and “Goal Likely”=2 (a fair amount). While the protective features provided some mitigating effects, the stronger influence of risk factors ultimately influenced the model’s high-risk prediction.

‎

‎

Figure 9 shows the prediction interpretation for the same individual using SHAP. A consistent group of answers to the questions that drive the prediction toward a high risk was identified, including “Family Gambles,” “Feeling Lonely,” “Makes Family Proud,” “Goal Likely,” and “Trouble Sleeping,” although the significance level ranking shows slight differences. Meanwhile, not “Blaming Herself” and “Managing Day-to-Day” were identified as mitigating factors that attempt to reduce this high-risk prediction.

Figure 10 provides LIME’s local interpretation of the same individual. The protective factors largely align with the other 2 methods, except for “Need Help with Housing”=0, chosen as a protective factor by LIME, which was not picked as a top protective factor by the other 2 methods. LIME selected the same group of risk factors as SHAP, except for “Trouble Sleeping,” which was not chosen by LIME but was selected by both SHAP and EBM.

‎

‎

High Influential Factor Interpretation by PDP

Through interpretation conducted, we identified 2 highly significant questions impacting the prediction outcomes: “Blaming Herself” and “Feeling Lonely”. We further adopted their corresponding PDP plots to reveal their individual relationships with the average predicted outcome shown in Figure 11. In the case of “Blaming Herself,” mothers who hardly ever blame themselves are associated with a low risk. However, starting from “A Little” and progressing to “Sometimes,” “Often,” and “Almost Always,” there is a significant increase in the predicted risk of perinatal mental health issues. This suggests that as the frequency of self-blame rises, even starting from “A Little,” the associated risk shows a significant increasing trend. There is little difference in the impact between the categories “Sometimes,” “Often,” and “Almost Always,” as they all show a significant level of relation with higher risk. Similar observations were made for the case of “Feeling Lonely,” where mothers who reported “Hardly Ever” feeling lonely are associated with a low risk.

We further generated a PDP plot to visualize the interplay between these 2 significant factors. The heatmap color gradient represents the predicted risk level, with light cream indicating a low risk (closer to 0) and purple representing a high risk (closer to 1). Mothers who reported “Hardly Ever” blaming themselves and “Hardly Ever” feeling lonely are in the lightest zone, suggesting the lowest predicted risk. However, the deeper color zone is pronounced for respondents who reported feeling lonely starting from “A Little” and beyond and blaming themselves starting from “A Little” and beyond. This combined emotional state of frequent loneliness and self-blaming puts them at a much higher predicted risk for perinatal mental health issues. Regardless of whether the response was “Sometimes,” “Often,” or “Always” for loneliness and self-blame, it led to the highest level of risk, with almost the same effect.

‎

Discussion

Principal Findings

The prediction model performance analysis indicates that EBM and RF performed best overall, offering a strong balance between accuracy, F₁-score, and AUC. Ensemble models (RF, CB, XGBoost, and LightGBM) demonstrated strong predictive performance, benefiting from their ensemble-based architecture, which enhances generalization and robustness. While CB, XGBoost, and LightGBM are powerful, they may be more prone to overfitting on small datasets. In contrast, RF uses bagging, which reduces variance and tends to be more robust for small datasets. Among the models, KNN exhibited high precision but low recall, indicating a tendency to miss high-risk cases, which could be a critical limitation for mental health risk detection. In contrast, EBM and RF provided a better balance, making them more suitable for this task.

The global feature importance analysis using EBM, SHAP, and RF identified key predictors of perinatal mental health risk and revealed important interactions between features. The question concerning “Feeling Lonely” consistently emerged as the most influential predictor across models, followed by questions concerning “Blaming Herself,” “Makes Family Proud,” “Life Not Worth Living,” and “Managing Day-to-Day.” Moreover, EBM showed specific multidimensional interactions that add increased weighting to the model’s predictions. For example, “Feeling Lonely” in combination with “Life Not Worth Living,” “Blaming Herself,” or “Partner Angry/Controlling” placed greater predictive power. Similarly, interactions between “Blaming Herself” and “Family Gambles” or “Makes Family Proud” were identified as key joint effects affecting model predictions.

Variations in feature rankings across EBM, SHAP, and RF likely arise from methodological differences in how global feature importance is measured. RF determines importance based on node splits and impurity reduction but does not explicitly capture feature interactions. SHAP estimates feature contributions by considering their marginal effects and interactions across all predictions. EBM computes global importance by aggregating the learned effects of each feature in an additive framework while explicitly modeling interactions through pairwise terms. By using these different global feature importance methods, we can get a more comprehensive understanding of feature importance and uncover key patterns that enhance model interpretability.

Comparison to Prior Work

Recent advancements in AI-driven predictive models for perinatal mental health have demonstrated varying levels of effectiveness. Previous studies introduced an EBM trained with Aboriginal lived experiences, highlighting the need for culturally sensitive AI applications [27]. Similarly, another study showed that ML models, particularly RF and SVM, could effectively predict psychological distress among Aboriginal perinatal mothers [32]. Other studies have explored the broader application of AI in perinatal health, such as AI’s role in predicting preterm birth and postpartum depression [20,48]. Our study builds upon these findings by integrating XAI techniques and incorporating Aboriginal knowledge and lived experiences to improve transparency in decision-making and support the development of culturally safe AI applications in perinatal mental health.

At the individual level, where responses are highly personal, local explanations from XAI techniques provided case-specific insights, distinguishing between protective and risk factors and illustrating their respective contributions. In combination with global feature importance results, positive family relationships emerged as a key protective factor in mitigating poor perinatal mental health, aligning with findings from Ratajczak [9] and Carlin et al [12]. Similarly, risk factors such as feelings of loneliness and poor partner relationships were consistent with the findings of Carlin et al [12]. This study may offer new ways to identify protective and risk factors in Aboriginal perinatal mental health from an explainable AI-based quantitative perspective and predictive modeling approach. Such models could facilitate the early detection of at-risk individuals and support more personalized, culturally sensitive, strengths-based care.

Limitations and Future Directions

This study has several limitations that should be acknowledged. First, our model was trained on the dataset obtained through convenience sampling, without a formal sample size calculation. The limited sample size and nonrandom sampling approach may introduce selection bias, potentially limiting the model’s generalizability and increasing the risk of overfitting. While cross-validation techniques were applied to mitigate these risks and assess generalization capability, they cannot fully compensate for the limitations posed by the sampling method and dataset size. Second, the absence of established population parameters prevents direct statistical comparisons with broader populations, making it challenging to assess selection bias and affecting the study’s generalizability. Third, potential biases in assessment responses, such as nonresponse and social desirability bias, may affect data quality and influence model outputs. While XAI techniques provide a way to identify potential distortions, they do not fully quantify these biases, making it difficult to comprehensively assess model fairness and accuracy. Future research should focus on expanding the dataset by incorporating a more diverse and representative sample across different regions, performing external validation using data from different regions, and systematically assessing model fairness. These steps could help enhance the model’s performance, generalizability, and reliability in practice. Fourth, there is a slight imbalance in the outcome class ratio between low risk and high risk, at 0.65 versus 0.35. Given that the class ratio is relatively moderate and ML models especially ensemble techniques like RF can naturally handle this level of imbalance, and class imbalance-robust performance metrics were used for evaluation, no class imbalance techniques were applied. In the future, as the dataset grows and if the class imbalance increases, additional techniques could be implemented to further improve predictive performance. Fifth, the current visual outputs generated by the XAI techniques can be refined through the co-design process to improve their readability and explainability for Aboriginal women and clinicians. Creating a user-friendly, culturally sensitive visual prediction model will ensure that all practitioners can accurately and responsively interpret the results in practice.

Conclusions

We developed and evaluated several ML models powered by XAI techniques to predict perinatal psychological distress in Aboriginal mothers. The explanations provided by different XAI techniques revealed largely consistent patterns of influential protective and risk factors, their interactions, and their impact on prediction outcomes. Continuous collaboration informed by Aboriginal knowledge and lived experience, will further enhance the model. Such a model may have the potential to assist health care professionals in providing more culturally sensitive clinical reasoning, improving holistic assessment interpretations, and reducing unnecessary child protection notifications. Future studies are needed for clinical validation.