This systematic review was performed in accordance with the recommended PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines [13]. The PRISMA Checklist for this study is presented in Multimedia Appendix 1.

As no patients were involved in the study, this study was exempted from institutional review board approval.

The PubMed database was searched for citations published between January 1, 2000, and April 30, 2020. We used the National Library of Medicine Medical Subject Headings (MeSH) terms to search for citations indexed with “artificial intelligence” and at least one other MeSH term from the OB/GYN domain: “obstetrics”; “gynecology”; “reproductive techniques, assisted”; or “pregnancy.” This search was restricted to English-language publications with an abstract, using the following query: “artificial intelligence” [MeSH Terms] AND (“obstetrics” [MeSH Terms] OR “gynecology” [MeSH Terms] OR “reproductive techniques, assisted” [MeSH Terms] OR “pregnancy” [MeSH Terms]) AND 2000/1/1:2020/4/30 [Date Publication].

The results of the query were considered final on November 30, 2020, to cover all publications with potential delayed indexation in PubMed.

All retrieved citations were classified according to disciplines defined in Web of Science (WoS) for the Journal Citation Reports (JCR) and grouped in the following 9 discipline categories: OB/GYN core disciplines journals, other medical clinical disciplines journals, medical nonclinical disciplines journals, medical genetics/biology disciplines journals, medical imaging journals, medical informatics journals, computer science disciplines journals, engineering disciplines journals, and other science disciplines journals. The detailed disciplines and discipline categories are presented in Table S1 of Multimedia Appendix 2 for all journals.

We included all publications from journals or conference proceedings of the core OB/GYN WoS disciplines, namely Obstetrics & Gynecology, Surgery, Oncology, Developmental Biology, Reproductive Biology, Andrology, or Urology & Nephrology. The publications were excluded if no AI process was used in the study. Review, editorial, and commentary articles were also excluded.

The publication selection was independently performed by two researchers (FD, JB) by full-text review to assess the actual use of any AI process in the study. Discrepancies on AI process assessments between reviewers were resolved during meetings with KB and JMJ.

The data collection and the qualitative analysis of the citations comprised six different tasks: (1) classification of publications into 5 OB/GYN domains (obstetrics, gynecology, assisted reproductive medicine, early pregnancy, and fetal medicine), (2) description of the AI method used in the study (eg, ML, knowledge base), (3) description of the AI algorithm used in the study (eg, ANN, support vector machine, bioinformatics knowledge bases), (4) description of the type of data used in the AI process (eg, image data set, omics data set), (5) contribution of the AI process (eg, new algorithm, hypothesis generation, fully functional software), and (6) description of the validation of the AI process (eg, validation on one data set, validation on more than one data set, clinical validation). The statistical synthesis of this systematic review was performed by computing the proportion of publications by groups defined in the qualitative analysis.

The evolution over time of the scientific production related to AI in OB/GYN was assessed by a trend analysis of publications per year during the entire review period for OB/GYN core journals and other science journals. The respective contributions of all scientific disciplines in the retrieved citations were assessed by the analysis of their distribution across all WoS disciplines and the proportion of citations in each of the 9 science discipline categories.

The PubMed search retrieved 579 citations. The 161 publications from OB/GYN core disciplines journals were reviewed for eligibility assessment. A total of 66 publications met the selection criteria [14-79]. The flow chart of the publications reviewed is presented in Figure 2.

All OB/GYN domains were represented in these selected publications (N=66): obstetrics (n=27, 41%), gynecology (n=2, 3%), assisted reproductive medicine (n=22, 33%), early pregnancy (n=1, 2%), and fetal medicine (n=14, 21%). The detailed distribution of the publications in these domains is presented in Figure 3.

The two main AI methods in the 66 selected papers were represented by ML methods [14-47,66-70] (59%, n=39) and by knowledge base methods [50-65,71-79] (38%, n=25). In obstetrics and in fetal medicine, ML was more common than other AI methods in 70% (19/27) and 71% (10/14) of the publications, respectively. ML methods and knowledge bases were used in 45% (10/22) and 55% (12/22) of the publications in in the assisted reproductive medicine domain, respectively.

The ML methods comprised mainly ANNs (25/39, 64%) [14-25,27-30,33-35,41,43-45,47,68]. Diverse other ML approaches are used, ranging from classical ML tools such as support vector machine [26] and genetic algorithms [31] to more recent methods such as random forest [38,46] or gradient boosted trees [37,42,67]. Very recently, more evolved and combined neural networks are used in deep learning to process complex data for image segmentation (eg, [41,44]) or classification (eg, [39,43]). The knowledge base methods comprised bioinformatic processes involving mainly Gene Ontology (88%, 22/25) but also other omics knowledge bases, text-mining processes leveraging ontologies, and semantic reasoning processes based on domain ontologies.

The data sets used with all AI methods in the selected studies are detailed in Table 1. ML methods dealt primarily with ultrasound imaging (2D, 3D, video), numerical, and clinical data sets, whereas knowledge base methods dealt mostly with omics data sets.

The contribution of using AI methods were for algorithm development (53%), hypothesis generation (42%), or software development (3%).

When using knowledge base methods, the main AI contribution was to generate hypotheses in physiology or physiopathology (reproduction and implantation, preeclampsia, fetal growth, or breast cancer). When using ML methods, the AI contribution was to build prediction algorithms (implantation success, neonatal outcome, preterm delivery, fetal weight, aneuploidy risk, or postpartum complications). The detailed contributions for all AI methods are presented in Table 2.

Most ML methods were applied to one data set (87%, 34/39) and the use of two or more data sets was less common (13%, 5/39). No external clinical validation of ML methods was identified in the selected articles. Knowledge base methods were applied on one data set in all cases and validated in one clinical study.

We observed a significant rising trend in the scientific production over the last two decades, mainly outside the OB/GYN core journals (Figure 4). A total of 67 science disciplines covered this scientific production (579 PubMed indexed citations), 18% of which were in OB/GYN core disciplines journals. The distribution of citations in the other discipline categories is shown in Table 3. The science discipline was not defined in WoS/JCR for 6% of the citations.

In this review, we have demonstrated that AI contributions are emerging in OB/GYN journals and that a wide range of AI approaches (symbolic and nonsymbolic) are applied across all OB/GYN subdomains. ML is the most common nonsymbolic AI approach (59%) and articles are based mainly on ANNs (64%). Knowledge bases are the most common symbolic AI approach (38%) and are based on ontologies in most articles (88%).

However, most of the AI publications related to AI in OB/GYN (82%) remain out of the scope of the usual OB/GYN journals. Additionally, formally validated AI contributions reported to date suffer from an overall poor level of validation (one data set in most cases and no external validation in all cases) and actual AI contributions remain at the level of “proof of concept” or “proof of feasibility.”

The reported AI contribution to OB/GYN in the core discipline journals was 18% in comparison with 82% in journals of other disciplines. This can be explained by the early stage of research in AI or by the absence of clinical validation, meaning that the results are more relevant for the AI and computer science community. When novel algorithms are developed, computer science journals are naturally preferred [80-84]; for example, one of the first convolutional neural networks able to perform automated plane recognition during a fetal ultrasound scan was reported in a computer science journal [85]. In addition a clinically validated ML-based fetal biometric system was reported in a general medical imaging journal, not in a core OB/GYN discipline journal [86]. Another contribution based on logic and semantic reasoning for early pregnancy diagnosis was reported in a medical informatics journal [87]. These examples illustrate that core OB/GYN discipline journals await clinical value demonstration of AI-based research rather than reporting on novel systems. This pattern might also suggest that the time has come for the OB/GYN community to take over some valuable early-stage AI contributions within its core discipline journals.

Additionally, we have observed more advanced AI techniques and architectures applied to OB/GYN in computer science journals than in OB/GYN discipline journals. Moreover, the well-established and most robust ANN architectures (eg, U-net, ResNet) are no longer published in computer science journals and are largely published in OB/GYN discipline journals to present another application context [39,41,44]. As a result, a strong representation of experts in AI methods in editorial boards could improve editorial choices, which would help to fill in the delay of translation of advanced AI to the OB/GYN readership.

Interestingly, reported AI methods are applied in unconnected data silos in the field of OB/GYN (images, omics data, clinical data, other data modalities) and mixed AI methods in the field of OB/GYN are in early stages. Thus, approaches involving both ML and knowledge bases is a new direction that we expect to emerge. For example, the Smart Ultrasound in Obstetrics and Gynecology (SUOG) initiative (EIT-Health Innovation program) [88] combines knowledge bases for differential diagnosis and ML for image analysis to develop an AI-based ultrasound diagnosis assistant.

The low level of validation of AI processes in medicine has been previously reported [89]. We also observed significant heterogeneity in the description of AI processes in this review, with an overall limited level of description in most publications and with a poor level of clinical validation. This can be explained because, until recently, there were no AI-specific guidelines for medical publications. Indeed, most AI notions are new to the medical readership, medical editorial boards, and medical literature indexing. Some medical publications have proposed glossaries and definitions of basic AI notions, and the first reference guidelines for reporting medical studies involving AI were published in 2020 [90-95]. Although these initiatives should improve the reporting of AI-related publications, these guidelines only cover ML approaches. For example, the extension of SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials) guidelines for clinical trial protocols using interventions involving AI (SPIRIT-AI) [92] lists the items of interest for AI publications but does not cover knowledge representations, ontologies, semantic reasoning, nor knowledge bases. In addition, we found that 38% of the articles in this review leverage these AI approaches. Consequently, a further extension of these recommendations could advantageously provide guidelines for symbolic AI approaches.

Albeit not covered in the guidelines for AI-related research, some “routine” methods in statistics (eg, logistic regression, multivariate logistic regression) and in data visualization (eg, K-means clustering) are also considered as ML approaches [96]. In this review, we excluded studies based on these methods [97-99]. However, from a perspective of consistency, some statistical methods involving ML techniques could also be covered by AI-related research guidelines.

There are recurrent debates on ethical and legal considerations in AI methods in the news and social media; therefore, we were surprised that most publications do not elaborate on these aspects. The majority use nonexplainable approaches such as ANNs; while using such nonexplainable methods is acceptable, some limitations need disclosure, and their reproducibility needs proper assessment. The most straightforward assessment method of reproducibility relies on external validation, which remains critical prior to application of all methods, but even more so if nonexplainable. Human responsibility in using AI-based processes also depends on the level of autonomy of the process [100] and on recommendations to use such processes [101].

This is the first systematic review on AI contributions reported in OB/GYN core journals. This study was performed by a pluridisciplinary group of experts from both the OB/GYN and computer science communities [102]. We have limited our paper selection to citations in PubMed and used the science disciplines as defined by WoS/JCR, thus controlling potential bias in the definition of journal domains. Although our method is reproducible and complies with systematic review guidelines, it is by essence subject to bias in publication indexation. For example, articles with ML methods mentioned only in one paragraph (eg, [103]) are not covered in this study. In addition, for papers with a scope in decision support (eg, [104,105]), the indexation will not fall under the MeSH term “artificial intelligence” in PubMed but rather under the MeSH term “diagnosis, computer-assisted” that is a distinct notion. However, unlike systematic reviews of clinical therapies, this limitation is less of a problem as we were still able to ascertain general trends in this relatively novel field of study.

All reviewed papers on robotic surgery were indexed in PubMed with the MeSH term “robotics” and under the MeSH term “artificial intelligence.” Currently, in MeSH, “robotics” is a subcategory of “artificial intelligence.” As a result, all robotic surgery papers are considered to be AI papers, which is not always the case. A revision of MeSH terms and/or indexation policies could be a solution for disambiguation. Additionally, the use of appropriate AI-oriented keywords provided by authors at the time of submission could improve the characterization of AI-based research.

Until mid-2020, mostly preliminary work in AI applied in OB/GYN has been reported and published outside the usual OB/GYN journals. When published in OB/GYN journals, multiple data set validation and clinical validation of AI processes remain unmet prerequisites. Clarification in AI methods could be achieved by improving the MeSH indexing of publications in PubMed. Additionally, the reporting of AI applications should be improved by the new 2020 guidelines and recommendations in medical research involving AI. These are promising for future clinically relevant and methodologically valid clinical trials publications. However, these guidelines are covering only a part of AI approaches involved in the articles reviewed in this study, and updates need to be considered, especially to cover symbolic AI approaches.