We conducted a cross-sectional analysis using ClinicalTrials.gov, a trial registry and results database maintained by the US National Library of Medicine. We sourced data from the Clinical Trials Transformation Initiative Aggregate Analysis of ClinicalTrials.gov (CTTI AACT) [15], which allows open access to the complete set of trials registered in ClinicalTrials.gov, including additional fields that are not readily available in direct exports from ClinicalTrials.gov. The CTTI AACT data dictionary is publicly accessible [16]. A static version of the CTTI AACT database was downloaded for analysis on February 6, 2024, via PostgreSQL (pSQL). The pSQL codes used are provided in Multimedia Appendix 1.

We used text-based search to identify relevant AI/ML studies. Our search strategies were informed by systematic reviews published during 2019-2023 [9,17-20]. We used search terms related to AI/ML methodologies and specific model architectures as used in the literature. Terms are not necessarily mutually exclusive or hierarchical (eg, “multilayer perceptron” under “machine learning”). The search strategy is detailed in Table S1 in Multimedia Appendix 2, with corresponding SQL codes available in Multimedia Appendix 1.

First, SQL was used to exclude studies containing any of the following terms: protocol*, design*, review*, systematic review*, scoping review*, literature review*, meta analys*s (* denotes a wildcard, equivalent to % in SQL codes). Second, the lead author selected studies with fewer than 5 search-word hits to confirm their relevance, and irrelevant studies were excluded (eg, those involving “neural networks” of a biological nature). Third, 2 reviewers independently examined study descriptions and assessed eligibility. Discrepancies were resolved through discussion and by revisiting details on ClinicalTrials.gov. Where study descriptions were unclear or insufficient, we opted to include studies unless they explicitly met the exclusion criteria, prioritizing sensitivity over specificity.

The study flow diagram is provided in Figure S1 in Multimedia Appendix 2. The pSQL codes used are provided in Multimedia Appendix 1.

The ClinicalTrials.gov search yielded 3378 records. Of those, 272 studies were excluded: not AI/ML-related (n=265); systematic reviews or meta-analyses (n=2); and the term “artificial intelligence” merely being a part of the organization, product, or study name (n=5). After excluding 272 studies, 3106 studies were included for analysis (Figure S1 in Multimedia Appendix 2).

Tables 1 and 2 show the study characteristics. Overall, the number of AI/ML studies increased rapidly after 2017, and 62.8% (1951/3106) started in the 2021 to 2023 period. Of the 3016 studies, 842 studies (27.1%) were completed, 1694 (54.5%) were active (not yet recruiting; recruiting; enrolling by invitation; active, not recruiting), 95 (3.1%) were stopped, and 475 (15.3%) were unknown. There were more observational studies (61.6%) than interventional studies (38.4%); more parallel assignments (89.2%) than crossover, factorial, or single assignments; more open-labeled studies (63%) than ones that used any form of masking; more single-center (77.2%) than multicenter (22.8%) studies; and more studies with 2 arms (56.8%) than 1 (34.3%) or 3 (9%) arms. The study phase was mostly “not applicable” (93%). Studies involving adults aged 18-65 years (58.1%) were most common, followed by adults and seniors aged 18 and above (27.5%). While some studies accepted healthy volunteers (35.2%), the majority did not (64.8%). Only 235 (7.6%) were FDA regulated.

Among observational studies, a prospective design was more common (58.9%; 1126/1913) than retrospective (22.5%; 430/1913) or cross-sectional (11%; 210/1913) designs. Among trials, 56.2% (670/1193) were randomized, 10.9% (130/1193) were nonrandomized, and 32.8% (393/1193) were “not applicable” (Table 1). Figure 1 shows the numbers of all studies, including randomized controlled trials (RCTs) and prospective, retrospective, and cross-sectional studies, over time. From 2017 to 2023, the proportion of RCTs increased by 42.1%: from 18.52% (15/81) to 26.32% (179/680). The proportion of prospective studies increased by 22.6%: from 29.63% (24/81) to 36.32% (247/680). In contrast, the proportion of retrospective studies decreased by 11.9%, from 12.35% (10/81) to 10.88% (74/680), and cross-sectional studies declined by 28.5%, from 8.64% (7/81) to 6.18% (42/680).

Data on primary purpose were only available for interventional trials. Among the 1190 trials, the most common purposes were diagnostic (28.2%; n=335), treatment (24.4%; n=290), “other” (12.5%; n=149), and prevention (8.2%; n=97). However, the top 2 rankings were reversed when restricted to RCTs only: treatment (31.4%; n=210), diagnostic (18.6%; n=124), “other” (11.1%; n=74), and prevention (10.9%; n=73). Over time, studies focused on diagnosis and treatment increased faster than others (Figure 2).

The top 10 accounted for 81.1% (2631/3245) of all clinical specialties (Table 2). The 3245 specialties included neoplasms (12.9%; n=420); nervous system diseases (12.2%; n=395); cardiovascular diseases (11%; n=356); pathological conditions (10%; n=325); respiratory tract diseases (8.5%; n=275); digestive system diseases (7.8%; n=253); mental disorders (6.7%; n=219); endocrine, nutritional, or metabolic diseases (5%; n=161); female urogenital diseases and pregnancy complications (4.3%; n=138); and skin and connective tissue diseases (2.7%; n=89). Most of these started to increase during 2018-2020 (Figure S3 in Multimedia Appendix 2). See Table S3 in Multimedia Appendix 2 for the entire list of specialties over time.

We compared lead sponsor sectors classified in ClinicalTrials.gov versus our reclassification based on sponsor names (n=3106). In the original classification, the most common sector was “OTHER” (83.4%; n=2590), followed by “INDUSTRY” (12.4%; n=384) and “OTHER_GOV” (3.2%; n=98). However, our reclassification revealed the strong presence of hospitals/clinics (44.2%; n=1373) and academia (28%; n=869), followed by industry (13.1%; n=407) (Table 2; Figure S8 in Multimedia Appendix 2). Studies sponsored by hospitals/clinics, academia, and industry have increased since 2017 (Figure 3).

The enrollment data were skewed to the right: maximum 13,977,257; mean 16,962 (SD 288,155); and median 255 (IQR 80-1000). The most common size category overall was 101-1000 (44.8%). Among trials, 1-100 (42.7%; 497/1163) was most common, followed by 101-1000 (41.2%; 479/1163), whereas 101-1000 (46.7%; 893/1913) was most represented among observational studies (Table 1). This pattern did not materially change when stratified by sponsor sectors (Table S4 in Multimedia Appendix 2). Traditionally, the largest studies tend to be industry-sponsored [23], but this was not the case in our data. Among observational studies, the hospital/clinic sector, followed by academia, represented the highest number of studies in all size categories, including the largest (>30,000). Among trials, the hospital/clinic sector represented the most up to the 10,000 category. While interventional trials in the 1-100 and 101-1000 categories and observational studies in the 101-1000 category have increased the most over time (Figure S5 in Multimedia Appendix 2), 16.1% (187/1163) of trials and 29% (551/1898) of observational studies were large, with enrollments above 1000.

We also compared the overall enrollment distribution (1-100, 101-1000, and >1000) with that of prior reports that used the ClinicalTrials.gov data (Figure S6 in Multimedia Appendix 2). First, our data distribution was similar to that of previous research on AI studies up to March 2022 [1]. Second, when compared to 2 previous reports on non–disease-specific trials (non-AI/ML) during 2000-2019 and 2007-2010, the proportion of the largest category (>1000) was higher in our AI/ML trials (16.1%; 187/1163) than in the non-AI/ML trials: (3.8%; 5174/135,144 in 2000-2019 [23] and 3.9%; 1103/28,467 in 2007-2010 [24]). When restricted to RCTs alone, the proportion of the >1000 category was even higher in our AI/ML trials (19.8%; 129/653). Conversely, the proportion of the smallest category (1-100) was lower in our trials (42.7%; 497/1163) than in the non-AI/ML trials (63.2% [23] and 62.3% [24]).

The majority (92.2%; 2863/3104) was not sex-specific, followed by women-targeted (6.2%; 192/3104) and men-targeted (1.6%; 49/3104) studies (Table 2). Over time, AI/ML studies for women have increased since 2018 (Figure S7 in Multimedia Appendix 2).

The most common regions were Europe (34.5%; 973/2821), Asia/Pacific (29.8%; 841/2821), and North America (25.9%; 731/2821; Table 2), followed by the Middle East (3%; 84/2821). The most common countries were the United States (21.8%; 677/3106), China (17.7%; 550/3106), France (6.6%; 205/3106), the United Kingdom (6.2%; 193/3106), Italy (5.2%; 162/3106), and Taiwan (3.6%; 111/3106). See Table S2 in Multimedia Appendix 2 for a full list of countries, with the World Bank’s GNI-based distribution.

In Figure 4, 71 countries are color-divided into 38 high-income countries and 33 non–high-income countries. Most AI/ML studies were conducted in high-income countries (75.3%; 2340/3106), followed by upper-middle-income countries, including China (21.7%; 675/3106), lower-middle-income countries (2.8%; 88/3106), and low-income countries (0.1%; 3/3106) (Table 2). Studies in non–high-income countries have started to increase since 2020, particularly in Turkey (41 vs 50), Egypt (21 vs 24), India (15 vs 23), Brazil (15 vs 16), Russia (11 vs 14), and Mexico (8 vs 11), where the numbers in parentheses represent the totals for 2020-2023 compared to the totals for the entire period of 2010-2023.

As shown in Figure 5, studies in Europe, Asia/Pacific, and North America increased since 2018, and so did studies in high-income countries, with a modest increase in upper-middle-income countries, particularly China. The entries of lower-middle-income countries (in 2018) and lower-income countries (in 2021) lagged.

Overall, only 5.6% (47/842) of completed studies had results available on ClinicalTrials.gov. Among studies with posted results (n=47), the median time from “primary completion date” to “results first posted date” was 505 days (IQR 399-700; minimum 93, maximum 1200). During 2018-2022, as the rapid increase in AI/ML research became evident, completed studies increased from 55 to 227, but the reporting rate stagnated, remaining between 5.3% and 7.3% (Table 3). Consequently, the number of completed studies without posted results increased (Figure 6).

The most frequently used term in study descriptions was “artificial intelligence” (37%), followed by “machine learning” (31.4%), “deep learning” (12.3%), “convolutional neural network” (2.6%), and “random forest” (2.1%; Figure S2 in Multimedia Appendix 2). The use of “artificial intelligence,” “machine learning,” and “deep learning” has increased since 2017, especially the former two (Figure S4 in Multimedia Appendix 2).

Previous research on AI studies registered on ClinicalTrials.gov as of March 2022 concluded that the proportion of trials with prospective (59.92%) and randomized (52.7%) designs was insufficient [1]. We also observed similar proportions (prospective: 58.9%; randomized: 56.2%) as of December 2023. However, from 2017 to 2023, the proportions of RCT and prospective designs increased by 42.1% and 22.6%, respectively, while retrospective and cross-sectional studies decreased by 11.9% and 28.5%, respectively. This is deemed an improvement compared to the period when retrospective assessments were dominant [11].

Although the choice of RCTs depends on the stage of development, the “phase” data, more aligned with drug development, was mostly “not applicable” (93%) in our data (Table 1). Referring to a stage-specific reporting guideline for the early and live clinical evaluation of decision-support systems based on AI, which compares development pathways among drugs, AI, and surgical innovation [25], the recent increase in prospective and randomized designs may be a reflection of enough studies progressing from the in silico evaluation stage to the early live clinical evaluation stage (trial phase 1-2 in drugs) and the comparative prospective evaluation stage (trial phase 3 in drugs).

AI models, especially deep learning models, often require large datasets to achieve robust performance [26], with sample sizes above 1000 being common. This was empirically reflected in our data overall. When compared to non-AI/ML trials registered on ClinicalTrials.gov during 2000-2019 [23,24], the proportion of smaller trials (1-100) was lower, and the proportion of larger trials (>1000) was higher in our data (Figure S6 in Multimedia Appendix 2). Furthermore, 16.1% (187/1163) of trials had a sample size of >1000, 72.2% (135/187) of which were not completed yet. Similarly, 19.8% of RCTs (129/653, excluding withdrawn) were in the >1000 category, but 68.2% (88/129) were still ongoing. Thus, the majority of large AI/ML trials in our data are still in the pipeline and yet to be captured in future systematic reviews.

Our reclassification of the “other” groups revealed the strong presence of hospital/clinic or academic sponsorship over industry (Figure 3). The hospital/clinic or academic sectors might be more likely than the industry to engage in AI/ML research even if it lacks commercialization potential. The higher representation of hospitals/clinics and academia might reflect a high volume of nonproprietary AI/ML.

We examined two areas. First, the underrepresentation of women in research [27,28] is well documented. In this study, we found a gradual increase in women-targeted studies over time. Second, 75.3% (2340/3106) of AI/ML studies were conducted in high-income countries, and only 2.8% (88/3106) and 0.1% (3/3106) were conducted in lower-middle-income and low-income countries, respectively. If AI/ML models poorly generalize to people other than those whose data were used to train the algorithms, it may exacerbate existing disparities. While external validation and model recalibration may help mitigate health care inequities due to data disparity [14], inappropriate external validation (and the lack thereof) in AI/ML research was already highlighted [9,29,30]. Validation using unseen data is particularly important in models built with single-center data [31], and most of our sample (77.2%; 2177/2821) indeed comprised single-center studies. However, external validation was mentioned in only 47 of 3106 studies (1.5%). Of these 47, only 9 (19.1%) were conducted in non–high-income countries: China (n=6), Argentina (n=1), Ecuador (n=1), and Russia (n=1), as part of a 10-country study (all were high-income countries, except Russia).

While the number of newly initiated studies increased, the number of completed studies without posted results also increased. The limited reporting of even basic results on ClinicalTrials.gov contrasts with the rapid advancements in the field and the registry’s stated purpose of registration and result submission, which includes mitigating publication bias and outcome reporting bias [32].

Outside the AI/ML field, high rates of results dissemination on ClinicalTrials.gov have been reported, including for cancer trials and small studies [33]. With AI/ML research results more accessible on ClinicalTrials.gov, the registry can play a crucial role in reducing publication bias, making it a valuable resource for future systematic reviews or meta-analyses. Although posting results on ClinicalTrials.gov means disseminating data without independent scientific review, summary results are reported in structured tables, and the results information is devoid of conclusions or “spin,” as the system requires objective data rather than subjective narratives [34]. This allows for broad scrutiny by the scientific community, the public, or competitors. Although results posting on ClinicalTrials.gov is mandated only for certain types of trials, all registered studies are eligible for results submission, and the responsibility of providing knowledge given the use of human data should equally apply to both trials and observational studies.

Our study limitations are as follows. The first is representativeness. Sponsors of FDA-regulated studies and trials with sites in the United States are required to submit information to ClinicalTrials.gov [35]. In our sample, only 7.6% (235/3106) of studies were FDA regulated, while 78.2% (2429/3106) did not involve a US site. When registration is voluntary, it is hard to determine what the data are truly representative of. Interventional trials are more likely to be registered in a public registry due to this being a requirement of the International Committee of Medical Journal Editors as a condition of consideration for publication [36]. However, as this requirement started in 2005 and precedes our data cutoffs (2010-2023), it is unlikely to have influenced our findings (RCTs were infrequent until 2017). Moreover, registered studies would be more likely to share results than unregistered ones, but the reporting rate in our sample was unexpectedly low. Nevertheless, by evaluating trends over time under the same representativeness constraints, we made the best use of the most recent data available.

Second, as we prioritized sensitivity over specificity, our sample may have included studies in which AI/ML played a minor role. Identifying such studies would have required manual free-text data mining with prespecified criteria, which was impractical during testing due to substantial variations in reporting quality, length, and clarity. Poor reporting in AI/ML research has already been highlighted [8,9], and AI-specific reporting guidelines and documentation standards now exist to increase awareness of key dimensions that should be described in study protocols (the SPIRIT-AI Extension, based on SPIRIT [Standard Protocol Items: Recommendations for Interventional Trials]) [37], prediction models (the TRIPOD+AI Statement, based on TRIPOD [Transparent Reporting of a Multivariate Prediction Model for Individual Prognosis or Diagnosis]) [38], or the minimum information about clinical AI modeling (the MI-CLAIM checklist) [39]. The same principle could also apply to study descriptions on ClinicalTrials.gov, and more standardized content within the platform would benefit its audience.

Much of the rapid growth in AI/ML studies comes from high-income countries in high-resource settings, albeit with a modest increase in upper-middle-income countries (mostly China). Lower-middle or low-income countries remain poorly represented. The increase in randomized or prospective designs, along with large studies (n>1000), which represent a quarter of all studies (mostly ongoing), may indicate that enough studies are progressing from the in silico evaluation stage toward a prospective comparative evaluation stage. However, the ongoing limited availability of basic results posted on ClinicalTrials.gov contrasts with the rapid advancements in this field and the public registry’s role in reducing publication and outcome reporting biases.