Principal Findings
Although the span of 28 months may seem a short period to assess changes in researchers’ expectations on a given topic, the case of AI in diagnostic medicine is peculiar. This is an area where transformations are quick, and innovations are the norm [51]. One indicator of this rapid transformation is the surge in publications on this topic. For instance, a quick search on WoS between January 1, 2015, and September 20, 2020, showed >6000 articles on AI in diagnostic medicine. However, between September 21, 2020, and December 30, 2022, this number rose to >12,000, an increase of 113.5% (the search strategies used to identify these publications are the same as those described in Textbox 2).
AI is expected to improve the quality of health care for patients, mainly by addressing patient safety and reducing medical errors [18,70,71]. A study compared deep learning algorithms and physicians in diagnosing skin conditions from photos and dermoscopic images to evaluate how AI can support or automate health care decision-making. The results showed that the algorithms performed better than the average physician [18]. Image reconstruction using deep learning algorithms is another example of how AI can benefit patients. This technique can potentially reduce radiation exposure and the time needed for image acquisition and segmentation in preparation for radiotherapy, which may reduce side effects and material costs [72].
Survey respondents in both waves agreed that AI could positively impact quality over a 10-year time horizon by (1) aiding the prediction of disease progression, (2) increasing diagnostic reliability, and (3) improving treatment compliance. Furthermore, they believed that AI might improve quality in diagnostic medicine in <10 years by producing more reliable, compliable, and predictable outcomes. The answers to the questionnaire (Figure 2) show that AI-based quality improvements are associated with patient outcomes, but the technology may also benefit health care practitioners by reducing human error and fatigue [18,73-75].
Information accuracy is fundamental for achieving better-quality health care outcomes. However, technology and information accuracy may become a barrier to adopting AI [20,76]. For example, most of the researchers (W1: 255/535, 47.7% and W2: 399/789, 50.8%) who selected the difficulty of incorporating AI into clinical practice as the main barrier expressed their concern that the use of AI might be hampered by its misalignment with the specific context of clinical practice. This issue is associated with the generalizability and reproducibility of AI algorithms [20], which are often trained on clean datasets with no poor-quality information but are applied in a real clinical setting where data may be incomplete or erroneous. When comparing responses in both waves, this topic was more relevant for researchers in W2 than those in W1, which may indicate a growing concern about this misalignment. Furthermore, data accuracy issues may influence the quality of AI algorithm outputs by introducing biases [20,71,77,78] due to discrepancies in patient population (eg, individuals from low socioeconomic backgrounds and underrepresented minority groups) and organizational settings (eg, hospitals and primary care) [79,80].
Cost reduction is another potential benefit of AI in diagnostic medicine, as it relates to the optimization of resource use [72], processes, and services [71], as well as financial management [18,70]. As a result, the same therapy could be offered at a lower cost, or more services could be provided at the same cost. The use of AI to process large image datasets, for example, could offer workflow and productivity gains and reduce the workforce in many back-office activities, such as billing, clinical appointments, and staffing [72]. AI could have a variable impact on patient-, organizational-, and system-level tasks, potentially leading insurers to revise their reimbursement policy to reduce health care costs and improve quality [18]. Moreover, AI may make costly and time-consuming screening programs more affordable in resource-limited countries [23,81].
A total of 3 questions assessed the respondents’ beliefs about the changes in costs driven by AI. Most respondents (W1: 982/1360, 69.3% and W2: 1514/2243, 67.5%) in both waves anticipated that screening costs and physician workloads would be reduced in <10 years and patients’ hospitalization time would be shortened. These changes could potentially lower the cost of health care services and improve their quality, as physicians could perform more tasks concurrently and focus on more skilled work by delegating repetitive tasks to AI algorithms [18,72]. These benefits could be offset by the increased costs and risks for patients and the health system due to unnecessary testing and treatments prompted by AI [20]. Furthermore, there are concerns about the initial investments required for state-of-the-art AI products, the potential for long-term cost savings, and the need for a balanced evaluation of costs and benefits when considering AI integration [82,83].
Conversely, when appropriately implemented, AI can enhance diagnostic accuracy and reduce the need for redundant or follow-up tests. This efficiency is already leading to cost savings and a reduced burden on the health care system, and there is potential for further reduction of redundant and follow-up tests [84]. Therefore, AI value analyses should consider the technical aspects, short-term returns, and the overall value of automating processes [20].
The survey results in both waves have highlighted the anticipated transformation of AI in diagnostic instruments within a decade. In particular, image analysis and interpretation applications, such as x-ray, skin malignancy, and histopathologic diagnosis, have received optimistic perspectives. This may relate to advances in deep learning techniques for image analysis, such as convolutional neural networks and transformer architectures [85], which have significantly improved tasks such as image classification and object detection [3,29,46]. Deep learning is already the most used AI technique in health-related applications [71], and combining computing power, large datasets, and convolutional neural networks is also responsible for the subsequent revolution in medical imaging [18]. Since deep learning was introduced in 2012 for image recognition, it surpassed the human accuracy rate in specific, large data-labeled datasets after 5 years [72].
Furthermore, technology’s maturity may relate to perceptions of the use of AI in medical imaging. Digital medical images have been in use since the 1960s, and mature technologies have been built around them. Most radiology departments maintain picture archiving and communication systems containing historical images [18]. Cardiology applications highlighted by the respondents in both waves (eg, heart rhythm interpretation and disease diagnostics) have been in use for many decades, but recent developments are seen as paradigm shifts in clinical practice [86,87]. Because AI algorithms rely on the design of distinctive features to learn from data, more mature technologies are expected to provide a better set of such features, as is the case for these diagnostic instruments [79,88].
Conversely, the least optimistic results in both waves were related to applications that use other data formats, such as electronic health record texts (eg, identification of sepsis symptoms and prediction of clinical outcomes) or 1D signal data (eg, intrapartum monitoring and remote monitoring of gait). Multiple reasons may account for that, for example, data heterogeneity in electronic health records, including mixed data types, such as free text clinical notes, radiological reports, medication dosages, and medical codes [70,89]. Moreover, privacy policies and data interoperability impact information transfer across institutions that construct large datasets for deep learning representations of electronic health records [89]. For gait and intrapartum monitoring, some issues, such as the lack of large datasets and the use of clinical context as a complement for decision-making, also affect its performance [90].
Another factor that may help explain these results is regulation. In 2018, the Food and Drug Administration published a fast-track approval plan for AI medical algorithms for different applications [72]. Since then, the number of algorithms approved has soared, and there are now almost 400 algorithms associated with radiology and 58 associated with cardiology, many of which are dedicated to x-ray diagnostics and heart rhythm interpretation, respectively [10]. Thus, technological innovation and maturity combined with adequate regulation seem correlated to the respondents’ perceptions of AI’s impact on diagnostics in the coming years.
According to the respondents’ answers in both waves, the difficulty of incorporating AI into clinical practice, despite the reported benefits in terms of health outcomes, may hamper these innovations. These issues may be related to resistance to change in adopting AI in diagnostic medicine [91-93]. Another relevant challenge to AI adoption was ethical and regulatory issues. Accountability and legal responsibility for AI clinical decisions were marked as the most relevant issues in this topic in both waves. Accountability issues arise when AI algorithms make autonomous decisions about diagnoses and treatments beyond their role as support tools [20,23]. As respondents in W2 (941/2164, 43.5%) rated ethical and regulatory issues as more relevant than those in W1 (487/1296, 37.6%; in fact, as the most relevant), this seems to be a growing concern. As more AI applications become available to the public and cause social changes shortly, it is unsurprising that this issue increased in relevance from one wave to another. Large language models, such as ChatGPT, are examples of how AI has become more accessible to people and how this may raise more concerns over time [94,95]. For example, the United Kingdom’s Medicines and Healthcare Products Regulatory Agency points out some issues with incorporating large language models in medical devices. While it anticipates their use in health care soon, it emphasizes the need to uphold safety, effectiveness, and ethical standards [96].
In short, there are still doubts about who should be held responsible if a patient experiences an adverse event due to AI-based technology [34]. Insurers and regulators will have to be able to distinguish algorithmic errors from those resulting from misuse by the clinician, the organization, or even the patient. This issue is exacerbated by the black box nature of AI systems [20]. The increased use of AI in medicine will probably lead to legal challenges regarding medical negligence attributed to complex decision support systems [18]. Radiologist associations globally have been evaluating the ethical integration of AI in medical imaging. The Canadian Association of Radiologists underscores the potential of AI in medical imaging, emphasizing the importance of addressing its ethical and legal challenges [97]. Furthermore, the Royal Australian and New Zealand College of Radiologists has formulated ethical guidelines and deployment standards in response to AI’s rapid growth in radiology [98]. Simultaneously, a Joint European and North American Multisociety Statement, representing several radiological entities, emphasizes AI’s potential to enhance radiology efficiency and introduce systemic errors, advocating for ethical AI use that prioritizes patient well-being and transparent practices [99].
According to the respondents in both waves, other relevant issues are accessing and sharing large amounts of patient data. The use of big data raises concerns that may hamper the potential of AI in diagnostic medicine. These relate to data protection and confidentiality and the need to access the large volumes of data needed to train AI algorithms. Regarding data protection and confidentiality, key concerns include the data’s origin, obtaining patient consent, authorization for reuse, data ownership and responsibility, who can access and reuse the data, and the conditions under which the data can be used [20]. For example, a recent report from the European Union Agency for Cybersecurity on the pivotal role of AI in medical imaging diagnosis shows the importance of cybersecurity and privacy controls in properly addressing these many concerns [100]. Other relevant issues raised in the literature are the risk of the inappropriate use of databases and imprecise consent terms [34]. Technology-related obstacles include limited deidentification techniques and the need to integrate and standardize large amounts of data [34].
A shortage of well-annotated datasets for training AI algorithms is a key obstacle to the large-scale introduction of these systems. Furthermore, the absence of clear, standardized regulations could lead to the illicit collection of data from unknown sources [23]. These concerns relate to the very nature of machine learning technologies, as they need large amounts of training materials to be taught. On the other hand, as clinical data are gathered from multiple and diverse sources, it becomes easier to trace them to patients, threatening their privacy [49]. As ubiquitous data collection becomes commonplace, consensus must be reached for a consent framework to guide health-related data sharing [18]. In addition, there is growing concern about the lack of clinician involvement in the development of AI applications. A recent systematic review showed that developers typically consulted clinicians late in the design process, and many applications lacked validation against clinical expertise or detailed algorithm descriptions [101].
A particularly sensitive problem is iatrogenic risks related to a lack of transparency in the algorithm training processes. Before an AI algorithm can be unleashed in clinical practice, it has to be debugged, audited, simulated, and validated, along with prospective scrutiny [72]. Respondents identified these problems as major issues in both waves, with higher percentages in W2 (170/789, 21.5%). Besides the need for certification of AI systems, professionals, and teams, the rapid evolution of machine learning–based models presents an enormous challenge to regulation as more data are collected and used in algorithm development. A critical problem is how these updates should be evaluated and audited [18]. Quality control instruments for AI algorithms are needed to prevent abuse by AI system developers, as clinical decision support systems could be programmed to favor certain drugs, tests, or devices without users being aware of this manipulation. Transparency will be difficult to achieve if companies make their algorithms purposefully opaque for proprietary or financial reasons [34].
Another issue, which is quite present in the reviewed literature but received relatively little attention from the respondents (W1: 32/489, 6.5% and W2: 60/940, 6.4%) who answered this question, relates to embedded bias present in AI algorithms due to a lack of inclusion of minority individuals in datasets [72]. Such biases, resulting from underrepresenting minority groups and those considered vulnerable in the datasets used to develop AI systems, could reinforce discriminatory practices based on race, sex, or other features [34]. Machine learning datasets must be large, but the often-used clinical trial research databases are derived mainly from majority populations. The resulting algorithms may be more likely to fail when applied to underserved and possibly underrepresented patient groups [49]. Algorithms trained on health care datasets, which inherently mirror biases in health care spending, have been observed to exacerbate racial disparities in access to care within the United States and can affect medical diagnosis [102]. This problem will require careful attention from regulatory agencies [103].
Limitations
This study relies on previous studies that have explored future scenarios of science and technology by surveying researchers [39,54-58,60]. However, it also inherits its limitations. The first one is the lack of diversity among respondents, who were identified and selected based on their publications in scientific journals. Therefore, most respondents (W1: 1007/1251, 80.5% and W2: 1607/2095, 76.7%) are researchers and professors affiliated with universities and research organizations. While their insights offer valuable academic and technological perspectives on AI use in diagnostic medicine, the relatively lower representation of clinicians might not capture the complete range of insights from the frontline of clinical practice. Considering that our respondents were sourced from scientific publications, a domain where professors and researchers are more commonly the authors, this was somewhat expected given our methodology.
Another limitation is the potential optimism bias of respondents. As researchers involved with AI and diagnostic medicine, they may have more positive expectations about the future of this technology than other groups (eg, patients, managers, businesspeople, and politicians). Nevertheless, they are among the most qualified to inform about future possibilities of AI use in diagnostic medicine as they are helping to advance the scientific and technological knowledge in the field. The self-attribution of knowledge level by the respondents in the questionnaire is another limitation. We cannot verify the accuracy of the respondents’ self-assessments or assign knowledge levels to them. Therefore, the self-assigned knowledge level reflects how the respondents perceive their own knowledge in the area. However, this study only includes participants who are authors of peer-reviewed scientific articles related to AI and diagnostic medicine indexed in WoS, which minimizes the risk of incorporating opinions from people with no knowledge of the topic.
A potential limitation of our study is the varied interpretations of the term “artificial intelligence” among respondents. AI is a broad and multifaceted concept, encompassing everything from machine learning and neural networks to natural language processing and specialized rule-based systems [1,2]. The use of thesaurus terms to identify relevant respondents and survey questions may have introduced biases based on how individuals conceptualize AI. This variation in understanding could influence responses, particularly regarding the anticipated impact and integration of AI into diagnostic medicine. Hence, future studies may benefit from providing a clearer, more uniform definition of AI to respondents, ensuring a more consistent interpretation across the survey population.
We included radiology-specific terms in the search for respondents because this field is one of the most likely to be affected by the advances of AI in diagnostics [104-106]. We acknowledge that there is a potential selection bias in extending the search to radiology experts. However, it is important to highlight that the query also included general terms, such as diagnostics, medicine, and medical, which mitigates this potential bias by engaging a more diverse range of diagnostic experts.
Furthermore, there is the possibility of regional demographic bias. Collecting respondents’ emails from scientific publications may introduce a bias toward specific demographic groups, possibly leading to an underrepresentation of experts from certain regions. Our methodology for identifying experts does not allow us to have demographic information on nonrespondents. The lack of this demographic information prevents an extensive analysis of nonresponse bias and demographic balance. However, we have included a regional breakdown of the responses in Multimedia Appendix 3 to address this concern and provide additional information. While slight percentage variations exist in the responses from each region, the overarching response patterns remain consistent across them.