Artificial intelligence (AI) is widely considered to be the new technical advancement capable of a large-scale modernization of health care. Current AI applications are advanced in their analytical power and precision to optimize health care logistics and support health care professionals in clinical decision-making [1]. AI applications can therefore play a vital role in improving the speed and accuracy of triage, diagnosis, and the quality of health care provided. Driven by AI’s funding opportunities to accelerate the use of AI in health care, manufacturers have developed a plethora of health care-related AI applications, yet the implementation and embedding of AI applications into health care currently face challenges that limit these applications’ success in clinical practice [2,3]. For example, van de Sande and colleagues [4] showed that more than 90% of AI prediction models developed for use in the intensive care unit remain within the prototyping environment, and very few make it to clinical practice. Key implementation challenges include issues around data sharing and privacy, transparency of algorithms, interoperability across multiple platforms, and concerns for patient safety [2].

Before a nationwide partnership of UK researchers on AI implementation in health care, we conducted a series of patient and public involvement and engagement (PPIE) consultations to understand public perceptions of AI in clinical practice at the earliest possible opportunity.

A primary challenge for AI applications is that their use of patient data will change existing relationships among patients, health care staff, clinicians, and health care systems [5]. Consequently, it is essential that patients and the wider public are involved in the design, development, implementation, and embedding of AI applications in health care. “PPIE” in the United Kingdom transforms research into a collaborative process, addressing and supporting the public’s needs [6,7], much like “patient-centered research” in the United States [8]. PPIE uses various types of qualitative data collection and project management methods to support social interaction–based collaboration between researchers and members of the public [6-10]. It allows patients and members of the public who will likely be affected by research outcomes to be involved in the research process, shaping research questions, collecting data, disseminating results, etc [6-10]. Building on the National Institute of Health and Care Research’s (NIHR) guidelines, we conducted a series of PPIE workshops or “consultations” with “public collaborators,” members of the public who discuss and provide feedback on research with researchers [6,9,10].

PPIE can prevent inaccurate information or misconceptions about AI applications from spreading while also yielding AI applications that reflect the public’s understanding, needs, and trust [11]. AI developers and purchasers who fail to establish PPIE in the development, implementation, and embedding of AI applications risk harming patients, losing public funds, damaging reputations, and having limited impact [12].

Presently, PPIE in AI research has focused on the public’s willingness to share medical data to enable “training” of AI applications [5]. Correspondingly, research literature reflects limited insights into public perceptions of the challenges and opportunities of AI in health care and how members of the public can and desire to be involved in the development, implementation, and embedding of AI [5]. A recent scoping review on PPIE and AI-assisted mental health care echoes this, arguing that additional research is required to understand which PPIE methods are most useful across differing public lived experiences and how to best apply them throughout AI development processes, especially given ethical issues surrounding AI and patient data and clinician and patient support for AI [13]. Interestingly, gray literature differs from this and instead focuses on the public’s broad and specific apprehensions around AI and research, some of which include algorithm bias, health inequalities, and damaging clinician-patient relationships [14-16].

We conducted this consultation to begin a process of understanding and supporting connections between patients, the public, and academic communities, given that patients and the public are becoming more aware of AI applications’ roles and that researchers across Applied Research Collaborations (ARCs) are increasingly interested in AI in implementation [12].

This consultation aimed to (1) understand patients’ and the public’s perceived benefits and challenges for AI and (2) clarify how to best conduct PPIE in projects on translating AI into clinical practice given public perceptions of AI.

This PPIE consultation was conducted in the United Kingdom, where the NIHR funds and supports applied health research and research on the implementation of health and care evidence into clinical practice [17]. The NIHR achieves this through 15 regional interdisciplinary research entities, ARCs, covering all of England [18]. ARCs include institutional affiliations such as health care providers, universities, charities, and local government authorities and aim to improve patient and public outcomes and the quality, delivery, and efficiency of health care services.

ARC North Thames (ARC NT) circulated an invitation for participation in the PPIE consultation through email to the 15 ARCs’ PPIE teams, requesting to share with their PPIE and public advisor networks. As each ARC handles PPIE differently and possesses a unique public advisor network, recruitment varied across the nation. Public collaborators were invited to respond to the invitation with their interest and answers to two questions: (1) Why would you like to join the AI interest group? (2) What skills and experience you would bring to the group?

Once the organizers of the PPIE consultation (WL and ARC NT core team staff) received no new notifications of interest (roughly 3 weeks’ time), they reviewed all responses (n=26) and assembled a convenience sample based on public collaborators’ stated interest. A total of 3 respondents did not give additional details, nor did they answer the questions included. The organizers followed up with them but received no responses. All individuals who wrote with interest and provided responses to the questions (n=23) were invited to participate, of whom 17 participated. The 6 respondents who abstained gave no reason for not participating. Per NIHR guidelines, participants were remunerated at £25/hour (US $31) and another £5/meeting (US $6) to support the cost of internet [19]. Overall, 4 of the 17 participants were known to WL before the PPIE consultation through ARC NT PPIE activities. WL is the PPIE lead at ARC NT and regularly conducts similar PPIE consultations and activities. Further communications were conducted through email.

We used a combination of public involvement and focus group (FG) methodologies to reach public collaborators’ understandings, experiences, and perceptions of AI applications in health care [20]. We designed a semistructured FG topic guide incorporating deductive and open-ended inductive questions to allow ideas to emerge from the discussion while providing flexibility to probe topics and clarify understandings (Multimedia Appendix 1) [21]. A total of 17 public collaborators were distributed across 3 consultations to support a variety of experiences and perspectives in each group. This allowed us to explore public collaborators’ thoughts with differing questions and contexts, allowing for the illumination of different “…concepts, meanings, and explanations” [22].

Groups were cofacilitated by WL and CS with 5-7 public collaborators each. One more note-taking facilitator from the ARC NT staff was present for 2 sessions. All sessions were held through Zoom (Zoom Video Communications) to logistically support public collaborators’ participation from across England. At the start of each session, public collaborators verbally consented to contribute to the session’s recording. We ensured all had the greatest opportunity to participate by encouraging chat communications, reading chats aloud, supporting participants to deactivate video as needed, and using breakout groups to minimize pressure to speak.

Consultations were structured as follows:

We treated breakout and collective group discussions as FGs to take advantage of the social context of discussing ideas in a group format. This allows public collaborators to “reflect and refine” their ideas, enriching their insights into their perspectives [23]. Breakout groups were held before CS’s specific project presentation to hear more “raw” perceptions of AI, unaffected by official explanations, and later enriched in the group discussion following the presentation.

Consultations were recorded and closed captioned with Zoom’s features, then used to create raw transcripts that WL edited, cleaned, and pseudonymized. All authors reviewed transcripts, discussed discrepancies, and WL made clarifying edits.

For the thematic analysis conducted in the study, data saturation was deemed of less importance than information power, requiring an information-dense sample rather than a large sample [24]. Analytic “codes” illuminated “themes” or descriptive narratives that summarize key discussions across all FGs [25].

All authors participated in an iterative qualitative coding process. SS and JH, 2 public collaborators in the FGs, also coded as public coauthors and coanalysts to assure the integrity and validity of the analysis. The analysis was done in 3 iterations, as shown in Textbox 1.

We use the Consolidated Criteria for Reporting Qualitative Research (COREQ) [27] to elaborate on our methodology (Multimedia Appendix 2).

Research ethics approval for PPIE consultations is not required, per the NIHR [10]. However, as stated above, collaborators consented to FG participation.

This group of public collaborators participating in the consultation is not intended to be representative of the public in general but rather represents a subset interested in collaborating on research. They offer layers of perspective based on their experiences, some of which included those of patients, caregivers (“carers” in the United Kingdom), research public advisors and collaborators, health or research volunteers, health care professionals, and in-patient participation networks. In line with the information density principle described in the Methods section, most public collaborators represented 2 or more roles from which they could contribute to the study. We did not collect data specifically on demographics in a standardized way, as we wanted to leave participants open to speak about their identities, backgrounds, and subjectivities as they felt led. Some participants offered up this information during the FGs and their expressions of interest in participating. Additional details on their interests and backgrounds can be found in Multimedia Appendix 3.

Public collaborators varied in their experience with and knowledge of AI in health care. These included (1) some with no preexisting knowledge of AI; (2) some with preexisting knowledge through AI research and digital innovation in health care, panels awarding funding for AI projects in health care, and research or implementation projects of small-scale AI projects in health care; and (3) 1 participant with experience constructing AI scripts.

Public collaborators represented 6 different NIHR ARCs (North Thames, North West Coast, South London, West Midlands, Yorkshire and the Humber, and South West Peninsula).

Through iterative analysis, we identified 3 primary deductive themes, with 7 corresponding inductive subthemes (Table 1).

Public collaborators’ perceived advantages of implementing AI in health care focused on subthemes 1.1: system improvements and 1.2: improve quality of patient care and shared decision-making.

Three main subthemes emerged regarding public collaborators’ perceived challenges of AI in health care, including subtheme 2.1: challenges with security, bias, and access; subtheme 2.2: public misunderstanding of AI; and subtheme 2.3: lack of human touch in care and decision-making.

The 2 main subthemes emerged regarding public collaborators’ recommendations on PPIE for AI in health care, including subtheme 3.1: experience, empowerment, and raising awareness; and subtheme 3.2: acknowledging and supporting diversity in PPIE.

This consultation explored public collaborators’ perceptions of the implementation of AI in health care, and it found 3 primary deductive themes with 7 corresponding inductive subthemes.

Primary theme 1: advantages of implementing AI in health care, and its 2 subthemes: system improvements and improve quality of patient care and shared decision-making.

Primary theme 2: challenges of implementing AI in health care, and its 3 subthemes: challenges with security, bias, and access; public misunderstanding of AI; and lack of human touch in care and decision-making.

Primary theme 3: recommendations on PPIE for AI in health care and its 2 subthemes: experience, empowerment, and raising awareness; and acknowledging and supporting diversity in PPIE.

Across these themes, public collaborators noted several advantages of AI in health care, such as improved quality and efficiency of health care systems, as well as better diagnosis and treatment for patients. Public collaborators also discussed ethical and regulatory challenges for the implementation of AI as well as the resistance the public and patients feel toward AI due to concerns over bias and access, data protection, and perceived threats AI poses to patient care. Overall, public collaborators felt that patients and the public need to play a role in AI development and implementation to safeguard patient care and maximize AI’s efficacy in practice.

This consultation’s contribution lies in (1) consolidating recent findings across various stakeholders and fields of research (data, AI, co-design, PPIE, and qualitative research); (2) connecting key themes represented in patients to those represented in other stakeholders; and (3) connecting gray and research literature.

The current research literature on the implementation of AI focuses largely on health care professionals’ perceptions instead of those of patients and the public [28-32]. Interestingly, these professionals’ perceptions overlap with those of the public in our consultations. Like our “advantages” theme, health care professionals in the studies of Laï et al [31] and Choudhury and Asan [29], for example, found AI can save doctors’ time, reduce medical errors, and improve diagnosis and treatment for patients. Interestingly, they encountered similar concerns around AI as our challenge subthemes; for example, data accountability and security, ethical dilemmas, limited understanding of AI, and detrimental impacts on physician-patient relationships could be key barriers to the successful implementation of AI [28,29]. Several of these “concerns” were echoed in public involvement literature focusing on patient perceptions of health data use, which often touched on AI in health care [33,34]. These included health data used for profit, discrimination bias, “loss” of the patient, and weakening of the patient-clinician relationship [33,34]. Recent gray literature reports from the NHS AI Lab and NHS AI Lab awardees further illustrate extremely similar themes as our findings, including concerns about algorithm bias, harming clinician-patient relationships, health inequalities [13-15], and general fear and misunderstandings [16]. What is more significant is that our consultation not only reiterates several of these concerns and hopes but does so from a combination of researcher and public perspectives.

This consultation shows that public involvement is both desired by the public and key to the development and implementation of AI in health care. Successful implementation and adoption of AI in health care will require consideration of human factors beyond AI’s predictive power and technological capabilities [29]. Patients and the public can serve as collaborators with AI in health care, either through public involvement, coproduction, co-design, or any similar methodology, and can thus shape a form of AI that addresses these concerns, a position shared by other projects [13-16,33-35]. In facilitating relevant PPIE activities, researchers need to be especially careful of the themes identified by this consultation: experience, empowerment, raising awareness, and acknowledging and supporting diversity in PPIE. PPIE can assist in reducing the so-called development-to-implementation gap of AI, referring to the mismatch between the goals for which AI is developed by manufacturers and the needs of practitioners and patients in practice [36,37]. Patients and the public bring unique knowledge of what they need when undergoing health care and how AI can help fulfill these needs.

Our consultation suggests that PPIE can lead to more awareness of AI among patients and the public, but it also inspires a sense of ownership of one’s treatment. Not only can this support resolving some of the misconceptions and fears that the general population has toward AI, but it can also lead to increased trust in and acceptance of AI by patients and the public. In a time where personalized medicine and shared decision-making are at the forefront of accepted good clinical practice, engaging patients in the development and application of AI is just as important as engaging patients in their usual health care trajectories.

This study follows a few recommendations for policy and practice. Early engagement of patients and the public in the AI development and implementation journey as well as in designing policy on AI implementation is important. PPIE should represent people with varying backgrounds, such as different ethnic backgrounds, various health conditions and disabilities, various ages, and those with varying AI knowledge and expertise, as these determinants can impact the needs and therefore the usability of various AI applications. Educating patients and the public can aid in achieving successful PPIE by making sure that everyone can contribute and by resolving some of the misconceptions about AI [32].

This study not only engaged public collaborators as PPIE consultants but also involved 2 public collaborators in the analysis and interpretation of the data presented in this paper. It was also then reviewed by 3 additional public collaborators. These approaches strengthened these results by guaranteeing that the perspectives of public collaborators were respected and represented in the results. The size of the consultation can be considered a limitation of the study; however, given the depth of the data, it is unlikely that a larger sample would have changed the results substantially, especially given connections across other literature. Notably, similar PPIE consultations have had similar sizes [30]. Future research should expand beyond members of the public accustomed to PPIE activities, such as those included in this consultation.

We conducted a patient and public involvement consultation with individuals connected to NIHR ARCs from across England to center their perceptions on AI and the translation of AI into clinical practice in anticipation of a nationwide, cross-ARC research collaboration. This consultation supported similar studies and PPIE work conducted with different populations on AI and data sharing, illustrating the strength of these themes and their relevance across different stakeholders. This cements the need for AI in clinical practice to be co-designed with a diverse group of patients and members of the public.