Bridging Technology and Pretest Genetic Services: Quantitative Study of Chatbot Interaction Patterns, User Characteristics, and Genetic Testing Decisions

Introduction

Background

Access to cancer genetic services is a crucial factor for cancer prevention and education, especially for those with familial cancer predisposition [1]. Demand for cancer genetic services is increasing due to technological and genomic science advances [2,3]. In the traditional genetic service delivery model, patients meet with a certified genetic counselor through a referral for a pretest genetic counseling appointment to discuss details of genetic testing (eg, genetics basics and testing options and procedures), allowing patients to make an informed decision about testing [3,4]. Although cancer genetic services are vital, many patients who meet eligibility criteria do not receive them [5,6]. Historically marginalized communities especially face hurdles such as limited resources and inadequate referrals, restricting essential testing and care [6,7]. In addition, the traditional model can be time-consuming and difficult to schedule in the face of workforce shortages [3]. To improve access, alternative models of service delivery have been developed and tested [8-12]. These approaches include telehealth appointments [13], use of genetic counseling assistants to optimize genetic counselor time [14], video- or web-based education [15], and digital health tools such as chatbots [16]. Each of these alternative delivery models has its own unique attributes as well as distinct advantages and disadvantages compared to traditional in-person genetic counseling [3].

Among the alternative solutions, chatbots (or conversational agents) are being increasingly used for service delivery amid the current technological evolution [17]. Chatbots are automated, programmed agents designed to simulate human interactions. They can use natural language processing to interpret user inputs and provide appropriate responses in human language through text or voice [18]. Chatbots have been adopted in many health care contexts, such as health behavior change interventions for smoking cessation and substance use reduction [19,20], promotion of health habits such as daily physical activity and stair-climbing habits among office workers [21,22], and addressing mental health concerns such as psychological distress and stress [23,24]. Specific to remote health service delivery, the 2024 review by Laymouna et al [17] summarized health care chatbot roles under 3 main types: patient support and care management, education and skill building, and health behavior promotion. This study focused on chatbots in delivering cancer genetic pretest education and expanding access to genetic testing.

Recent research has explored how chatbots can be effective remote service delivery alternatives for cancer genetic services [25]. Chatbots can play a vital role in risk screening and assessment based on individuals’ family history, as well as providing pretest education and counseling for patients considering genetic testing [25-27]. Previous studies have shown the acceptability and feasibility of using chatbots for these purposes [26,27]. They have particularly noted that participants favor chatbots for tasks of moderate complexity, viewing them as a useful complement, rather than a replacement, to standard care [28]. The Broadening the Reach, Impact, and Delivery of Genetic Services (BRIDGE) randomized controlled trial showed that use of pretest cancer genetic services and completion of genetic testing were statistically equivalent between chatbot and standard-of-care service delivery models [29]. Previous studies have evaluated user interactive experiences with chatbots via a qualitative approach, such as through focus groups [30] and interviews [31]. Several survey studies targeting chatbot usability evaluations have included measures of user interactive experiences and service satisfaction within the surveys [26,32]. Notably, despite the growing number of studies on the accessibility and feasibility of chatbot health service delivery, limited attention has been paid to user interactions with chatbots in a real-world health care context—specifically, the reciprocal actions toward chatbot content [33]. However, valuable insights can be gained on user-chatbot interactions, such as identifying which prompts lead users to seek more information when they drop out of a conversation and the specific open-ended questions they ask. Gathering these data can guide future chatbot designs, making them more responsive to user needs. To address this gap, Chavez-Yenter et al [33] studied user interactions with the BRIDGE chatbot for pretest genetic education in a small feasibility study. Of 36 who started the chat, 30 completed it—most completers (21/30, 70%) showed interest in pursuing genetic testing, whereas the rest were uncertain. Participants who decided to take the test selected an average of 1.87 (SD 1.2) informational prompts and rarely asked open-ended questions, whereas participants who were unsure about taking the test selected an average of 3.67 (SD 2.9) prompts and typically asked at least one open-ended question [33]. Our study expands on prior work by examining the interactions of a large sample of users randomized to the arm receiving the pretest genetics education chatbot in the BRIDGE randomized controlled trial. This trial examined user interactions among primary care patients across 2 major US health care systems. In total, 3073 patients took part in the trial, including 1554 assigned to the experimental arm, all of whom were invited to use the chatbot.

Unlike the aforementioned descriptive feasibility study, our study also explored the potential influence of clinical and sociodemographic factors on use patterns. Understanding whether clinical factors such as inherited predisposition to certain cancers or having a primary care physician affect user interaction patterns can help tailor future chatbots to improve accessibility and interactivity, better addressing the needs of individuals with different clinical backgrounds [33,34]. To effectively leverage chatbots in expanding health care services and reducing inequities, it is crucial to enhance user experience across diverse population groups [17]. Key demographic factors such as age, sex and gender, race and ethnicity, and geographic location must be carefully considered to ensure equitable and inclusive service delivery. For instance, previous research in cancer genetics has demonstrated that, even when traditional barriers to genetic testing such as cost, insurance, and access are minimized, minoritized racial-ethnic patients remain less likely to receive recommended genetic care [35]. Gender-based differences in genetic testing awareness were highlighted in the focus group study by Hamilton et al [36]—male participants demonstrated a broader understanding of various disease contexts, including the use of genetic testing for paternity determination, whereas female participants primarily focused on their experiences with prenatal testing and reproductive health. Although not cancer-specific, the systematic review by Best et al [37] found that rural populations faced greater shortages in genetic resources than urban communities. This important issue needs investigation for chatbot delivery models. The 2023 review by Webster et al [25] of chatbot use in genetic cancer risk assessment and counseling found that only 2 out of 7 studies provided detailed data on patient demographics such as age, race, ethnicity, and sex. The 2021 study by Nazareth et al [27] revealed that 58.8% of the 61,070 genetic service chatbot users identified as non-Hispanic White, with users being significantly younger than nonusers [27]. The 2021 study by Heald et al [26] of 506 patients accessing a genetic service chatbot showed that 86% were White and 96% were non-Hispanic. It also found that individuals who were female, White, and received the chatbot link via a patient portal were more likely to engage with the chatbot, whereas Black individuals were less likely to do so [26]. The aforementioned insights show that genetic service delivery models work differently across various populations, highlighting the need to examine these differences further. In the context of chatbot-based delivery, Laymouna et al [17] found that chatbots can effectively serve diverse groups regardless of age, gender, race, or socioeconomic status but cautioned against replacing established health services given the potential to widen health disparities. Accordingly, our study explored whether certain groups may perceive the chatbot’s design or content more favorably. This underscores the importance of investigating how user interactions vary according to sociodemographic factors.

Overall, due to the limited research on how patients use cancer genetic service chatbots—particularly in large-scale, real-world health care settings—and the lack of empirical evidence on how clinical and sociodemographic factors influence those use patterns, this study was conducted to fill those gaps.

Objectives

To address these research gaps, this study explored the use patterns of a pretest genetics education chatbot in 2 large health care systems. The primary outcomes were to examine participants’ core interactive experiences: user interactions with informational prompts (how many times they opted to view more informational content), user open-ended questions (how many questions they asked and of what nature), and genetic testing decision after the completion of the chat. In addition, to better capture the nuances of user interactive actions, we explored 2 secondary outcomes, that is, user conversation completion status (whether an individual completed the entire content of the pretest education) and user dropout points (for participants who did not complete the entire chat, whether a user left the chat before any informational content). Furthermore, given the intention of using an educational chatbot to increase access to genetic counseling services for hereditary cancer, it is pertinent to explore whether clinical and sociodemographic factors are related to differences in the participants’ interaction with the educational content on genetics, as well as their genetic testing intention. Analyzing how patient characteristics related to their use of the chatbot, as well as their genetic testing decision after completing the chat, is important for a better understanding and development of future genetics education chatbots.

Methods

Study Design

Procedures

The BRIDGE randomized controlled trial has been previously described [38]. Eligible participants in 2 large health care systems (University of Utah Health [UHealth] and New York University Langone Health [NYULH]) were randomized 1:1 to either chatbot or enhanced standard-of-care genetic service delivery models. The chatbot was designed to deliver automated pretest cancer genetics education to patients who met National Comprehensive Cancer Network guideline-based criteria for cancer genetic testing. The primary aim of BRIDGE was to assess the equivalence of the chatbot delivery model to genetic counseling standard of care with regard to uptake of pretest genetic services and genetic testing [29]. The analysis presented in this paper focused on participants randomized to the intervention arm (n=1554) who were invited to interact with the pretest genetics education chatbot. These participants were sent a patient portal (MyChart in Epic) message inviting them to open the pretest genetics chat via a hyperlink, which launched the chatbot on a web browser independently from their electronic health record (EHR). Nonresponders received a reminder patient portal message 1 week later and up to 2 follow-up telephone calls. For those who started the chat, a transcript of the pretest genetics education chat was used to analyze user interactions in this study.

Participants

Patients in the BRIDGE trial were identified as eligible for cancer genetic testing based on their family history of cancer in the EHR [39-43]. Specifically, we used an algorithm-based system to screen family cancer history data at the population level stored in the EHR [44]. The Genetic Cancer Risk Detector algorithm identified unaffected primary care patients at the 2 study sites who met the criteria for genetic evaluation for specific patterns of family history of cancer, suggesting a hereditary syndrome (breast cancer, ovarian cancer, pancreatic cancer, multiple predispositions, or other) [45-47]. Additional eligibility criteria included being aged 25 to 60 years, being English or Spanish speaking, having had a primary care appointment at one of the study sites within the previous 3 years, no previous cancer diagnosis other than nonmelanoma skin cancer, and no previous genetic counseling or testing related to hereditary cancer. Finally, participants had to have an electronic patient portal account or be willing to create one. The inclusion criteria, including cancer family history, were verified by a genetic counseling assistant at each site.

Pretest Genetics Chatbot

The research team scripted the rule-based chatbot to reflect the content of a pretest cancer genetic counseling appointment [29,33]. To minimize the impact of users’ digital literacy on their ability to engage with the chatbot, the chatbot’s interface was designed to facilitate navigation through the chat by selecting predetermined responses. The chat began with a recorded video introduction from the lead genetic counselor at each study site. Participants were then asked to answer questions about the number of children and siblings they had, followed by viewing educational content. The educational content was scripted to ensure that all participants viewed a core set of information. They were also presented with 9 decision prompts throughout the chat. Each prompt appeared with 2 or 3 scripted choices that the participant could select, which allowed them to indicate that they would like to see additional information or additional examples for a topic or that they would like to move on. Specifically, the 9 prompts were genetics basics, gene mutations, positive result, risk percent, uncertainty in risk, medical record, genes included, lowering risk, and type of results. There were 4 examples of cancer risk information, including breast cancer as a core example illustrated in the risk percent prompt, and the other 3 examples (ie, colon cancer or Lynch syndrome, ovarian cancer, and pancreatic cancer) as optional examples for participants to choose from. After the branching following each prompt, the participant would always return to the core content in the chat script. The chat ended with a final question—“Would you like to move forward with genetic testing?” (response options: “yes,” “no,” and “I’m not sure yet”)—after which the chat was indicated as completed. At any point during the conversation, the participant could type an open-ended question into the chat. The system would use natural language processing to suggest potential questions. The chatbot was trained to provide scripted answers to questions. If the chatbot was unable to answer the question automatically, the question was sent to the study team. Multimedia Appendix 1 provides detailed information on the chatbot script. At any point during the chatbot session, participants had the option to exit the chat and discontinue their participation. For those who did not complete the session, a genetic counseling assistant contacted them up to 3 additional times via MyChart, phone, and mailed letter to answer questions; offer testing; and, in the case of the letter, provide additional information about recommendations based on their family history of cancer.

Data Collection and Outcomes

Overview

Transcripts were coded into a data collection tool in REDCap (Research Electronic Data Capture; Vanderbilt University) [48,49]. Data were extracted from the transcripts and coded by 2 independent coders. Data extracted included time stamps, participant responses, prompt decisions, and open-ended questions and responses.

Outcome Measures

Regarding primary outcomes, interactions with informational prompts were counted as the number of times a participant opted to view more informational content. The maximum number of informational prompts was 9. Open-ended questions were counted as the number of questions asked. Genetic testing decision was indicated as no (N), I’m not sure yet (U), or yes (Y).

Regarding secondary outcomes, chat completion status was coded as did not complete chat (0) and completed chat (1). Dropout point (for those who did not complete the entire chat) was coded as prior to informational content (0) and and after informational content (1).

Clinical covariates included algorithm criteria met based on specific patterns of family history of cancer suggesting a hereditary syndrome (breast cancer, ovarian cancer, pancreatic cancer, multiple predispositions, or other) and whether the participant had a recorded primary care provider in the EHR (yes or no).

Regarding sociodemographic covariates, we collected data on age (measured continuously), sex (male or female), race and ethnicity (White, Black, Hispanic, or other), number of children (0 or ≥1) and siblings (0 or ≥1), and preferred language (English or Spanish). We assessed urbanicity (urban or rural) using the 2010 rural-urban commuting area codes and categorized it as urban focused or rural city or town focused [50]. Neighborhood deprivation was measured continuously and assessed using the Neighborhood Deprivation Index [51]. We calculated the Neighborhood Deprivation Index for each zip code provided by our participants using the Housing and Urban Development–US Postal Service ZIP Code Crosswalk file from the 2023 quarter 3 data release [52].

Regarding time of day, we collected participants’ chat start and end times, which were categorized into 5 intervals: early morning (2 AM-10 AM), midday (10 AM-2 PM), afternoon (2 PM-6 PM), evening (6 PM-10 PM), and night (10 PM-2 AM).

Statistical Analysis

The data were cleaned and analyzed using SPSS Statistics (version 29.0; IBM Corp). Means and SDs were calculated for continuous variables, and counts and percentages were reported for categorical variables. Descriptive analyses were conducted for participant clinical information, sociodemographic information, and chat interactions. Pearson chi-square tests and independent 2-tailed t tests were used to examine differences in variables by study site.

We used logistic regression models to examine associations between participants’ clinical and sociodemographic characteristics and three user interaction outcomes: number of open-ended questions (categorized as 0 vs. ≥1), conversation completion status, and dropout point. We then used logistic regression models to assess associations between clinical and sociodemographic characteristics and participants’ genetic testing decisions (coded as 0=No/Not sure yet; 1=Yes). We also evaluated whether user interaction patterns were associated with genetic testing decisions. Therefore, the number of informational prompts and the number of open-ended questions asked were included as predictors. We constructed a multinomial logistic regression model to assess the relationship between clinical and sociodemographic characteristics and the number of informational prompts selected by the participants. On the basis of the distribution, we classified the number of informational prompts selected into 4 dependent variable categories: 0 prompts, 1 prompt, 2 prompts, and ≥3 prompts.

Initially, 481 participants began the pretest education chat. However, 2.8% (13/481) of user interaction transcripts could not be downloaded for log analysis from the web-based portal in the commercial platform due to nonresponsiveness of the commercial platform, resulting in a final analytic sample of 468 participants included in the analysis, with 226 (48.3%) from the NYULH site and 242 (51.7%) from the UHealth site. For the primary outcomes (ie, number of informational prompts, number of open-ended questions asked, and genetic testing decision), we included participants who completed the education chat (391/468, 83.5%) in the analysis, with 47.8% (187/391) from the NYULH site and 52.2% (204/391) from the UHealth site. For the first secondary outcome—completion status—we included all participants who launched the chat (n=468). For the other secondary outcome—dropout point—we included all participants who entered the chat but dropped out during the process (77/468, 16.5%). Due to missing data for race and ethnicity in the EHR (42/468, 9% for those who launched the chat; 34/391, 8.7% for those who completed the chat) in our sample, we conducted analyses on all target outcomes both including and excluding race and ethnicity. We present adjusted odds ratios (ORs) with 95% CIs. Statistical significance was determined at P<.05.

Ethical Considerations

The BRIDGE trial (approval number IRB_00115509) was approved as a single–institutional review board protocol by the University of Utah Institutional Review Board, consistent with current US National Institutes of Health policy for multisite RCTs. Because the trial compared 2 clinical service delivery models, the institutional review board approved a waiver of consent for the procedures described here. Data were de-identified. Participants did not receive compensation for the procedures described here.

Results

Overview

Participant Characteristics

Table 1 provides detailed information on the 9 informational prompts. Of the 1554 patients randomized to the chatbot arm, 468 (30.12%) started the pretest chat. For participants who opened and started the chat, their mean age was 43.3 (SD 9.5) years, as shown in Table 2. Most participants (n=360, 76.9%) were female, preferred the English language (n=460, 98.3%), identified as White individuals (n=323, 69%), lived in an urban area (n=437, 93.4%), and reported having a primary care provider (n=382, 81.6%).

Interactions With the Chatbot

Most participants initiated the chats in the afternoon (177/468, 37.1%) or evening (135/468, 30.5%). A similar trend was observed for chat end times, with most occurring in the afternoon (136/391, 35%) or evening (118/391, 30.3%). For participants who opened and started the chat (n=468), most completed it (391/468, 83.5%). Of those who completed the chat, most (315/391, 80.6%) decided to proceed with genetic testing. Of those who did not complete the chat, 74% (57/77) dropped out before any informational content. Table 3 provides detailed descriptive information on user interaction patterns.

Regarding informational prompts, more than one-third of the participants (151/391, 38.6%) opted for 1 prompt, followed by 23.8% (93/391) of users who opted for 2 prompts and 23.5% (92/391) of users who opted for ≥3 prompts. Only 14.1% (55/391) of the users opted for no informational prompts. Specifically, most of the prompts that users opted for fell under the following 4 categories: genetics basics (302/391, 77.2%), lowering risk (120/391, 30.7%), genes included (111/391, 28.4%), and medical record (110/391, 28.1%). The least opted for prompts were gene mutations (6/391, 1.5%), risk percent (4/391, 1%), and uncertainty in risk (0%). Except for the first prompt (genetics basics), which received the highest number of clicks, this likelihood of prompt selection was largely unrelated to their placement within the educational content. Regarding selecting additional cancer examples, few participants (3/391, 0.8%) chose to see the optional examples—only 0.5% (2/391) of the participants chose the ovarian cancer example, 0.8% (3/391) chose the pancreatic cancer example, and 0.5% (2/391) chose the colon cancer or Lynch syndrome example.

In terms of open-ended questions, among participants who completed the education chat (391/468, 83.5%), most (350/391, 89.5%) did not ask any questions, and 10.5% (41/391) typed at least one open-ended question for a total of 62 questions. Of the aforementioned 41 participants, 32 (78%) asked 1 question, 4 (10%) asked 2 questions, and 5 (12%) asked ≥3 questions. The most frequently asked questions were about the cost and insurance coverage of genetic testing (17/62, 27%). Other common topics were questions about logistics (16/62, 25.8%), more genetics or testing information (15/62, 24%), and personal health (9/62, 14.5%). In total, 3% (2/62) of the questions were concerned about whether genetic testing results would affect their future insurance status, and 2% (1/62) were family related. There were 7 other questions: 2 (29%) requesting a call from the service, 1 (14%) asking about coordinating the test with an upcoming blood draw, 1 (14%) seeking to change a previous answer, 2 (29%) inquiring about pausing and resuming the process, and 1 (14%) asking how privacy is protected. Table 4 provides detailed information on the open-ended questions asked by participants.

Primary Outcomes

Number of Informational Prompts Selected

Participants had the opportunity to opt for a maximum of 9 informational prompts throughout the conversation. The odds of opting for 1 prompt, 2 prompts, and ≥3 prompts relative to 0 prompts are described in Tables 5 and 6. The models, either including race and ethnicity in the predictive block or not, indicate that only the impact of the study site was significant—participants from the NYULH site were more likely to select ≥3 informational prompts in the chat than those from the Utah site (for the model including race and ethnicity: OR 3.12, 95% CI 1.35-7.24, and P=.008; for the model not including race and ethnicity: OR 3.75, 95% CI 1.72-8.17, and P<.001). None of the other factors, including clinical and sociodemographic characteristics, were significantly associated with the number of informational prompts opted for.

Number of Open-Ended Questions Asked

Participants had the opportunity to ask open-ended questions to the chatbot throughout the conversation. The odds of asking one or more questions are described in the “open-ended questions” column in Table 7. The models, either including race and ethnicity as a predictor or not, indicate that none of the clinical or sociodemographic characteristics were significantly associated with whether the participant would ask open-ended questions.

Genetic Testing Decision

After completing the pretest genetics education chat, each participant was asked whether they would like to proceed with ordering genetic testing. The odds of a participant saying yes and requesting genetic testing are described in the “genetic testing decision” column in Table 7. The models, either including race and ethnicity in the predictive block or not, indicate that none of the clinical or sociodemographic characteristics were significantly associated with whether the participant opted to pursue genetic testing. However, user interaction was associated with the following genetic testing intention—for the model not including race and ethnicity as predictor, participants who selected ≥3 informational prompts in the chat were less likely to opt for genetic testing than those who selected no informational prompts (OR 0.33, 95% CI 0.12-0.91; P=.03); participants who asked one or more open-ended questions were less likely to opt for genetic testing than those who did not ask any open-ended questions (OR 0.46, 95% CI 0.22-0.96; P=.04). The aforementioned significant associations were not found in the model including race and ethnicity as a predictor, which had 34 missing cases.

Secondary Outcomes

Conversation Completion Status

The odds of completing the pretest genetics education chat are described in the Chatbot completion status column in Table 8. The models, either including race and ethnicity as a predictor or not, indicate that only the impact of family cancer history was significant—in the model including race and ethnicity as a predictor, participants with family history of ovarian cancer (OR 3.26, 95% CI 1.30-8.20; P=.01), pancreatic cancer (OR 3.27, 95% CI 1.26-8.47; P=.02), and multiple or other cancers (OR 4.22, 95% CI 1.22-14.58; P=.02) were more likely to complete the chat than those with family history of colon cancer or Lynch syndrome. In the model not including race and ethnicity as a predictor, participants with family history of breast cancer (OR 2.66, 95% CI 1.06-6.70; P=.04), ovarian cancer (OR 3.72, 95% CI 1.58-8.76; P=.003), pancreatic cancer (OR 3.90, 95% CI 1.62-9.41; P=.002), and multiple or other cancers (OR 5.43, 95% CI 1.65-17.84; P=.005) were more likely to complete the chat than those with family history of colon cancer or Lynch syndrome. None of the other clinical or sociodemographic characteristics significantly predicted whether the participants would complete the chat.

Dropout Point

For participants who did not complete the chat conversation, they could have dropped out of the chat before viewing any informational content or after starting to interact with informational content. The odds of dropping out of the chat after the informational content are described in the “dropout point” column in Table 8. The models, either including or excluding race and ethnicity as a predictor, indicate that none of the clinical or sociodemographic characteristics significantly predicted whether the participant would drop out of the chat after beginning to view any informational content. However, it is important to note that the vast majority did drop out before viewing informational content.

Discussion

Principal Findings

Many patients have limited access to genetic services, and due to the increasing need to deliver cancer genetic services for improved health outcomes, alternative delivery models for pretest genetics education are gaining more attention [3]. Recently, researchers have been exploring how chatbots can be used as a cost-effective model for such delivery [25]. By taking on routine educational tasks, these automated tools help address the staffing bottleneck that genetic service providers often face, ultimately improving patient access to critical services and enhancing overall health outcomes [1]. However, limited attention has been paid to how users actually interact with these chatbots and how their sociodemographic backgrounds may influence these use patterns. This study explored user interaction patterns with our chatbot, examined how clinical and sociodemographic factors influence these patterns, and assessed how user characteristics and interaction behaviors affect decisions regarding genetic testing. In summary, our findings highlighted the chatbot’s acceptability to users—of the users who started the chat, most completed it and opted for additional information. In addition, participants demonstrated a strong willingness to undergo genetic testing. We found that most sociodemographic factors did not significantly influence user interaction patterns. This potentially indicates the chatbot’s scalability across diverse populations if they can access the chat. Future efforts could explore other potential predictors such as health literacy levels [53] and individual dispositional traits such as tolerance for uncertainty [54]. Importantly, participants with high informational needs (ie, those who selected ≥3 informational prompts or asked at least one open-ended question) were less likely to opt for genetic testing. This finding highlights the importance of further enhancing and personalizing the chatbot’s educational content to meet users’ informational needs, as well as the potential value of offering follow-up consultations with genetic experts to address remaining questions and concerns in clinical practice.

This study closely investigated how users interacted with the chatbot, focusing on whether they tended to seek additional information within the conversation and ask further open-ended questions. Regarding how users interacted with the informational prompts and relevant examples, we have 3 main findings. First, although we found that the vast majority of users selected at least one prompt, a closer examination revealed differences in the selection of individual prompts. Specifically, 4 prompts received a high number of clicks, whereas others, such as gene mutations, risk percent, and uncertainty in risk, were rarely selected. We discovered that the likelihood of a prompt being selected was largely unrelated to its placement within the educational content. Future research should aim to understand why users select particular prompts and explore ways to enhance the appeal of prompts, such as by improving prompt title framing, images, or page descriptions [55]. Second, we observed that users from the 2 different sites varied in the number of prompts selected. Participants from the 2 sites differed not only in their selection of informational prompts but also in various user characteristics and interaction details. These findings suggest a need to further explore differences in information needs among various patient populations and develop ways to tailor educational content and delivery formats to meet these needs. In addition, only a very small number of users chose to view the 3 optional examples of genetic risk. Considering the power of exemplars in information comprehension and various persuasion contexts [56], future research should focus on increasing users’ motivation to view examples to enhance informed decision-making. This could be achieved by enhancing the effectiveness and attractiveness of the first example viewed to increase users’ interest in viewing more or by personalizing example recommendations based on users’ family cancer history.

Our examination of how users engaged with open-ended question features can provide valuable insights into enhancing the usability of cancer genetic service chatbots. Low response rates and poor quality of answers to open-ended questions have consistently posed challenges in user surveys [57]. Our study revealed that only a very small number of users posed at least one open-ended question (especially knowledge related). This might suggest that the scripted educational content adequately addressed their information needs. However, it is important to note that open-ended questions generally require more cognitive effort from users than closed-ended questions and scripted educational content [57,58]. Future research could focus on facilitating user engagement with open-ended questions, such as incorporating voice input options [59]. In addition, evaluating chatbot usability could include direct inquiries after chat completion to assess whether users feel that their information needs are fully met. Regarding the types of open-ended questions asked, a significant portion related to financial and insurance matters. In contrast to the previous feasibility study, in which users primarily asked about whether and what portion of testing costs would be covered by insurance [33], in this study, users were also interested in whether their genetic test results might impact their future health insurance status. Future genetic education initiatives could provide additional pertinent information on this subject, helping alleviate user concerns regarding genetic testing results, such as perceived stigma and insurance discrimination [60].

We further delved into the specifics of use patterns. First, the high chat completion rate indicates the acceptability of the chatbot design and user overall satisfaction. Users with different cancer family histories showed differences in their likelihood of completing the conversation. This insight highlights the importance of making further efforts to provide more targeted information that addresses the needs of individuals with different cancer family histories, as well as personalizing educational content based on their specific clinical contexts. Tailoring content to match user characteristics may enhance patient engagement with educational materials [33]. Second, of the small number of users who exited the chat before completion, a substantial majority left before encountering any informational content. For those who dropped out of chatbot services, as stated previously, the team followed up via multiple channels (ie, phone calls, emails, and mailed letters) to answer any questions and provide additional information. Future efforts may focus on examining whether the chatbot format or interface discourages users from continuing the chat. Follow-up interviews or surveys with these participants could help identify specific reasons for dropout and provide valuable suggestions for improving chatbot design and features to better meet users’ needs.

Notably, although the chatbot service delivery model performed equivalently to the standard-of-care model in providing pretest cancer genetic services and promoting genetic testing, most patients invited and assigned to either model chose not to initiate the pretest genetic services [29]. While this is consistent with other studies of pretest services [61,62], future efforts may focus on strategies to encourage potential patients to initiate a chat or participate in any type of cancer genetic pretest services. Previous findings from the BRIDGE trial have shown that engagement with pretest genetic services under both service delivery models is positively associated with previous health care interactions, including frequent use of the patient portal and primary care visits [63]. These findings suggest that promoting regular use of patient portals through improved usability and more user-friendly digital health navigators may help expand the reach of genetic services [63,64]. Increasing access to primary care, building trust in the health care system, and enhancing overall health care accessibility may also promote greater engagement with genetic services [63,65,66]. In addition, it is essential to develop strategies that enhance individuals’ motivation to act on their genetic cancer risks and incorporate these risks into their overall health care considerations [67,68]. The public’s generally low level of knowledge about genetic cancer, combined with the absence of immediate symptoms, may lead to underestimation of these risks and reduced perceived value of pretest services [69,70]. To address this, future efforts should place greater emphasis on informal and broader public education delivered through health care providers and media platforms, especially by promoting foundational knowledge and awareness of cancer genetics [71,72]. Enhancing public understanding of these risks may, in turn, increase individuals’ interest in and motivation to engage with genetic pretest services [69,73].

The ultimate goals of developing alternative pretest education models, such as using chatbots, are to enhance patients’ comprehensive knowledge of cancer genetics and support their related decision-making processes [29]. Therefore, another core objective is to understand participants’ decisions regarding genetic testing following the chat. Most users expressed a willingness to undergo genetic testing after completing the pretest education content. Specifically, we did not find any clinical or sociodemographic factors that influenced their decision to pursue genetic testing. However, we observed that users with a very high level of engagement with the chatbot, such as those who opted for ≥3 informational prompts or asked open-ended questions, had a lower likelihood of undergoing genetic testing. Consistent with the findings from the feasibility study, we believe that this finding indicates that, when individuals have a strong informational motivation regarding a health issue—seeking more information and having more questions—they may find it more challenging to make an immediate decision regarding the health issue [33,74]. Recognizing that individuals with high information needs may not immediately pursue genetic testing, it is essential to meet their informational needs over time to support their decision-making process.

Precision medicine emphasizes the importance of enabling informed health decisions, helping individuals understand their personal health status, the available options, and the potential benefits and consequences of each choice [75]. This aligns with our original intention to enhance the public’s informed decision-making on cancer genetic issues through diverse and accessible delivery models [29]. Future chatbot development should focus on continuously improving the answer pool for user-submitted open-ended questions. Common concerns and frequently asked questions should be proactively integrated into the core educational content. Beyond enhancing the chatbot’s information delivery, it is also important to consider how human support can complement the service. Previous research has suggested combining chatbot education with human counseling to deliver cancer pretest genetic services [16]. Therefore, in clinical practice, follow-up mechanisms could be implemented where genetic counseling assistants respond to users whose open-ended questions were not fully addressed by the chatbot, similar to the procedures in our trial. For users who interacted with numerous informational prompts but did not proceed with testing, follow-up from the genetic counseling team may help identify specific concerns and provide additional guidance and emotional support. Moreover, it is essential to gain a deeper understanding of the characteristics of users with higher information needs. Although our current models did not identify significant sociodemographic or clinical predictors, future research could explore other relevant factors, such as personal or broader family health history beyond cancer. Health literacy and genetics-specific literacy may also influence the amount of information that users seek before making testing decisions [76,77]. In addition, psychological traits such as intolerance of uncertainty (the degree to which individuals find uncertainty stressful or difficult to tolerate) [54] may influence their desire for more information. By identifying these characteristics, we can design cancer pretest genetic services that offer more tailored, relevant, and personalized information to better support individuals with high information needs.

As new technologies, particularly generative artificial intelligence (AI), continue to advance, there is a growing interest in harnessing these innovations to enhance chatbot health service delivery. AI-driven or machine learning–based chatbots provide personalized and engaging interactions tailored to the unique needs of users [78]. However, concerns persist regarding the accuracy and unpredictability of the information they deliver, violation of scope boundaries (eg, providing health care advice outside the intended scope of a chatbot), and the potential violation of user privacy [79,80]. In contrast, traditional rule-based chatbots use scripted content created by expert research teams. In our case, the scripted material was derived from actual consultations during pretest cancer genetic counseling sessions. While this method ensures a high level of professional accuracy and consistency, it may lack sufficient personalization [81]. Future efforts may combine the strengths of both rule-based and AI-driven content to optimize user satisfaction and engagement with pretest genetic education.

In the field of cancer genetic testing, theoretical frameworks are still underused [82]. To enhance chatbot interactivity in a theory-driven manner, we recommend focusing on the following. First, drawing from the technology acceptance model, increasing user engagement with new technology (or services based on new technologies) requires enhancing both perceived usefulness and ease of use [83]. As advancements in IT and AI continue, it is critical to make chatbot content more informative and user-friendly to enhance user engagement and intention. Second, the theory of motivated information management highlights how individuals manage information-seeking behavior [84]. This process involves 3 stages: the interpretation phase (recognizing uncertainty or anxiety about an issue), the evaluation phase (weighing the benefits of seeking information and personal capacity to do so), and the decision phase (choosing how to manage the information search) [84]. To design better chatbots for cancer genetics, it is essential to identify the primary concerns of users and their key uncertainties and incorporate design features that enhance their confidence and motivation to engage with the chatbot. In summary, future cancer genetic chatbot designs should leverage insights from emerging theory-based literature on public acceptance of new technology while carefully considering the unique aspects of cancer genetic information and user motivations to seek it. This approach will lead to more effective and user-centered chatbot cancer genetic services.

Strengths and Limitations

Compared with the small feasibility study by Chavez-Yenter et al [33], the data from the full trial significantly expanded the sample size, study sites, and diversity in terms of race and ethnicity. This study conducted a more detailed examination of user interaction and use patterns in a real-world health care setting at 2 different sites, each with patients with distinct sociodemographic characteristics. However, several limitations need to be acknowledged. First, there were several technical issues with the commercial platform. As noted previously, we were unable to download 13 user interaction transcripts for log analysis. While this represents a small data loss (13/468, 2.8% of the participants), which likely did not impact the overall patterns of our findings, it nonetheless reduced the completeness of our dataset. In addition, the commercial platform used in this study only provided users’ session start and stop times but did not capture pause durations. This hindered our ability to calculate users’ “time on task,” which we believe is an important descriptive metric worth reporting and analyzing in future studies. Second, this trial offered the chatbot through patient portal messages (MyChart). While this approach was convenient and scalable, it may have introduced selection bias by primarily reaching individuals who are already digitally engaged—those with access to the portal, sufficient digital health literacy, and reliable internet access. Thus, this method may have excluded individuals whom the health care system has long underserved. Although we had a much larger and more diverse participant pool than the earlier feasibility trial, the participants in this trial were still predominantly non-Hispanic White and female individuals. Moreover, the number of Spanish-preferred users and rural participants was also very low. These demographics may reflect a bias in user composition due to disparities in access to digital health resources, such as the patient portal, and the presence of family history documentation within the EHR [85]. Accordingly, future efforts could be directed in the following ways. To improve equitable access to genetic pretest services, we recommend implementing additional outreach strategies to enhance the collection of information on family history of cancer. To alleviate the burden of digital literacy on chatbot engagement, we propose implementing and testing additional features, including AI-powered interactions and voice input options for submitting open-ended questions. These additional features may help mitigate the negative impact of limited digital literacy or experience on user interactions with the chatbot. For individuals without home internet or access to digital health tools, digital health approaches can be supplemented with in-person and telehealth approaches.

Third, our findings may be limited by incomplete or biased family history data in the EHR. Our inclusion criteria were based on the 2018 National Comprehensive Cancer Network guidelines. While the review of family history information by genetic counseling assistants ensured that those who receive testing met the criteria, our prior work has shown systematic underidentification due to biases in the available cancer family history information in the EHR [85]. This also highlights the importance of enhancing the collection of family history information within the health care system. Fourth, we may need to make additional efforts to capture users’ chatbot interaction experiences more effectively and identify more nuanced characteristics that influence their engagement patterns. In this study, we focused solely on observed and recorded interaction patterns as our main focus was to track and understand how users interacted with the chatbot. Future studies could include postinteraction surveys after the chat to assess user experience, such as their perceived satisfaction and helpfulness of the chatbot, offering a more comprehensive understanding of the overall interaction experience. Meanwhile, although we aimed to explore the association between clinical and sociodemographic factors and use patterns, we did not find many significant predictors. This may suggest some scalability of the chatbot content and features. However, this may also be because the factors we included were insufficient to capture some important and relevant characteristics. Future studies should explore additional clinical and sociodemographic factors such as general personal disease history and family disease history, medical mistrust toward health care systems, and perceived social support networks, as well as other potential predictors, including literacy levels and motivations for seeking information specific to cancer genetics.

Conclusions

Despite the limitations, the results of this large-scale trial demonstrate that chatbots can serve as a cost-efficient, convenient, and broadly applicable alternative for pretest cancer genetic education. Our analysis of user interaction patterns with the chatbot content revealed its effectiveness in engaging users, as indicated by the high completion rate and frequent selection of informational prompts. In addition, we examined how clinical and sociodemographic factors impact these interaction patterns, providing insights to improve the scalability of chatbot services across diverse population groups and support personalization to better address the needs of individuals from different backgrounds. Enhancing individuals’ genetic knowledge and cognitive skills enables them to make informed decisions about genetic service uptake, such as genetic testing [86]. By outlining user interaction patterns with cancer genetic education chatbots, our findings offer further empirical support for chatbots as an effective delivery model, potentially accelerating the enhancement of genetic literacy among the broader public and contributing to the prevention of cancer-related genetic diseases.