The study adopted a 2 parallel-armed randomized controlled trial design with repeated measures at baseline (T0), postintervention (T1), 3-month (T2), and 6-month follow-ups (T3). Qualitative interviews with focus groups involving participants in the intervention group were conducted postintervention to explore participants’ views about the intervention’s acceptance, strengths, and limitations, and suggested improvements.

This study was approved by the Institutional Review Board of the Hong Kong Polytechnic University (HSEARS20221031003). Informed consent (Multimedia Appendix 1) was obtained from all participants before their participation. Participants were assured that their responses would remain confidential and that all data would be anonymized and stored securely. Only the research team had access to the data, and all identifying information was removed before analysis. The trial protocol was prospectively registered in ClinicalTrials.gov. There were no deviations from the prospectively registered protocol.

The trial was conducted in Hong Kong between October 2022 and June 2024. Recruitment involved collaborating with local district health centers to deliver email, call, or poster invitations and social media promotion (ie, Twitter [rebranded as X] and Facebook [Meta]). Participants were included if they (1) are Hong Kong residents aged 18 years or more, (2) are vaccine hesitant (not taking COVID-19 vaccines or receiving involuntary COVID-19 vaccines such as receiving vaccines due to government, school, or employer mandate), (3) have internet access, and (4) are able to read Chinese or English. Participants were excluded if they (1) had medically diagnosed cognitive impairments or mental disorders, or (2) were currently engaged in other COVID-19 vaccine promotion programs. The sample size was determined using the GLIMMPSE 3.0.0 software (University of Colorado Denver). The calculation, based on estimated correlations among repeated measures (0.6 for T0-T1, 0.5 for T0-T2, and 0.4 for T0-T3), a statistical power of 80%, and an attrition rate of 20%, indicated a total of 168 participants (84 per group) would be the minimum required for the study. A purposive sampling by key informants in the intervention group (containing those who improved most and least on vaccine hesitancy postintervention) was conducted for focus-group interviews, until information saturation was reached as no relevant new codes were found in data.

Eligible participants were randomly assigned to 1 study group (1:1) using a stratified randomization based on the age groups (ie, 18‐29 y, 30‐44 y, 45‐59 y, and 60 y or older). To ensure allocation concealment, random assignment was performed by an external administrator using a web-based randomization service (sealed envelope website). Upon written consent and the completion of the baseline assessment, the external administrator inputted participant numbers into the Sealed Envelope system to determine group assignment for each age group. The group allocation was subsequently disclosed to the research team and participants. Due to the nature of the intervention, participants could not be masked but were reminded to keep it confidential to prevent contamination. Furthermore, the intervention and control groups interacted within separate platforms to receive their respective treatments. Only the intervention group was provided with accounts to access the AI digital assistant for completing the intervention. The data collector and data analyst were blinded to group assignment.

The development of the AI digital assistant was theoretically underpinned by the Vaccine Hesitancy Determinants Matrix Model and evidence-supported by qualitative interviews with Hong Kong adults. The AI digital assistant was validated by an expert review process and pilot-tested by users. The details of the intervention development, validation, and pilot test have been reported [31]. The AI digital assistant incorporates a web-based educational program with a MI-oriented, AI-driven chatbot “Auricle” (Digital Psychosocial Health Research Group, School of Nursing, The Hong Kong Polytechnic University). The web-based educational program consists of five modules designed to address vaccine hesitancy (Multimedia Appendix 2): (1) basic knowledge of COVID-19, (2) basic knowledge of COVID-19 vaccines, (3) common questions about COVID-19 vaccines, (4) myths about COVID-19 vaccines, and (5) efforts by the Hong Kong government. Each module was embedded with an AI-driven chatbot, which provided MI-oriented dialogues, encouraging users to reflect on the educational content and initiating their motivation to vaccinate. The 4 processes (ie, engaging, focusing, evoking, and planning) and 4 principles (ie, expressing empathy, developing discrepancy, rolling with resistance, and supporting self-efficacy) were included in MI-oriented dialogues. A logical algorithm was used to structure the MI-oriented dialogues into the AI-driven chatbot system by professionals in computer science. The AI-driven chatbot was powered by natural language processing to analyze user input and provide real-time and personalized conversations. “Auricle” also provided assessments of vaccine importance, confidence, and readiness with single-item questions rating from 0 to 10 at the end of dialogues [31]. Participants progressed through 1 module per week for 1 hour over a 5-week period. After self-learning each module’s educational content, scoring over 80% on multiple-choice questions, and finishing dialogues with “Auricle” within a week, a new module was released for learning in the next week. Figure 1 displays the interface of the AI digital assistant.

Participants in the control group received weekly WhatsApp (Meta) messages directing them to government websites relevant to basic knowledge, common questions, myths about the COVID-19 vaccine, and government efforts over 5 weeks. They were also required to complete multiple-choice questions each week, as the participants in the intervention group did. Adherence was evaluated based on their completion of multiple-choice questions.

Before the commencement of the intervention, participants in the intervention group received an electronic user manual. The manual instructed participants to use the AI digital assistant with detailed procedures, including guidance for the initial account registration, self-learning with the web-based educational modules, chatting with “Auricle,” and frequent questions and answers. An automatic email was sent to participants to remind them to complete the module learning weekly. Participants who completed at least 2 out of 5 modules were considered completers and were included in data analysis. A research assistant weekly checked the dashboard of the AI digital assistant to follow up on the intervention progress on module content learning, multiple choice practice, and the interaction with “Auricle.” The research assistant also sent messages and made calls to participants using WhatsApp to encourage their engagement in the intervention. Any adverse events occurring during the study were handled cautiously and recorded.

Quantitative data were numerically coded and analyzed using IBM SPSS version 26. Intention-to-treat analysis was conducted using the last observation carried forward for missing data. Descriptive statistics were used to summarize the data. Randomized testing between 2 study groups at baseline was conducted for categorical variables using the chi-square test, while continuous variables were analyzed using independent t tests or Mann-Whitney U tests. Intervention effectiveness was assessed using the generalized estimating equation model, controlling for covariates identified by the randomized testing. The model evaluated group, time, and interaction effects, with post hoc comparisons performed to identify significant between-group differences using estimated marginal means [37]. Effect sizes for between-group comparisons were calculated using Cohen d [38]. The 30% top outliers of the outcomes were removed in data analysis [39]. A sensitivity analysis was conducted, and the generalized estimating equation model results using complete case analysis are provided in the Multimedia Appendix 3 supporting the robustness of the findings. Tape-recorded interviews were transcribed into Cantonese by 1 researcher (T-CS) and cross-checked by the other researcher (ML). Qualitative data generated from interview transcripts were content-analyzed independently by 2 researchers (T-CS and ML) with training in qualitative content analysis following the process of open coding, creating categories, and abstraction [40]. Any discrepancy in coding and categories was discussed and finally confirmed by the research team and explained with verbatim data.

The participants’ recruitment occurred from October 2022 to May 2023, with follow-up concluding in June 2024. The study flowchart in Figure 2 adheres to the CONSORT (Consolidated Standards of Reporting Trials) guidelines (the CONSORT-EHEALTH [Consolidated Standards of Reporting Trials of Electronic and Mobile Health Applications and Online Telehealth] checklist is provided in Checklist 1). A total of 178 participants were selected from 455 eligible participants using stratified randomization based on 4 age groups. However, 1 of 178 participants lost contact. Finally, 177 eligible participants were allocated using stratified randomization based on 4 age groups, with 91 assigned to the intervention group and 86 to the control group. Table 1 presents the detailed number of participants in each age group. The adherence rate of completing required sessions was 76.3% and 88.6% for the intervention and control group, respectively. No direct intervention-related injuries or adverse effects were reported.

Table 1 describes participants’ characteristics at baseline. Among 177 participants, the mean age was 43.38 (SD 16.42) years, 65% (115/177) were female, 64.4% (114/177) had college or above education, 54.8% (97/177) were employed, and 68.4% (121/177) had a monthly household income above HK $20,000 (US $2548). The majority (152/177, 85.9%) of participants reported not having any chronic illnesses, and 50.2% (89/177) considered their health status as very good or good. In addition, 68.4% (121/177) had a history of testing positive for COVID-19, 75% (129/177) experienced side effects after receiving the COVID-19 vaccine, and 78.5% (139/177) had received 3 doses of COVID-19 vaccines. The sample showed moderate vaccine hesitancy (mean 30.20, SD 4.84; range 10‐50), low vaccine readiness (median 2 (0, 5), range 0‐10), moderate low trust in government (mean 40.38, SD 26.80; range 0‐100), and moderate vaccine-related health literacy (mean 10.64, SD 2.54; range 4‐16) at baseline. There are no significant group differences among all characteristics and outcomes at baseline.

Figure 3 presents weekly changes in vaccine importance, confidence, and readiness assessed by the chatbot among participants in the intervention group. During the 5-week intervention period, vaccine importance (from 6.06 to 7.44), confidence (from 6.43 to 8.00), and readiness (from 5.98 to 7.47) showed increasing trends. Participants in the intervention group reported moderate satisfaction with their participation and no significant group difference was identified when compared with the control group (mean 19.74, SD 0.91 vs mean 19.57, SD 0.96; t₁₄₁=0.949; 2-tailed P=.34).

Table 2, Table 3, and Table 4 display the intervention effects between groups and Figure 4 provides graphical representations. For the primary outcome, decreases in vaccine hesitancy were observed in the intervention group at T1, T2, and T3. However, no significant time-group interaction was detected. For secondary outcomes, the intervention showed significant effects for time (P<.001), group (P=.03), and their interaction (P=.042) in vaccine readiness. A significant group difference at T1 was identified (P=.048) with a medium effect size (Cohen d=0.52, 95% CI 0.12-0.86) in favor of the intervention. The intervention also showed significant effects for time (P=.002), group (P=.04), and their interaction (P=.04) in trust in government. A significant group difference at T1 was identified (P=.049) with a medium effect size (Cohen d=0.54, 95% CI 0.20-0.88) in favor of the intervention. For vaccine confidence, significant effects were found for time (P<0.001), group (P=.048), and the interaction (P=.02). Although no significant group differences were observed in the post hoc comparisons, a small between-group effect size (Cohen d=0.39, 95% CI 0.05-0.72) was identified at T1 in favor of the intervention. Increases in vaccine-related health literacy were observed post the intervention and significant time effect was found (P=.01).

Table 5 summarizes the characteristics of the 14 interviewees across 4 focus groups. In total, three categories were identified from content analysis of the interview data: (1) improved vaccine literacy, confidence, and trust in government; (2) hesitancy varied while readiness improved; and (3) facilitators, barriers, and recommendations of modifications on the intervention. Table 6 outlines the categories, subcategories, and example quotes.

The study results suggest that integrating AI chatbot and MI techniques can benefit vaccine promotion by enhancing vaccine-related health literacy and cultivating positive attitudes to improve individuals’ readiness for vaccination. The findings underscore the potential of digital tools combined with communication techniques in public health strategies for coping with vaccine-preventable diseases more effectively.

The AI digital assistant did not significantly improve vaccine hesitancy, aligning with previous research that has shown mixed results for interventions such as virtual games, apps, and MI [41]. One possible explanation for insignificant changes is the deeply rooted nature of vaccine hesitancy, which may not be easily altered by short-term interventions [42]. Another potential explanation might be attributed to the intervention conducted postpandemic [43]. Our participants suggested in the interviews that the AI digital assistant is helpful in reducing hesitancy due to increased vaccine knowledge and confidence. However, they perceived a low necessity for vaccination due to the perceived low susceptibility and severity of infection post pandemic as well as the lack of compulsory policies. The findings implied both the internal and external factors should be considered for addressing vaccine hesitancy [34]. Specifically, joint efforts should be made by health professionals and health departments to increase individuals’ vaccine health literacy and confidence, improve their perceptions of susceptibility and severity, and introduce policies supportive of vaccination and acceptable to the general public [44]. In addition, the vaccine hesitancy scale was not specifically designed for COVID-19 vaccines but for general vaccines, possibly affecting the sensitivity of detecting changes in attitudes specific to the pandemic context [45]. Future research is recommended to use the measurement designed for specific vaccines and further explore the intervention effects.

The AI digital assistant achieved a significant improvement in vaccine readiness with a moderate effect size immediately post intervention [46]. This aligns with studies showing that tailored communication strategies, MI, and educational components can effectively boost short-term vaccine readiness through enhancing vaccine-related knowledge and confidence, though their long-term impact remains uncertain [41,47]. Qualitative feedback from our study indicated that participants perceived the information provided by the AI digital assistant facilitated their readiness for vaccination when they considered taking vaccines. Vaccine readiness was suggested to be directly associated with vaccination behavior. To preserve the initial gains in vaccine readiness for long-term effects, it should be beneficial to explore approaches such as “top-up” sessions, updating the intervention content to reflect current health concerns, or integrating reminders and prompts that reinforce the importance of vaccination over time [48]. While AI-driven digital tools have proven effective for short-term interventions, their long-term success likely depends on adapting content and delivery methods to better address evolving needs and maintain the motivation to vaccinate [49]. Interestingly, despite positive changes in vaccine readiness, vaccine hesitancy remained high in this study due to a low perceived vaccine necessity in the current postpandemic context. This finding revealed the complex nature of vaccination decision-making and the dynamic relationship between vaccine hesitancy and readiness over time. In addition, it is important to highlight that while the statistically significant improvement in vaccine readiness is noteworthy, it does not necessarily guarantee meaningful clinical changes in practical vaccination promotion [50]. Future implementation studies are needed to explore these findings further and address existing contextual barriers [51].

The intervention significantly boosted vaccine confidence from baseline. Although the intervention group maintained higher confidence, the between-group differences were not statistically significant, likely due to the small effect size and limited statistical power [52]. This aligns with studies on the impact of digital interventions on vaccine confidence, which suggest that both digital approaches combining educational content and educational interventions alone contribute to the increase in vaccine confidence [53,54]. Qualitative data indicated that the AI digital assistant’s tailored information and communicative features effectively motivated participants to seek information and addressed their concerns about vaccine safety, thereby enhancing vaccine confidence. Future research should consider maintaining these positive effects by developing interventions that continuously engage participants and address evolving concerns. In addition, the intervention led to a significant increase in trust in the government with a moderate effect size immediately post intervention, which could be attributed to a specific module content visualizing the government’s pandemic management efforts and validating government-provided vaccine information [55]. Evidence demonstrated that enhanced trust in government positively correlates with vaccine confidence and readiness, which may contribute to overall vaccine acceptance [13]. Personalized and empathetic communication, facilitated by chatbots’ real-time and tailored responses, played a crucial role in addressing individual concerns and building confidence in government initiatives [47]. This highlights the value of using the MI-oriented AI chatbot to deliver clear, comprehensive communication to maintain trust and enhance vaccine attitudes.

The AI digital assistant led to a slight and long-term increase in vaccine-related health literacy at postintervention and 3-month follow-ups. The slight decrease observed at the 6-month follow-up in our study could be attributed to the lack of ongoing reinforcement [56]. This finding is consistent with previous research, which has reported mixed results regarding the sustainability of vaccine-related health literacy. Some studies have shown lasting gains, while others have noted declines post intervention, potentially due to variations in the content, delivery methods, and the degree of participant engagement over time [57,58]. Qualitative feedback highlighted the AI digital assistant’s benefits in boosting vaccine health literacy, specifically with increased information-seeking motivated by the MI-oriented chatbot and improved understanding through the web-based educational modules. The findings suggest the application of emerging AI-powered conversational agents and MI techniques in health education for behavior change [59].

This study has several limitations that should be acknowledged. First, the reliance on self-reported measures and single-item measurements may have introduced response biases and measurement error, potentially affecting the reliability and validity of the data. The Adult Vaccine Hesitancy Scale used to measure vaccine hesitancy was designed for general vaccines, not specifically for COVID-19, which may have affected its sensitivity. The generalizability of the findings is also limited due to the study’s focus on a specific demographic—Hong Kong adults. Furthermore, the limited number of intervention sessions may have been insufficient to produce significant changes in vaccine health literacy and hesitancy, particularly concerning effect size. The relatively lower adherence rate in the intervention group compared with the control group suggests that maintaining long-term engagement with the digital vaccine intervention was a significant challenge. Although the AI-driven digital assistant demonstrated encouraging short-term effects, it proved challenging to maintain user engagement over a longer period, which may have an influence on long-term outcomes. In addition, the postpandemic environment likely influenced vaccine attitudes, as many participants perceived a lower necessity for vaccination due to the reduced perceived severity of COVID-19. This could have tempered the intervention’s effects on reducing participants’ vaccine hesitancy.

The findings from this study have significant implications for public health strategies aimed at improving vaccine attitudes and coverage. Integrating MI techniques with AI-driven chatbots in health promotion efforts effectively addresses individual concerns and resistance about taking vaccines, enhancing vaccine readiness, and demonstrating their potential as practical tools for increasing vaccination rates [31]. Furthermore, qualitative feedback from participants suggests that refining the intervention content could further improve its efficacy. Specifically, incorporating more relevant and personalized information that addresses specific concerns could enhance the intervention engagement and long-term effectiveness [60]. Future studies should also examine the applicability of these digital interventions to other vaccine-preventable diseases and health behavior promotion, as well as their impact across diverse populations and settings [61,62]. Comparing the cost-effectiveness of AI-driven digital assistants with human-delivered MI interventions could provide valuable insights for optimizing resource allocation in public health initiatives. This broader perspective is essential for formulating more resilient and efficient ways to enhance vaccine attitudes and increase coverage on a wider scope. Our findings also suggest the importance of timely, attitude-targeted interventions in effectively shaping individuals’ acceptance of vaccines. Our program provides a foundation for making rapid adjustments to tailor the AI digital assistant for future emerging infectious diseases and newly developed vaccines.

The theory- and evidence-based, MI-oriented AI digital assistant resulted in significant changes in vaccine readiness, while the effects on vaccine hesitancy were insignificant and remain to be examined in future research. Despite the notable immediate improvements in vaccine attitudes, the effects diminished over time, highlighting the necessity for continuous intervention engagement and refinement. The study findings underscore the potential of integrating MI-based communication techniques with AI technologies to enhance public health strategies aimed at promoting positive vaccine attitudes. This innovative approach offers a promising direction for future research and practice in health communication, emphasizing the need for accessible, sustainable, and tailored interventions to boost vaccine acceptance for coping with future pandemics.