Between June and October 2021, several months after COVID-19 vaccines became widely available in the United States, we held 4 web-based focus groups with a total of 7 individuals aged 25-61 years working in public health or vaccine outreach roles and 13 people aged 18-28 years (see
Participant demographics.
| Characteristic | Participant | |
|
|
||
|
|
Female | 16 (80) |
|
|
Male | 4 (20) |
|
|
||
|
|
High school graduate | 4 (20) |
|
|
Some college, no degree | 3 (15) |
|
|
Bachelor’s degree or higher | 13 (65) |
|
|
||
|
|
American Indian or Alaska Native | 1 (5) |
|
|
Asian or Pacific Islander | 4 (20) |
|
|
Black or African American | 5 (25) |
|
|
Hispanic or Latino | 3 (15) |
|
|
White | 4 (20) |
|
|
More than 1 race or ethnicity | 3 (15) |
|
|
||
|
|
18-29 | 15 (75) |
|
|
30-49 | 3 (15) |
|
|
50-69 | 2 (10) |
| Self-reported annual household income (2021; US $; n=19), median (range) | 56,000 (25,000-200,000) | |
| Vaccinated (N=20), n (%) | 18 (90) | |
Analysis of FGDs identified themes describing the message preferences of young adults and public health workers in urban American communities. In the following quotes, we describe participants as either public health workers (“advocates”) or young people.
The credibility of a chatbot message was the predominant theme influencing response selection. Both young participants and advocates said messages achieved credibility through (1) message directness and (2) the establishment of rapport between the user and chatbot through conversational syntax and empathetic, reflective statements. Of the 26 total responses, 6 (23%) consisted of an empathy-factual composite style, beginning with a user-centered, reflective message such as “it sounds like you have concerns” or “this is an important question for a lot of people.” Empathetic responses used casual, nontechnical language to answer questions using evidence in what participants said was a transparent, contextualized response. A young woman described the style as, “kind of sticking to the facts in a colloquial/conversational manner—doesn’t feel like I’m reading a newspaper or a research paper” (Participant 09).
Another young woman liked the conversational tone and “extensive” detail provided in a factual-style message about side effects, which stated that the vaccine’s side effects “should resolve within one or two days of vaccination.” She said, “It covered it pretty extensively. It sets me up for what I can expect. And then I would feel more secure knowing...the chatbot, it’s giving me [a] correct answer” (Participant 19).
Both groups of participants regarded scientific messages as credible, saying they trusted the message since it was communicated by a Johns Hopkins University chatbot. In a typical response about how the brand affected message reception, a young man said: “I felt this [was] trustworthy, [be]cause I knew it came from like Johns Hopkins, which has a strong reputation” (Participant 07).
Although advocates and young participants both preferred empathetic statements prefacing a full, factual response, some felt such messages seemed inauthentic. The phrase “having doubts is normal,” in the words of a male advocate, “makes [the chatbot] more humane, more human-like, and more accepting” (Participant 08). Meanwhile, participants felt the phrase “I hear you” was marginally reassuring, but the statement “I wondered about that too” seemed “weird and fake [from a chatbot]” to one young woman (Participant 05).
Message attributes supporting and hindering credibility with young focus group participants. Textual excerpts coded with both directness- and rapport-related variables (eg, each cell shows a textual passage double coded with a directness-related variable and rapport-related variable). P: participant.
Principled responses sought to persuade users to get vaccinated by appealing to an individual’s responsibility for the well-being of a larger community and using reasoning instead of evidence (see example in
However, the style had some appeal with 4 young participants of color. One participant, a young man, felt such direct messaging was warranted:
I like [principled response] D because at this point in time, I think we need more aggressive messaging. Like, guilt people, shake people, let’s be serious...don’t put your friends and family at risk. Just get the shot.
Meanwhile, advocates aged >30 years often preferred such messages, sometimes wanting messaging to be “stronger” and “louder” to combat misinformation around vaccine myths. As one advocate said, “I think that message should be really pushed out a little louder. [The vaccine] prevents you from getting deathly ill” (Participant 16).
Advocates preferred principled responses that emphasized shared responsibility to prevent COVID-19 spread. As one young advocate said, these responses “made me think about the risks I posed not just [for] myself, but the people around me” (Participant 01). Moreover, advocates shared concerns about their family members and discussed feeling surrounded by people that “weren’t doing their part.” As one advocate said, “We’ve been getting hit hard, especially in the Black and Indigenous, Latinx, API communities, and this thing isn’t going anywhere anytime soon” (Participant 18).
An advocate recalled seeing community members previously hospitalized with COVID-19 “still not wearing your mask...it just made me more weary” (Participant 17). Such fatigue with community members not taking precautions to protect themselves and one another was linked with participant preference for principled messaging that was direct and insistent on communal responsibilities.
Participants rarely preferred the rational arguments shown. We tested the following message in response to the question, “Are COVID vaccines worse than the disease itself?”
The trouble with that logic is that it’s difficult to predict who will survive an infection without becoming a COVID-19 long hauler. Almost 30% of people who’ve survived COVID-19 still experience long-term side-effects!
One participant commented that the tone of the response sounded “judgmental.” Similarly, a young woman said “it was like the most convincing argument as to why you would want to get the vaccine because like it shows how many people get long-term side effects, but I did agree that the tone...was a little condescending” (Participant 20).
Almost unanimously, participants disapproved of the humorous response shown and said it mocked people for asking questions. Both young people and advocates said it was “pushy,” “saucy,” and “condescending.” Moreover, participants said it did not fully answer the question or provide context to support statements.
After “comprehensiveness,” the code “credibility” was most likely to overlap with “transparency,” indicating the importance of directly answering a question without seeming to withhold information. Although both advocates and young people discussed wanting responses to be both direct and comprehensive, the participants did not agree about explicitly highlighted scientific uncertainty. For instance, one message said, “scientists aren’t totally sure” about whether vaccines stop transmission. Advocates said acknowledging ongoing studies was appropriate, since “we’re still learning about it,” in the words of an advocate (Participant 18). However, young participants disagreed, saying phrases acknowledging uncertainty were unsettling and did not promote vaccination, with a young participant stating “she [the chatbot] seemed super uncertain” or “it almost makes it seem like people should wait to see more studies [to get the vaccine]” (Participant 19).
Young participants considered the use of a testimonial-style quote attributed to a Harvard physician to be elitist. One male advocate responded by saying, “Why do I care? It’s throw[ing] that he just has a title at my face” (Participant 08). Advocates aged >30 years agreed that the testimonial was not helpful, citing the politicization of doctors and science and suggesting the chatbot display testimonials from frontline health workers, such as emergency medical technicians.
To triangulate and strengthen our qualitative analysis, we tallied the number of votes the participants cast indicating their preferences for the messages on each of the 7 cards shown. FGD participants voted for a preferred message a total of 84 times (some did not select a response for each card shown). Participants voted for empathetic-factual messages 40 (48%) times, over 50% more times than factual-only messages—which at 24 (29%) votes was the second most preferred message style. However, although young participants most often (51%, 31/61) selected the empathetic-factual messages presented on message cards, public health workers most often (38%, 8/21) selected a principled response; young people rarely (8%, 5/61) selected this style with a few exceptions described above. Participants infrequently (8%, 7/84) preferred rational arguments, and participants never selected the testimonial or humorous messages, although just 1 example of each were shown on the cards. Although these quantitative results are not statistically significant given the qualitative study design and small sample size, the overwhelming preference for empathetic-framed responses among young participants is notable.