This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.
Improving confidence in and uptake of COVID-19 vaccines and boosters among long-term care workers (LTCWs) is a crucial public health goal, given their role in the care of elderly people and people at risk. While difficult to reach with workplace communication interventions, most LTCWs regularly use social media and smartphones. Various social media interventions have improved attitudes and uptake for other vaccines and hold promise for the LTCW population.
We aimed to develop a curated social web application (interactive website) to increase COVID-19 vaccine confidence (a 3-arm randomized trial is underway).
Following user-centric design and participatory research approaches, we undertook the following 3 steps: (1) content identification, (2) platform development, and (3) community building. A LTCW and stakeholder advisory group provided iterative input. For content identification (step 1), we identified topics of concern about COVID-19 vaccines via desktop research (published literature, public opinion polls, and social media monitoring), refined by interviewing and polling LTCWs. We also conducted a national online panel survey. We curated and fact-checked posts from popular social media platforms that addressed the identified concerns. During platform development (step 2), we solicited preferences for design and functionality via interviews and user experience testing with LTCWs. We also identified best practices for online community building (step 3).
In the interviews (n=9), we identified 3 themes: (1) LTCWs are proud of their work but feel undervalued; (2) LTCWs have varying levels of trust in COVID-19–related information; and (3) LTCWs would welcome a curated COVID-19 resource that is easy to understand and use-"something for us". Through desktop research, LTCW interviews, and our national online panel survey (n=592) we found that participants are interested in information about COVID-19 in general, vaccine benefits, vaccine risks, and vaccine development. Content identification resulted in 434 posts addressing these topic areas, with 209 uploaded to the final web application. Our LTCW poll (n=8) revealed preferences for personal stories and video content. The platform we developed is an accessible WordPress-based social media web application, refined through formal (n=3) and informal user experience testing. Users can sort posts by topic or subtopic and react to or comment on posts. To build an online community, we recruited 3 LTCW “community ambassadors” and instructed them to encourage discussion, acknowledge concerns, and offer factual information on COVID-19 vaccines. We also set “community standards” for the web application.
An iterative, user-centric, participatory approach led to the launch of an accessible social media web application with curated content for COVID-19 vaccines targeting LTCWs in the United States. Through our trial, we will determine if this approach successfully improves vaccine confidence. If so, a similar social media resource could be used to develop curated social media interventions in other populations and with other public health goals.
COVID-19 vaccination rates among long-term care workers (LTCWs) vary across the United States, partly due to a patchwork of legal challenges to a Centers for Medicare and Medicaid Services vaccination mandate [
Long-term care settings have been major outbreak sites throughout the COVID-19 pandemic, leading to illness and death among vulnerable residents and staff. Researchers of the Centers for Disease Control and Prevention estimate that more than 2300 LTCWs and 151,000 residents have died from COVID-19. COVID-19 outbreaks threaten LTCWs themselves and their often underserved communities [
Because LTCWs care for the frailest elderly, increasing vaccine confidence and uptake in this population while simultaneously supporting and retaining the LTCW workforce is a critical public health need.
Although improving vaccine confidence and uptake among LTCWs is challenging, social media is a promising potential solution. Social media–based interventions can suit marginalized groups and LTCWs already relying on social media for information (personal communication, Matthew Cantrell, April 2021)[
We aimed to fill this gap by developing a curated social media web application for LTCWs with low confidence in COVID-19 vaccines.
We conducted a user-centered design and participatory research study to develop, qualitatively assess (usability and acceptability), and deploy a social media web application (interactive website) called the COVID-19 Social Site [
We reported results using the Consolidated Criteria for Reporting Qualitative Research (CORE-Q) and the Checklist for Reporting Results of Internet E-Surveys (CHERRIES) [
Our National Association of Health Care Assistants (NAHCA) partners were critical to web application development. Their deep expertise in long-term care informed development from conception to launch.
We recruited 10 LTCW partners from diverse backgrounds and positions as part of a stakeholder advisory group, which met regularly. Our partners were instrumental in designing and developing the web app, and providing iterative feedback and advice throughout all stages of the project. Their feedback is captured throughout this manuscript (
We conceived this intervention within a broader comparative effectiveness trial with 2 interventions guided by the theoretical framework by Peretti-Watel et al for vaccine hesitancy, which considers vaccine hesitancy a decision-making process, not a static state [
Dartmouth College’s Committee for the Protection of Human Subjects approved this study (STUDY00032340).
We first identified the common questions and concerns associated with low COVID-19 vaccine confidence by reviewing information from the published literature, public opinion polls, and social media. Given the evolving pandemic, we had a flexible search strategy (
We developed a dynamic list of questions and concerns by consulting with our team (including LTCW partners and other stakeholders) and cross-checking with existing resources [
We shared our top-level topics with our LTCW stakeholder advisors via semistructured interviews. We have provided details on the interview population, recruitment, procedures, and analysis (
We polled our LTCW partners to assess the content mix they wanted on the web application, including questions about the source (platform and creator), type, quantity, and tone of posts. Our questionnaire is presented in
We deployed a Qualtrics (Seattle, WA) survey (
We sourced material from popular social media web applications according to the preferences identified by LTCWs via interviews and a poll. We plan to continue sourcing content (
The content team identified social media posts of interest and logged them along with basic details (eg, date posted, the platform of origin, and engagement metrics). We used a category-based system with hashtags.
We designed a fact-checking process (
We presented our initial concept for the web application to our LTCW stakeholders, soliciting information about the desired look, feel, and functionality iteratively and during the semistructured interviews mentioned in Step 1. We have provided details on the methods in
We shared our initial concept informed by stakeholder insight with a web design and development company. They advised on the functionality, and look and feel of the web application. The web team developed initial wireframes and the preliminary site on WordPress, primarily using Blade, Javascript, Hypertext Preprocessor (PHP), and Sassy Cascading Style Sheets (SCSS) scripting languages. We iteratively modified and tested the site with the study team, LTCW partners, and other stakeholders.
After testing and modifying initial wireframes with LTCW partners and other stakeholders, we conducted one-on-one user experience (UX) and user interface (UI) testing sessions with LTCWs who were naive to the study and its goals. We also solicited UX and UI feedback from LTCW partners and other stakeholders during meetings.
During interviews and stakeholder meetings, we asked LTCWs how they wanted the web application to operate and what kind of moderation or guidance we should use. We reviewed the community standards of popular Facebook groups focusing on COVID-19 vaccine discussion. Further details on our approach to community building are provided in
LTCWs were integral to every stage of this web application development project. Our LTCW partners noted:
Certified nursing assistants are often overlooked and dismissed. We help individuals every day, but it is an honor to be given the opportunity to help so many people on such a large scale.
To have my opinion considered and appreciated gives me great satisfaction and encourages me to continue empowering my profession.
We grouped questions and concerns about COVID-19, and COVID-19 vaccines and boosters into the following 5 general topics: access and process, benefits and efficacy, side effects and harm, development process, and the pandemic overall.
We interviewed 9 LTCWs (5 certified nursing assistants, 1 food services worker, 1 activities director, 1 maintenance worker, and 1 other direct care worker) working in skilled nursing facilities, home care, and hospice care across various regions in the United States. The LTCWs interviewed included 5 females and 4 males. Four participants were White, 2 were Asian, 2 were Black, and 1 was biracial. All LTCWs indicated English as their preferred language, except for 1 who selected Cantonese. Interviews lasted approximately an hour and occurred over Zoom (Zoom Video Communications) between September 16 and November 5, 2021.
We found 3 major themes (
Long-term care worker interview themes. LTCW: long-term care workers.
All interviewees expressed pride and unique vocational identity as LTCWs. They spoke impassionedly, with phrases like
Where my heart is.
I don’t call it my passion, I don’t call it my calling; it’s my ministry.
A minority of participants noted frustration that LTCWs are generally unrecognized in favor of nursing or other health care workers.
LTCWs spontaneously cited their care for and connection to long-term care residents as central to their professional identities.
I love that it’s such a sense of family when we’re there.
This sense of responsibility was a powerful motivator for the LTCWs who decided to get COVID-19 vaccines.
In my line of work, I work with the demographic most at risk [so] I have a moral responsibility to other people.
The web of professional and emotional connections magnified the loss and trauma LTCWs experienced during the pandemic, with most noting death and illness among their residents, colleagues, families, and communities. One participant made the following statement:
Seeing these people that I work with – and I love – [...] in a very quick amount of time go from a healthy senior to gone was very devastating.
Another participant recounted their experience of near hospitalization due to COVID-19, which prompted them to get vaccinated.
During the COVID-19 pandemic, LTCWs received information from various sources, including their employers, families and friends, the government, and news and social media. Participants were more likely to trust their families and friends about COVID-19, and its vaccines and boosters than other sources. Sometimes these influences resulted in vaccination, and other times they increased hesitancy. One participant made the following statement:
[My husband] is the one that is hesitant [about] the vaccine, and I am with him. He is the one that influenced me not to get it right now.
LTCWs noted their unique access to up-to-date information, citing the medical directors at their facilities and industry leaders as strong influences. Interviewees viewed most other mainstream information sources, including the government and the news media, as unreliable. One participant made the following statement:
I am not a scientist or a doctor. It’s very hard to find information that’s accurate.
In part because of their mixed information sources and partly due to the contradictory nature of official COVID-19 messages, LTCWs have outstanding questions about COVID-19 vaccines. One participant made the following statement:
Even the CDC and the FDA [...] still have different voices about the vaccine.
Another encapsulated LTCWs’ concerns with the following statement:
It’s [...] still a trial. They do not know 100% the consequences, the side effects, long-term side effects, [...] the ingredients.
Of particular concern were vaccine effectiveness, the vaccine development process, and potential harms, including long-term effects (fertility and unknown future problems) and serious reactions (myocarditis and blood clots). One participant recalled a colleague who made the following statement:
Her only reason was because she didn’t want to end up on a commercial 10 years from now that said, “Have you developed this, this, and this and this because of the [...] COVID vaccine that was forced on you 10 years ago? You may be liable for a lawsuit [sic]”
Outstanding questions about COVID-19 and vaccines meant most LTCWs wanted a dedicated place where they could find trustworthy information.
Targeted at who you're trying to get
Knowing they could trust the information was critical.
Just not knowing how to find out, not knowing how to research, or not knowing how to look into the resources. I think it’s really important that be made easily and readily available to people.
LTCW participants said the best way to get curated and trustworthy information was through personal stories and plain clear communication. One participant made the following statement:
Having actual stories to all of this kind of stuff is going to probably be key. People need to connect with other people.
Clear simple communication is paramount, especially given the busy nature of LTCWs’ lives. One interviewee said they needed the following:
Solid information, but in a simpler way, like easy to understand.
Other participants preferred visual information.
I think pictures speak louder than words to a lot of people, especially with social media.
Concerning the web application content itself, our LTCW interviewees appreciated the 5 proposed topics but suggested we eliminate the access and process topic. It was clear to LTCWs where and how they could get vaccinated. Additionally, they emphasized the importance of including content about COVID-19 vaccine effectiveness, the development process, and the potential short- and long-term risks. They also worked with us to refine the topic names for clarity (
When asked about the desired characteristics of the COVID-19 Social Site’s content, we found that the LTCW stakeholders wanted overall diversity in post format. Infographics and text-based content were most and least favored, respectively. The group strongly preferred content from Facebook and YouTube, with TikTok as the least preferred. They favored posts from certified nursing assistants, scientists, and laypeople and disfavored content from journalists. The LTCW stakeholders slightly preferred serious content over lighthearted content. Finally, they slightly preferred evergreen content to content covering new developments. Additional details are provided in
A total of 592 participants comprised the final survey sample. Participant flow and characteristics are detailed in
A large proportion of participants rated each information topic as at least “a little” important (
Ranked importance of COVID-19 topics in a national online panel survey.
Topic rank | Ratings of “a little important” or more per topic category (averaged across items) | Topic category most frequently of top importance per respondent | ||
|
Category | Percentage | Category | Percentage |
1 | How well the vaccines work | 94% | How well the vaccines work | 46% |
2 | Overall COVID-19 impact | 92% | Overall COVID-19 impact | 37% |
3 | Vaccine benefits | 91% | Vaccine creation | 35% |
4 | Vaccine creation | 90% | Long-term vaccine problems | 34% |
5 | Short-term vaccine side effects | 89% | Short-term vaccine side effects | 33% |
6 | Long-term vaccine problems | 89% | Vaccine benefits | 24% |
By launch, we had identified 434 content items for the web application, with 209 items ultimately uploaded to the live site (
Most posts (n=125) addressed the benefits of getting vaccinated, such as preventing death and illness. About a third (n=75) of the content covered COVID-19 as a disease and the severity of the pandemic. Other posts were about vaccine development (n=39) and the potential risks associated with vaccination (n=35), including side effects and heart problems. Some posts addressed multiple categories. Finally, a minority (n=25) of posts were not specifically related to these topic categories. These posts were largely health care workers’ personal stories, discussions of how vaccine misinformation works in general, or memes from certified nursing assistants or LTCWs.
We worked to reflect the LTCWs’ content mix preferences and the national survey result topics in our final content set, although some preferences competed, including the preference for video but suspicion of sources on TikTok. Given that the concerns about TikTok were related to its perceived untrustworthiness, we worked to rigorously fact-check each post to mitigate this concern.
Content identification, fact-checking, and screening flow diagram.
Final content characteristics.
Characteristic | Value (N=209), n (%) | |
|
|
|
|
Video | 133 (63.6) |
|
Image | 65 (31.1) |
|
Text | 11 (5.3) |
|
|
|
|
56 (26.8) | |
|
TikTok | 54 (25.8) |
|
YouTube | 51 (24.4) |
|
32 (15.3) | |
|
8 (3.8) | |
|
6 (2.9) | |
|
Other | 2 (1.0) |
|
|
|
|
Medical expert | 87 (41.6) |
|
Journalist | 27 (12.9) |
|
Healthcare organization | 22 (10.5) |
|
Government | 20 (9.6) |
|
Layperson | 20 (9.6) |
|
Long-term care worker or certified nursing assistant | 18 (8.6) |
|
University or education organization | 14 (6.7) |
|
Study team | 1 (0.5) |
|
|
|
|
Vaccine benefits | 125 (59.8) |
|
About COVID-19 | 75 (35.9) |
|
Vaccine creation | 39 (18.7) |
|
Vaccine risks | 35 (16.7) |
|
General | 25 (12.0) |
aIndividual content items may address multiple topics.
We created a WordPress-based social media web application called the COVID-19 Social Site (
Users could react to posts with emojis (labeled Like, Love, Haha, Wow, Sad, and Angry). They could also comment on posts and reply or react to other users’ comments. A notification bell alerted users to new activity, and we sent them email updates. We optimized the web application for desktop and mobile use. Interactivity was consistent with other popular social media web applications [
Additionally, we tracked user interaction across the social web application using Google Analytics. We informed users of the data collection via cookies with a pop-up dialog box that appeared on their first visit to the site. These web analytics services allowed us to determine which topics or posts received the most views, measure overall user engagement, and troubleshoot navigational or technical issues.
COVID-19 Social Site login and home page.
LTCW stakeholder consultation participant characteristics are detailed in Step 1. The subthemes that emerged from our interviews with LTCWs related to platform development were that (1) the web application needed to be accessible and easy to use, and that (2) LTCWs should have a safe space to interact with each other, particularly through comments and likes.
Although the user habits and levels of digital literacy of the LTCWs and their peers varied, almost all interviewees wanted a clearly laid out web application with simple navigation. We reflected these preferences in the site’s final design, prioritizing ease of use.
LTCWs wanted the web application to facilitate engagement with other LTCWs via comments. A few participants expressed concerns about the possibility of interpersonal conflict or vaccine-related misinformation but noted that careful moderation by the study team could mitigate this risk. The LTCWs also wanted to interact with posts and comments with likes, emojis, or similar. Multiple interviewees shared that while browsing established social media platforms like Facebook, they were most likely to view and actively engage with posts that had many reactions and comments.
We conducted 3 formal UX interviews with LTCWs and various informal UX tests with study team members and stakeholders [
We found that users easily recognized the web application as a social media platform and understood the layout. All interviewees agreed that the language and topics featured were valuable. We uncovered various opportunities to decrease user pain points by improving the navigation experience. For instance, we added an instructional video and cues when hovering over the main navigation menus to encourage clicking. Additional details are provided in
Our final community standards (
Given the emphasis on a safe space specifically for LTCWs, we decided LTCWs should play a visible role in the web application instead of the research staff, who were present behind the scenes but not visible. Through stakeholder connections, we recruited 3 students as “community ambassadors.” All 3 had long-term care experience and were training to become nursing home administrators.
The ambassadors were special users and community members, empowered by the study team. They used the site freely, commenting and reacting based on their views and positionality as LTCWs.
We asked the community ambassadors to report any comments of concern for individual review by moderators. This approach allowed the site to function as a partially self-moderating community, as regular users could also flag comments for review.
In addition to monitoring the site for community standard violations, we instructed the community ambassadors to encourage participant discussion by actively engaging with other users’ comments. We also asked the community ambassadors to respond to questions and concerns using information relating to COVID-19, the vaccines, and the boosters vetted by the study team (
Additional details on the community standards and the community ambassadors are outlined in
We successfully co-developed a novel social media web application featuring curated content specifically tailored to LTCWs with low vaccine confidence. LTCWs are a difficult-to-reach population who we were able to engage through participatory research and user-centered design. Although LTCWs report feeling overlooked professionally, they enthusiastically participated as partners in our project. Testing of the web application in a randomized trial is ongoing.
Through formative semistructured interviews, we found that LTCWs are proud of their work but feel undervalued, particularly in light of the COVID-19 pandemic. They often distrust official sources of information, including the government and popular media. Due to this combination of professional identity and distrust, they welcomed a social media platform specifically customized to them that is trustworthy and easy to understand and use.
To our knowledge, this is the first co-developed web application using curated content from social media to influence attitudes and behaviors about COVID-19 vaccines and boosters. It is also the first web application of its kind specifically for LTCWs, a critically important but understudied population.
We developed the web application in response to the COVID-19 pandemic, a rapidly evolving and urgent public health crisis. Web application development, testing, and launch occurred on an expedited timeline. Under more favorable conditions, we would have conducted more extensive user testing before the trial launch.
A key limitation of our qualitative stakeholder work is that we conducted most of our formative interviews with LTCWs who were already engaged in the project and vaccinated. This sample may have affected the responses. Additionally, LTCWs who agreed to become partners in a vaccine confidence project may be meaningfully different from other LTCWs. While most of our stakeholders were vaccinated, they still had questions and concerns about COVID-19 vaccines and boosters.
Although we designed our national online panel survey to include individuals who were demographically representative of LTCWs, the information preferences of LTCWs may be inherently different from those of the survey participants. Additionally, our content mix poll included a small sample, limiting its representativeness.
Our social media web application, siloed in its own space online, is materially different from most other social media as it exists separately from the rest of the information ecosystem. This separation was necessary to isolate the effects of our intervention and prevent contamination in a larger randomized trial.
Through our user-centered design and participatory research approach, our intervention evolved to reflect other successful interventions in this space more closely, namely a social media web application trialed by Glanz et al that improved vaccine uptake among children of participating parents [
Other research teams have successfully developed patient-facing communication interventions using participatory research methods [
Concerning the qualitative experiences of LTCWs overall, our findings that they feel underappreciated professionally are consistent with the findings in the literature, including research that has emerged during the COVID-19 pandemic [
It is not yet certain that the COVID-19 Social Site will positively influence vaccine confidence or uptake. The results of our randomized trial will be available in 2024.
If successful, our intervention could become a template for other populations with low COVID-19 vaccine confidence or uptake. Additionally, this approach could suit different contexts where changing attitudes or behaviors could be beneficial for public health. Our web application is a light-touch, low-cost intervention that may be relatively easy to replicate and implement by governments, employers, and others. Additionally, the resources associated with launching web applications such as these could be scaled up or down. Without the constraints of a randomized trial, we could also replicate elements of this approach using established social media platforms.
With user-centered design and participatory research, we developed a novel social media web application featuring curated internet content specifically for LTCWs.
Long-term care worker interview select quotes.
Content identification methods and results.
Long-term care worker interview methods and results.
Long-term care worker interview guide.
Long-term care worker content mix poll questionnaire.
National online panel survey questionnaire.
National online panel survey methods and results.
Fact-checking methods.
User experience testing methods and results.
Community building methods and results.
Abridged COVID-19 fact-checking list.
long-term care worker
user interface
user experience
Research reported in this manuscript was funded through a Patient-Centered Outcomes Research Institute (PCORI) Award (COVID-2021C2-13181). The statements presented in this manuscript are solely the responsibility of the authors and do not necessarily represent the views of the PCORI, or its Board of Governors or Methodology Committee. We acknowledge the broader CONFIDENT study team’s support and advice, particularly our long-term care worker partners. We thank the following collaborators: Feng Chen, Patrick Delva, Jon Edwards, Branden Fillbrook, Tammy McIlnay, Katie Page, Lupita Rodriguez, Rowena Sheppard, Mary White and Celeste Wooten.
Conceptualization: CHS, GS, GE, MC, and MAD. Methodology: CHS, GS, GE, and MAD. Software: CM and AJ. Formal analysis: CHS, AS, JP, JH, and MAD. Resources: GE and MAD. Data curation: AS. Writing-original draft: CHS and AS. Writing-review and editing: all authors. Supervision: LJ, GS, GE, and MAD. Funding acquisition: GE and MAD.
CHS holds copyright for the consideRATE suite of tools for serious illness. GE has edited and published books that provide royalties (Shared Decision Making, Oxford University Press and Groups, Radcliffe Press). GE’s academic interests are focused on shared decision making and coproduction. He owns copyright in measures of shared decision making (collaboRATE) and care integration (integRATE), a measure of experience of care in serious illness (consideRATE), a measure of goal setting coopeRATE, a measure of clinician willingness to do shared decision making (incorpoRATE), and observer measures of shared decision making (Observer OPTION-5 and Observer OPTION-12). He is the Founder and Director of &think LLC, which owns the registered trademark for Option Grid patient decision aids. He is the Chief Clinical Research Scientist to abridge AI Inc. MAD has contributed to the development of Option Grid patient decision aids, and EBSCO Information Services sells subscription access to Option Grid patient decision aids. She receives consulting income from EBSCO Health and royalties. All other authors report no relevant conflicts.