In April 2020, a quaternary academic medical center, Stanford Health Care (SHC, Palo Alto, California, United States) in partnership with a large technology company (Apple, Cupertino, California, United States) launched the Stanford Medicine COVID-19 Guide for First Responders and Essential Workers app (COVID-19 Guide App) with the goal of providing evidence-based information related to COVID-19, screening guidance, and access to testing services to support 1.1 million first responders and essential workers in the San Francisco Bay Area [38-40]. The dissemination effort by the institution included emails to local first responder and essential worker representatives, facilitated in part through previously established partnerships between SHC emergency physicians and emergency medical services (EMS) and fire organizations within the community developed through the institution’s EMS Fellowship. This study was conducted by an interdisciplinary evaluation group led by the Stanford Medicine Evaluation Sciences Unit. First responder interview participants were recruited from Bay Area county police, fire, and EMS departments and among physicians connected to the health system’s EMS physician fellowship. Essential worker interview participants were recruited from local grocery stores, gas stations, care facilities, and civil infrastructure support businesses.

The COVID-19 Guide App includes a screening questionnaire that provides testing recommendations with links to access further testing and care, COVID-19 resources from Stanford Health Care physicians and scientists, and foundational information on viruses and prevention (Figure 1) [38].

First responder participants were drawn from police, fire, and EMS departments in Alameda, San Mateo, San Francisco, and Santa Clara counties through email and telephone call outreach to local departments. All fire personnel are cross-trained in EMS, although not all EMS personnel have fire training [41]; participants from both groups were recruited. Emergency medicine physicians who partnered with local fire and police departments through the EMS Fellowship [42] at Stanford School of Medicine were also interviewed using a convenience sampling approach. Snowball sampling was leveraged across all categories to connect with additional participants in harder-to-reach groups. Interviews were conducted until thematic saturation within each professional subgroup was reached.

Essential workers were recruited using a convenience sampling approach in which researchers sought interviews in person at local businesses. When a paucity of Spanish-speaking essential workers was recognized, we shifted to a telephone-based convenience sampling approach targeted at this population to better reflect demographics of the region [43] until thematic saturation for both English- and Spanish-speaking essential workers was reached. Pre-existing experience with the app was not a requirement for participation, and the interview protocol itself accommodated both prior experience and no prior experience with the app (Multimedia Appendix 1).

From May to September 2020, 30-minute semistructured interviews using a protocol grounded in the Consolidated Framework for Implementation Research were conducted by multiple authors (SV, NS, AA, MF, CBJ) (Multimedia Appendix 1) [44]. Interviews were conducted with two or more researchers present, with rare exceptions for note-taking, training, and analytic purposes. First responder interviews were conducted by telephone and videoconference; essential worker interviews were conducted by telephone and in person. For Spanish speakers, a professional interpreter was enlisted to interpret interviewer questions and interviewee responses. First responder interviews were tracked confidentially, recorded, and transcribed verbatim using Rev [45]. Essential worker interviews were not recorded to protect the anonymity of these more vulnerable interviewees in their work-based settings. For these interviews, near-verbatim field notes were taken, and two or more researchers who were present at each interview contributed to the field notes in a debrief discussion immediately following the interview.

Descriptive quantitative data, including the total number of downloads at the conclusion of the evaluation on September 1, 2020, were drawn from the app administrator account. User characteristics, including demographic and use data, were not available due to restrictions in data use agreements.

Interview transcripts were analyzed using an inductive rapid analytic process in which two researchers coded all interviews (SV, AA) [46,47]. We used an inductive-deductive approach to identify a priori themes related to access to information and testing services and emergent themes. Excerpts were presented in a matrix format following rapid analytic procedure methodology [48]. The matrix was populated with notes and quotations (AA) and reviewed by qualitative team members (SV, AA, MF) to reach consensus and compare themes across worker subgroups [49]. The Implementation Outcomes Framework guided the evaluation and presentation of implementation outcomes (appropriateness, acceptability, and adoption) [50].

Throughout the assessment, early findings were reported back to operational leads to inform ongoing improvement using the Lightning Report Method described in 2019 by Brown-Johnson et al [51]. This project was reviewed by the Stanford Institutional Review Board and did not qualify as human subjects research (Protocol ID 56126). All participants gave verbal consent to be interviewed, and first responders gave consent to be audio-recorded.

A total of 27 interviews were conducted with 17 first responders (63%) and 10 essential workers (37%) (Table 1). The number of participants within each subcategory was comparable, including 5 police (29%), 7 emergency medical technician (EMT) (41%), 4 EMT with fire training (EMT/Fire, 59%), and 5 physician police/EMS advisor (PoliceMD/FireMD, 29%) interviewees in the group of 17 first responders and 5 food and agriculture (50%) and 5 transportation and civil infrastructure (50%) interviewees in the group of 10 essential workers. The food and agriculture subgroup included 4 grocery workers and 1 dining supervisor, while the transportation and civil infrastructure subgroup included 3 gas attendants, 1 bus driver, and 1 electrician. First responders tended to be male (16/17, 88%), 30-49 years old (11/17, 65%), and White (10/17, 59%), and most reported working in their field for more than 10 years (10/17, 59%). The essential workers were 50% male (5/10), of varying ages, mostly Hispanic/Latino (7/10, 70%), and had worked in the field for either less than 5 years (4/10, 40%) or more than 10 years (5/10, 50). The primary language of half of the essential workers (5/10, 50%) was Spanish, whereas all first responder interviews were conducted in English.

Implementation outcomes of the COVID-19 Guide App differed between the first responder and essential worker groups. Perspectives on the app varied by group, with moderate to low adoption, mixed appropriateness, and mixed acceptability seen in first responders, and low adoption, high appropriateness, and high acceptability seen in essential workers.

First responders and essential workers experienced a number of related challenges, described in the following qualitative themes: a need for personalized information with no clear playbook, parsing information takes time, misinformed beliefs fuel conflict, priority testing gets them back to work, fear of bringing an amorphous enemy home, and a lack of resources to fight the pandemic. First responders faced additional challenges related to the need for interprofessional coordination and a double-edged “culture of heroism” that could both protect against and exacerbate mental and health vulnerabilities (Table 2).

Participants provided several solutions to address these challenges, some of which they thought could be delivered through an app (Table 3). To address misinformation and fear of SARS-CoV-2, several participants validated the potential benefit of succinct, accurate information delivered through an app. Others thought that sharing accurate information through social media would be most effective given the increased visibility of these platforms among essential workers. Yet others felt the need for “higher touch” strategies, possibly including holding local question and answer (Q&A) sessions (FireMD 2). The personalization of information in any solution was a recurrent theme:

In the absence of precision information, many solutions were developed in the field: “Some of the crews just kind of came up with stuff on their own” (FireMD 4). Other essential workers found support in managing conflict with the public from their employers (Grocery 4) or through their own volition. One bus driver reported company instructions to call a hotline when riders were not compliant with wearing masks, but she instead refused to drive the bus: “I use my [face covering gesture] until they put it on” (Bus Driver 5).

Expanding the physician-first responder partnerships to a broader audience, particularly to the police, where only informal relationships existed, was suggested. A specific request was for a designated number or person within the health system to call when the COVID-19 status of a transported patient was under question.

Personalized information also encompassed health system–specific COVID-19–related information. Some first responders expressed a desire for health systems to be forthcoming about their own infection control practices and current rates: “That’s where you just put the notion of, ‘Hey, we’re [health system] safe…You can transport patients to our hospitals and we won’t get you contracted’” (EMT 4). First responders also wanted detailed, localized COVID-19–positivity data, expressing a desire to better understand the geographic spread of cases throughout the county (Police 4, EMT/Fire 2, 7).

A number of participants desired a stronger connection to a health worker during COVID-19. The physician partners themselves felt they were often the best resource to support first responders during the pandemic: “we are the best conduit [through] the already established trust that we've built up through the years” (FireMD 1). These participants felt their role encompassed the development and maintenance of long-term, trusting relationships with these first responder communities (FireMD 1, PoliceMD 5). First responders corroborated the special nature of the relationship in times of crisis, as one physician partner was described as “a really good resource. He's been coming by the station now too, to talk with us and to check in on COVID related topics, as well as other [topics]” (EMT/Fire 7).

In addition to their formal physician partnerships, participants also received information through informal networks into the health care community, such as a former employee who worked in the health system at the time of the interview (Grocery 3). Those with shift-based work welcomed the idea of a health worker presence at the beginning of their shift: “That may be an opportunity…to have someone present during those briefings to talk on any new developments or anything COVID related” (Police 5, Grocery 2). The human, in-person element of this relationship was emphasized.

Digital initiatives such as the Stanford COVID-19 Guide App have the potential to equip first responders and other essential workers with accurate information and access to testing resources, although adoption may depend on targeted publicity and early involvement of stakeholders with pre-existing relationships within these communities. While first responders are often included under the broader category of essential workers during outreach efforts, our findings suggest they face both overlapping challenges—related to the need for personalized and accurate information, access to testing, and ongoing personal safety—and divergent challenges, such as the need for interprofessional coordination and a double-edged culture of heroism, seen in the first responder sample alone. Optimal strategies for supporting each group should therefore differ based on their divergent needs, channels for outreach, appetite for digital and relationship-based approaches, and relationship to the health system (Table 4).

Physician partnerships emerged as a key resource for first responder departments, and these relationships are recognized as a foundational underpinning of a pandemic response [52-54]. Formalizing and nourishing these relationships holds promise in strengthening the overall ability of the health system to support its community during a pandemic. Interprofessional coordination was also cited as a major challenge, hampered by interoperability challenges between the electronic health records of EMS and the receiving health system [55]. Given these challenges, a trusted point of contact (eg, emergency department nurse line) between EMS and the health system may be explored as an alternate solution in a pandemic setting.

Our findings also suggest that first responders see increased risk as an integral part of assuming their role and may therefore adapt more readily to a new pandemic. Conversely, non–first responder essential workers decidedly do not see their roles as inherently risky outside a pandemic scenario. Without the training and cultural reinforcement around managing increased risk as a part of standard work, essential workers may suffer disproportionately from the onset of an infectious disease pandemic. The contributions of these workers during a pandemic should therefore not be diminished, and prioritized health services and hazard pay may be justified [56].

Similarly, misinformation emerged as a widespread challenge among the broader community. Although first responders have at least a basic degree of medical training and serve as a source of information to laypersons during a pandemic [57], essential workers often lack this formal training. Health literacy, defined as the ability to acquire, understand and enlist health information [58] becomes essential during a pandemic as first responders and essential workers come into regular, direct contact with members of the community. The reported reliance of essential workers on social media for information is concerning given its link to the spread of unvetted information [59-61]. Efforts to build health literacy, not only through digital interventions but also through channels with widespread use among vulnerable populations, including essential workers and non-English speakers, are urgently needed.

We note that the early learnings described here must be interpreted with the context that the COVID Guide App was rapidly deployed in just under three months after the first reported COVID-19 case in the United States [62]. Balancing the accuracy of an intervention in achieving its aim with the goal of rapid deployment will no doubt remain a challenge during this pandemic and other disasters; we recommend a standing advisory board with representation from target populations that can be mobilized in the short term to inform intervention development. In the long-term, community-based participatory research and human-centered design are parallel and overlapping methodologies that emphasize user expertise and creative problem-solving, with documented success in vulnerable populations [63-65]. These tools may inform improved evolutions of community interventions.

Limitations of this study include a lack of systematic sampling across professions in the non–first responder essential worker population, which resulted from challenges accessing this group in the field. We are unaware of similar work reaching this vulnerable population, which suggests this sample is still of value. Further, Spanish-speakers predominated our non–English-speaking sample, which reflects local demographics but does not account for the full variety of languages spoken in the community [43]. Furthermore, qualitative data capture differed between the first responders (verbatim transcriptions) and essential workers (near-verbatim field notes). This decision was made to protect essential worker anonymity given their potentially vulnerable position near their place of work. Efforts to mitigate data loss from an absence of recordings included a dedicated note-taker and immediate debrief discussions between two or more researchers to expand data capture. Finally, our learnings are limited to our local community, as the Bay Area may not be representative of populations and resources available elsewhere.

Future work will benefit from increased attention to the variety of first languages that exist within the community, including non-Spanish languages. The incorporation of a mental health component to this digital solution, reported as a need here and evaluated in other settings [66,67], may also be an opportunity for future work. Finally, evaluating digital support systems for EMS personnel as they face an increased number of deaths in the field as a result of the pandemic [68,69]—supporting decisions related to triage and/or end of life care—is another important area for future research.

First responders and essential workers face shared challenges related to obtaining accurate information and testing services, and securing their personal safety. Digital interventions such as mobile applications have the potential to combat these challenges through the provision disease-specific information and access to testing services. Such solutions are likely to be most effective if delivered as part of a larger ecosystem of support, and with early and direct input from those in these professions to understand how best to meet their specific needs. Given varying challenges between first responders and non-first responder essential workers, our results indicate that differentiated interventions may leverage shared insights while also acknowledging differences in their occupational requirements and culture.