K Health is a novel AI-driven digital health app; at the time this paper was written, it was available for download in the United States, Mexico, Indonesia, and Israel [14]. The app was built using a data-driven approach in collaboration with Maccabi Health Services (MHS). MHS is the second largest health maintenance organization in Israel, with over 2 million members [15,16]. Data from the MHS health records were used to develop an AI-driven symptom-checker, whose methodology has been described elsewhere [14]. The K Health app was launched in the United States in 2018 and has been used by over 4 million adults [14]. The app is free to download, and the symptom checker is available for free to adults over 18 years of age; thus, the app provides the public with reliable and personalized information on diagnoses related to their symptoms and medical history.

In June 2019, K Health began offering a service that enables users to consult directly with a remote physician. The platform provides the users with the option to receive a diagnosis, prescriptions, lab referrals, a referral to remote management of mental health concerns, or a referral to primary or emergency care. Consulting with a remote physician involves a fee or enrollment in a relevant health plan.

Early in April 2020, a protocol-based COVID-19 self-assessment was added to the app to provide users with up-to-date guidance for managing suspected cases of COVID-19. The self-assessment was developed by a team of board-certified physicians and was based on guidance issued by the World Health Organization and the US Centers for Disease Control and Prevention [17,18]. The user was asked to report the presence and severity of COVID-19–related symptoms as well as the presence of concomitant conditions known to increase risk of severe disease. Following the self-assessment, users were provided with one of four recommended actions based on their risk profile: social distancing, quarantine, isolation, or seeking immediate medical evaluation. Users were also informed if they were at increased risk for COVID-19 complications, and users with risk factors and symptoms were encouraged to consult a physician. For further details, including the full questionnaire, see Multimedia Appendix 1.

This study describes use of three digital tools, namely a protocol-based COVID-19 self-assessment tool, an AI-driven symptom checker, and communication with a remote physician, by adults (>18 years of age) seeking COVID-19–related health information and services during the 10-week period between the launch date of April 8 to the date of data extraction on June 20, 2020. Seeking COVID-19–related health information was defined as use of the COVID-19 self-assessment tool. This self-assessment tool can be activated by the user, and it is also activated automatically when users begin a symptom check with a complaint of cough, dyspnea, or fever. The study did not include repeated self-assessments.

After users perform the COVID-19 self-assessment, they are prompted to use the AI-driven symptom checker to receive additional information about other conditions that may be related to their symptoms. After receiving the results and information regarding these conditions, the users can choose to communicate with a remote physician for definitive medical management.

This study analyzed deidentified data. The protocol of the study was reviewed by the Western Institutional Review Board and qualified for exemption in accordance with 45 CFR § 46.104(d)(4). K Health is compliant with the Health Insurance Portability and Accountability Act (HIPAA) and the General Data Protection Regulation (GDPR). Encrypted transportation and storage were used at all stages of data management.

Deidentified data on the characteristics of the digital interactions in the app were collected, including user-inputted demographic and clinical characteristics; COVID-19 exposure reporting; reports of COVID-19 testing, including results; and symptoms. We also collected the outputs provided by the three digital modalities: risk categories according to the protocol-based COVID-19 self-assessment; the most common conditions for similar people and symptoms (“potential diagnoses”) provided by the AI-driven symptom checker; and diagnoses, management, and disposition by remote physicians.

User characteristics included age, sex, risk factors, and comorbidities. Comorbidities included hypertension, smoking, obesity, diabetes (type 1 and type 2), cardiovascular disease, chronic lung disease (asthma, chronic obstructive pulmonary disorder, and interstitial lung disease), chronic renal and liver disease, cancer, and immune suppression.

Symptoms were obtained from two types of interactions on the platform: first, as part of the structured protocol-based COVID-19 risk assessment, and second, as part of the dynamic AI-driven health dialog.

The COVID-19 self-assessment included questions regarding symptoms known to be associated with COVID-19 (eg, cough, dyspnea, and fever) as well as symptoms hypothesized to be related to COVID-19 (eg, eye symptoms), which were inquired about for further research and development of the COVID-19 protocol. The symptom checker dialog allows the user to spontaneously discuss a wider array of 331 symptoms; it also queries the user regarding additional symptoms and symptom-specific attributes, such as the severity, timing, duration, and quality of a symptom (eg, dry or productive cough), as previously described [6].

The analyses assessed the user characteristics, reported symptoms, and output of the aforementioned digital services, namely the self-assessment protocol, AI- and data-driven symptom checker, and remote physician consultation. This assessment was performed using descriptive statistics and bivariate analyses.

Descriptive univariate summary statistics were developed to assess the user characteristics, reported symptoms, diagnoses, and disposition. Symptoms were assessed for both the COVID-19 self-assessment and the symptom checker. Bivariate analyses were conducted for between-group comparisons of characteristics and symptoms. These included two primary analyses.

The first analysis explored the potential utility of self-reported data from digital health tools to identify symptoms associated with COVID-19. This analysis compared the self-reported symptoms by digital health users who reported testing positive for COVID-19 with those of users who reported testing negative. To ensure that the symptoms of cough, fever, and dyspnea were optimally captured, the presence of these symptoms was collected from both the self-assessment and the symptom checker dialog. More details are provided in Multimedia Appendix 1.

The second analysis aimed to identify predictors related to the choice to consult with a remote physician. This analysis compared data provided by individuals who opted to consult with a remote physician with those provided by the entire cohort. Specifically, this comparison aimed to evaluate whether consultation with a physician was related to risk of COVID-19, symptom severity, or comorbidity.

Appropriate statistical tests were used to assess the significance of between-group differences: chi-square tests for differences in proportions and t tests for differences in continuous measures. Test assumptions were assessed analytically and graphically and were judged to be adequately met. We considered P values of .05 or less to be significant and did not correct for multiple comparisons. Analyses were conducted using Python version 3.6.9.

This study describes the characteristics and management of >71,000 individuals who used digital tools to seek health information and services related to COVID-19. These individuals were relatively young and predominantly female. Users received information regarding COVID-19 risk and management, received AI-driven information about other relevant diagnoses, and consulted with remote physicians. Users who chose to communicate with a remote physician were more likely to have been classified as requiring immediate medical evaluation by the COVID-19 self-assessment. Correspondingly, these users were also older, more likely to report severe symptoms, and more likely to report characteristics cognate with risk of COVID-19 (such as known exposure to COVID-19 and loss of sense of smell or taste). The majority of consultations with a remote physician (1940/2523, 76.9%) were resolved without need for referral to an in-person health visit or to the emergency department.

Taken together, the differential communication with remote physicians according to symptom severity and the high resolution rates without need for referral to in-person visits suggest that the digital tools provided information and advice that assisted users in making health decisions. These tools may therefore reduce the burden on health care systems during times when resources are limited and may help minimize unnecessary physical interactions that could lead to iatrogenic COVID-19 exposure. However, research on health care use and health outcomes following digital health tool use is needed to conclusively demonstrate this potential.

In addition, individuals who chose to communicate with a remote physician tended to have lower rates of comorbidity. This suggests that these individuals were more likely to have an established relationship with a health care provider whom they consulted regularly and were more likely to require an in-person evaluation and more complex care when presenting with significant symptoms.

This study also highlighted differences in self-reported symptoms between users who reported testing positive for COVID-19 and those who reported testing negative. The symptom most strongly associated with positive testing for COVID-19 was loss of sense of taste or smell (RR 6.66, 95% CI 5.53-7.93). Surprisingly, travel to an area with COVID-19 appeared to be associated with a lower rate of positive testing (RR 0.47, 95% CI 0.23-0.90); however. this estimate is based on a relatively small number of subjects and should therefore be interpreted with caution. As the decision to test or be tested is driven by the presence of risk factors for COVID-19, travel to an area with COVID-19 may appear protective because individuals who were tested due to this risk factor may be less likely to have other stronger predictors of positivity, such as exposure to an individual with COVID-19 or the presence of COVID-19 symptoms. Additional features associated with COVID-19 included other widely recognized COVID-19 symptoms, such as fever and cough. These results emphasize the potential utility of taste and smell as strong signals of COVID-19 positivity in the community setting, and they are similar to results reported in recent studies of self-reported symptoms among individuals who tested positive for COVID-19 [19,20].

Finally, in our study, we found that dry eye (RR 1.87, 95% CI 1.57-2.22) and eye pain (RR 1.61, 95% CI 1.29-2.01) were more common among individuals who reported testing positive for COVID-19. These symptoms were added to the self-assessment to explore the possible link between COVID-19 and eye symptoms as well as the utility of these symptoms in evaluating suspicion of COVID-19. Although some early hospital-based studies reported low rates of ocular symptoms in patients with COVID-19 [20], several other studies have suggested that ocular symptoms are common among individuals with COVID-19 [21-23]. Ocular symptoms have been documented in some cases as the first [24] and even only [25] symptomatic manifestation of COVID-19, and studies have documented positive reverse transcriptase–polymerase chain reaction COVID-19 test results from ocular secretions [23,26]. This study adds to the body of evidence suggesting that the manifestations of COVID-19 include ocular symptoms and that they may be more common symptoms of COVID-19 than generally recognized.

Seen from a wider perspective, these results demonstrate that self-reported symptoms on a digital app can replicate symptoms known to be associated with COVID-19, that they can help distinguish between individuals who test positive and negative for COVID-19, and that they may add to our understanding of symptoms associated with COVID-19. This exemplifies the potential of data generated from digital tools to improve our understanding of the clinical manifestations of COVID-19 and of patient-reported experiences in general.

The potential benefits of digital tools during the COVID-19 epidemic have been noted in multiple health policy commentaries [1-3,19,27]. This stance has been adopted by several leading medical associations, including the American Medical Association and the American Academy of Family Physicians [4,5]. However, there has been limited research to date on the actual usage patterns and impact of these tools during the COVID-19 epidemic. The body of literature on digital health for COVID-19 primarily features perspectives and opinion pieces, guidance papers, and a few studies. These studies tend to be small, focus on a single digital tool, and/or report primarily on survey results of user satisfaction. Two larger studies have provided some additional insight. The first study reported the results of a satisfaction survey of 6194 people following wide-scale deployment of digital tools for COVID-19 education, self-assessment, and symptom monitoring in the Netherlands; high satisfaction rates were reported [11]. The second study reported on the treatment and high satisfaction rates of 4589 patients in China using web-based physician consultation for COVID-19 concerns [13].

The demographic characteristics of digital health users described in this study match previous reports on digital health users. While the use of digital health technologies among seniors has been reported to be increasing [28], users of digital health applications are still predominantly younger adults [29]. The current low level of adoption of digital health by older adults is unfortunate, as digital health technologies have the potential to improve communication and collaboration and promote healthy and independent ageing. Indeed, recent research suggests that digital solutions tailored for older adults improve health management [30]. Barriers to the adoption of digital health by older adults include visual impairment, limitations in dexterity, and lack of self-confidence when using technology. Several solutions are being developed to circumvent these barriers, including the development of voice-based applications and unobtrusive sensors and trackers [31].

This study has a number of strengths. First, it uses a large sample of over 71,000 individuals to provide timely information on the use of a number of different digital health tools for managing COVID-19–related concerns. Second, the study reported on the differential use of these tools by these individuals as well as on both self-reported variables and physician-reported disposition and management. Third, the study was able to provide insight into self-reported symptoms of individuals who were tested for COVID-19; the results highlighted the strong link between COVID-19 and loss of sense of smell and taste and added to the body of evidence that ocular symptoms may be a more common feature of COVID-19 than is widely recognized.

The study has a number of limitations as well. The population of this study is not representative of the entire population, and the study uses data from digital tools developed by a single provider. However, the characteristics of the study population and the COVID-19 symptoms reported correspond with those in previous reports on digital health tools [11,20-22]. In addition, data on COVID-19 test status were based on self-reporting and may not accurately capture test results. Furthermore, the data are limited to individuals who used a tool for initial COVID-19 assessment. Tools enabling longitudinal logging and monitoring of symptoms can both provide additional utility to users and improve our understanding of disease progression and the time course of symptoms. Enabling users to provide unsolicited data on their experience may provide important insight as well, as the self-assessment tool described focused on a predefined list of questions. Lastly, while we had data on the disposition of users who chose to consult with a remote physician, the disposition of individuals who did not consult with a remote physician following use of the automated tools is unknown. Research on health care use and health outcomes of digital health users is needed to conclusively demonstrate the utility of these services in assisting individuals with health decision-making and reducing the burden on the health system.

This study describes the integration of three digital health tools for the direct management of COVID-19–related concerns. The results suggest that automated, data-driven digital solutions as well as remote care provided by a human physician can help provide health information and guidance during an epidemic. In addition, interactions across digital services can provide insight regarding the characteristics of new diseases. The integration of these tools can be an important resource for health care providers and policy makers.