To assess the potential of an AI-based sentiment analysis to understand public views and concerns, we analyzed data from two popular social media platforms—Facebook and Twitter. Facebook posts were extracted using the CrowdTangle platform [12], and Twitter posts were extracted from the COVID-19 Twitter chatter dataset, constructed by Panacea Lab (using a publicly available Twitter application programming interface) [13]. English-language Facebook posts and tweets posted in the United Kingdom from March 1 to October 31, 2020, were extracted, and thematically filtered using a two-step process, for keywords related to both COVID-19 and contact tracing apps. The Twitter dataset was already filtered with predefined COVID-19–related keywords as detailed previously [13], and the same keywords were used to apply an initial filter to the Facebook data. The following contract tracing app–related keywords for the second step filtering were selected by our interdisciplinary team: “covid app,” “tracing app,” “contact tracing,” “privacy,” “security,” “app security,” “app privacy,” “contain virus spread,” “movement tracking,” “tracking,” and “surveillance.” Following geographical filtering for the United Kingdom, a total of 2000 tweets and 8000 Facebook posts were obtained for the analysis.

A novel ensemble-based AI model developed by the authors in a recent study [14] was adapted and utilized for this study, by combining lexicon rule-based and deep learning (DL)–based approaches. Specifically, an average-weighting ensemble [15] of lexicon-based models, including Valence Aware Dictionary and Sentiment Reasoner (VADER) [16] and TextBlob [17], was combined with a state-of-the-art DL-based model Bidirectional Encoder Representations from Transformers (BERT) [18]. The overall hybrid rule-based ensemble method is illustrated in Figure 1.

The hybrid rule-based ensemble model was hyperparameter tuned based on manual validation by using 1000 randomly selected posts and labelled by the team. Specificity and sensitivity analyses showed that the lexicon-based model demonstrated better performance accuracy for positive sentiment and that the DL-based BERT model demonstrated enhanced accuracy for neutral and negative sentiments, as shown in the confusion matrices (Multimedia Appendix 1). The TextBlob lexicon-based model utilizes a higher weight (0.52) than the VADER model, as it demonstrated marginally better accuracy (VADER × 0.48 + TextBlob × 0.52) for positive sentiments. For the ensemble, weighted averaging was utilized to combine the two lexicon-based models. The output of the weighted average lexicons was combined with the DL-based BERT model through a rule-based approach. For positive sentiments, the weighted average output of the lexicon models was selected as the final output. For neutral and negative sentiments, the output of the DL-based BERT model was chosen as the final output. If-else logical constructs were used to determine the final sentiment class based on the base model outputs.

No ethical review was necessary since the data analyzed was fully in the public domain. A thorough assessment of the privacy risk to individuals posed by our research was conducted to ensure compliance with relevant sections of the General Data Protection Regulation (GDPR). We have also striven to comply with best practices for user protection to ensure no nonpublic material was included in our dataset.

The code and data used for data analysis and generation of figures are openly available on GitLab [19] for reproducibility and transparency of the analysis. Due to the computationally expensive nature of the code, we recommend using a high-performance computing resource.

This study has provided insights into some of the underlying issues and concerns surrounding contact tracing apps, and particularly how decisions surrounding system architecture (eg, centralized vs decentralized) can influence public attitudes.

More fine-grained spatiotemporal variation in sentiments across the United Kingdom, and globally, should be further studied. Failure to pay timely attention to negative attitudes and resistance to emerging technologies in population health can have potentially damaging consequences, whereas an understanding of barriers can facilitate the development of effective interventions.

Limitations of our preliminary analysis include that social media platform users are largely not representative of the total UK population (eg, users are younger, more left-wing, and have higher incomes) [33]. Only about 15% of UK adults are regular Twitter users, with activity levels varying among passive and active users, and the demographics of users do not reflect the demographics of the general population [34]. It has also been shown that sentiments of tweets can vary across geographical locations and may therefore reflect the demographics of the population being studied [35]. Variations in sentiments can occur at an individual level, influenced by personal experiences and changes not necessarily related to the subject of the tweet [36]. The developed ensemble-based sentiment analysis approach can be optimized by using additional labelled data for transfer learning and fine-tuning the BERT base model. Adaptive neuro-fuzzy inferencing [37] can also be utilized to replace the current nonadaptive (weighted-average and logical rule-based) constructs.

The current focus on selected social media platforms, outstanding issues around the accuracy of AI techniques (eg, around deceptive language), and the limited number of tweets and Facebook posts due to the specific search strategy used, as well as given that only geo-tagged tweets were used, may limit the generalizability of these findings. Thus, there is a need for a more refined and comprehensive search strategy using multiple social media and web platforms, as well as linked analysis with data from external trustworthy sources such as polls, census, surveys, and clinical notes. Considering the limitations, the proposed approach should currently only be used alongside other methods to assess public sentiment, including public consultations, surveys, and qualitative studies.

In future, mobility trends [38] can be integrated into this approach and demographic determinants can also be inferred [39] and included to provide further insights. Furthermore, subgroup analysis could explore reasons for low uptake in certain populations or communities (eg, anti-vaxxers [40]), which can inform the design of vaccine deployment strategies, including public messaging campaigns. The approach may help promote a learning public health policy cycle, where ideas can be tested against public attitudes before they are implemented, thereby maximizing their effectiveness and real-world applicability [41].

We advocate for future work on AI-enabled social media analysis of public attitudes in health care, which we believe has the potential to help facilitate the implementation of effective public health campaigns. Through this preliminary analysis, we show how such innovative methods may complement findings using conventional methods to monitor public sentiments (eg, surveys) while also providing greater spatiotemporal granularity.