Originally, Telegram aimed to provide secure communication (which is very important for post-Soviet societies [30]), but later, functionality was expanded; it added support for public channels, groups, video calls, and many other features [29]. Telegram groups may be public or private. If a group is public, it may be accessed via the Telegram search engine, and every user may read all its content. A main priority claimed by Telegram is security; users’ data are not disclosed, and only the user's screen name and picture are shown to the public. The largest Telegram channels have millions of subscribers.

The description of the “Sputnik_results” [28] public group states that its main aim is to collect information on AEs regarding the Sputnik V vaccine. Telegram users may post a description of their symptoms. Moderators of the group oversee the messages and verify that they contain only descriptions of AEs; otherwise, the message is deleted. An example message is as follows: “М, 33 года. V1 24.01.21 через 12 часов темп 39, боль в руке (все плечо целиком, мышцы), заложенность носа, диарея. На след день темп 38, боль в руке, заложенность носа. На третий день слабость, температура в норме” (translation: M, 33 years old. V1 24.01.21 after 12 hours, temp. 39, pain in the arm (the entire shoulder, muscles), nasal congestion, diarrhea. The next day, temp. 38, pain in the arm, nasal congestion. On the third day, weakness, temperature is normal).

In this study, we collected all messages from the “Sputnik_Results” group using Python Telegram Client telethon [31]. We saved only text messages that were posted in the group; users’ personal details were not extracted. In total, we collected 18,833 messages. After filtering messages that contained only pictures, 11,515 messages remained. The first message was sent on December 9, 2020, and the most recent message was sent on April 17, 2021. The dataset contained 25,660 unique lowercase words.

The gold standard used to identify adverse reactions is the MedDRA System Organ Class, which is applied in the European Union (EudraVigilance [32]), the United States (Vaccine Adverse Event Reporting System [VAERS] [33]), and the United Kingdom (MHRA Yellow Card scheme [34]). However, the system uses a specialized medical vocabulary. In our study, because users of social media communicated in colloquial language [12], we chose a simplified FDA classification system [35-37] that was subdivided into 2 groups: local reactions (ie, redness, swelling, and pain at the injection site) and systemic reactions (ie, fever, fatigue, headache, chills, nausea/vomiting, diarrhea, new or worsening muscle pain, and new or worsening joint pain). Moreover, muscle pain, joint pain, and pain at the injection site were categorized as a single class. However, we added the classes of pruritus, enlarged lymph nodes, and insomnia, which are common adverse reactions to anti-COVID-19 vaccines [38,39]. Insomnia was chosen due to its high frequency by simple keyword analysis on a sample of material from Telegram. The final list of 12 classes of symptoms of mild AEs, which were based on subjective experiences of a potential health issue, is provided in the Results section.

We utilized the LabelStudio data labeling tool [40] to label the dataset. We randomly sampled 1000 messages in the dataset, which were labeled by 3 raters who were native Russian speakers. The raters labeled each occurrence of an AE in the messages, thus making the dataset suitable for named entity recognition tasks. Because of such labeling and the existence of different descriptions of the same AEs in multiple sentences, we augmented the dataset by splitting each message into sentences. The resulting dataset contained 4579 entities.

Each message in our dataset could have included multiple AEs. We therefore adopted a multilabel text classification scheme. A formal definition of multilabel classification is as follows: Consider a dataset

where x_i ε X is the i-th observed variable for the dataset of cardinality n, y_i ε Y is the corresponding set of labels for the i-th element. Our goal was to learn a mapping ŷ_j = f(x_j,θ), where ŷ_j is the set of predicted classes and θ is a vector of parameters. To find the vector of optimal parameters θ, we needed to minimize the loss function L(y,ŷ) between the actual and predicted classes. Multiple machine learning methods may be applied to support multilabel classification. In the case of artificial neural networks (ANNs), the activation function of the last layer of the ANN is set to be a sigmoid:

and binary cross-entropy loss is used. In this case, ANN will map the probability of each class to a value between 0 and 1, and each data item could be mapped to multiple classes.

Because of the recent success of ANNs, specifically transformers, in text analysis tasks, we adopted a deep Bidirectional Encoder Representations from Transformers (BERT) architecture to perform our multilabel classification task [41]. We utilized a pretrained BERT model for the Russian language DeepPavlov [42]. We tuned the last layer of the model, which consisted of 12 sigmoid neurons. As a baseline, we used a standard long short-term memory (LSTM) ANN, which consisted of embedding as the first layer and 1 LSTM layer (100 cells), dropout (P=.20), and a subsequent multilabel dense layer with sigmoid as the activation function.

We trained the BERT and LSTM models using a stratified k-fold validation scheme where k=5. Because the classes were imbalanced, we utilized an up-sampling strategy; that is, underrepresented classes were up-sampled in the training dataset. The testing set distribution was not modified. Table 1 displays the evaluation results. Precision and recall were calculated for both micro- and macro-averaged aggregations [43]. As shown in Table 1, precision and F1 scores were reported for thresholds equal to 0.5. We utilized a computer with a Tesla T4 GPU to train the models. Table 1 shows that BERT outperformed the LSTM model by a large margin. We therefore chose the BERT model and trained it on 95% of the data; in this case, it returned a micro-averaged accuracy of 0.94 and an area under the receiver operating characteristic (ROC) curve (AUC) score of 0.991.

Regarding gender, age, and dose number (if available), we used counts of corresponding abbreviations and regular expression matching because the administrators of the group had provided detailed instructions for the reporting of this information.

To evaluate the time relationship between the number of reports and vaccination volume, a univariate linear regression coefficient was calculated. Because the number of reports (P<.001) and vaccination volume (P<.001) failed to be normally distributed based on the Shapiro-Wilk test, a Spearman correlation was calculated. Because the number of AEs failed to be normally distributed based on the Shapiro-Wilk test (P<.001), the difference between the 2 groups was analyzed with a Mann-Whitney U test. To compare frequencies of AEs between 2 samples of AEs, a Fisher test was applied. To compare the frequencies of 2 vectors of AEs, the normality was checked with the Shapiro-Wilk test, and Pearson correlations could be calculated (P=.10 and P=.07, respectively, comparing Telegram with the Argentinian Registry; P=.13 and P=.34, respectively, comparing Telegram with the Moscow trial). Community detection was conducted to evaluate the internal structure (co-occurrence) of AEs in the network representation.

The peak in the volume of self-reports corresponded with the time at which vaccinations were sped up (Figure 1). Moreover, after 3 months of vaccinations (the end of February 2021), the popularity of self-reporting started to decrease despite the increasing vaccination roll-out. However, the Spearman correlation coefficient between the volume of self-reports and doses administered from December 9, 2020 until February 28, 2021 was very high (r=0.75, P<.001), and the subsequent count of administered doses increased, while reports on AEs decreased (Figure 1).

Our analysis revealed that fever and generalized pain were the most commonly reported AEs (Table 2). Injection site irritations (local reactions) were an order of magnitude less likely to be reported than fever and pain (systemic reaction). Gastric symptoms (especially diarrhea, with a frequency of 0.6% per report) were less likely to be reported than the average prevalence among the general population (1%-5% for diarrhea [45,46]).

Gender was reported by 3992 women and 2762 men. On average, women reported 2.5 AEs (σ=1.79; Q1=1; Q2=2; Q3=4), and men reported 2.1 AEs (σ=1.64; Q1=1; Q2=2; Q3=3). Women reported statistically significantly more AEs (P<.001) according to the results of a Mann-Whitney U test (Table 3).

Age was provided by 6754 users. A linear regression analysis was performed for those who reported being at least 18 years old (minimal age of Russian registration [1]). We found a clear and significant linear relationship (β=.0457, SE=.0014), showing that with every year of life, users reported .0457 fewer AEs (Figure 2). In univariate regression analysis, β is an estimated coefficient with a given SE. Mild AEs among the elderly are known to be less frequently observed for most anti-COVID-19 vaccines [35-37,47].

AEs in response to other anti-COVID-19 vaccines have been found to depend on whether the vaccination was the first or the second dose (if applicable). For instance, AEs in response to mRNA vaccines have tended to be stronger with the second dose [36,37,47]. In contrast, AEs in response to vector vaccines have tended to be milder with the second dose [48,49]. Regarding the Sputnik V vaccine, this difference might be because a different vector is used in each dose, which might lead to different reactions. Among the self-reports, 4174 described AEs after the first dose, 1251 described AEs after the second dose, and 3049 described AEs after both doses. It is also possible that the users did not receive the second dose because of contraindications or just lost interest in reporting.

Here, we considered only reports that discussed the first and second doses separately. On average, there were 2.2 (σ=1.80; Q1=0; Q2=2; Q3=4) AEs for the first dose and 1.9 (σ=1.69; Q1=0; Q2=2; Q3=3) AEs for the second dose. According to the results of the Mann-Whitney U test, there were statistically significantly more AEs after the first dose (P<.001; Table 3).

To quantify the co-occurrence of symptoms, we calculated Spearman rank correlation coefficients between each pair of classified symptoms. We observed systemic, local, and gastric clusters (Figure 3). We also provided a network representation in which vertex size represents symptom prevalence and edge width represents co-occurrence as measured by the correlation coefficient. Only edges with a correlation coefficient above 0.09 are shown (Figure 3). An unsupervised weighted Louvain algorithm [50] for community detection was used for this purpose, and the vertices were colored the same if they belonged to the same community, which revealed a meaningful structure in which orange denoted systemic, green denoted local, and yellow denoted gastric communities of symptoms.

We compared our results with 2 available datasets of AEs in response to the Sputnik V vaccine. The first one was collected in Moscow. The second one was collected in Argentina.

Regarding vaccines registered by the EMA and FDA, lists of the frequencies of the most common adverse events are accessible; however, they vary across regulatory bodies. Thus, we chose a subset of symptoms for frequencies that were reasonably comparable (pain, headache, fatigue, fever, chills, and nausea). We built a distance (Euclidean) matrix of AEs based on clinical trial registries (EMA [48,52-54], FDA [35-37]) and from the Telegram group (Table 6). From the FDA dataset, for 2-dose vaccines, the dose with higher reactogenicity was selected. In clinical trials, pain is usually considered as pain at the injection site. Fever was the sum of pyrexia and fever in the EMA database. EMA used the injection site tenderness/irritation category. However, regarding redness/erythema, the FDA classified swelling and pruritus separately. Thus, erythema was not included. Sputnik V is a vector vaccine, as are those from AstraZeneca and Johnson & Johnson. The results showed that Telegram Sputnik V AEs were clustered with other vector vaccines, which was possibly due to similar safety profiles (Figure 4).

It is important to note that the Telegram users also submitted reports without any AEs at all. Thus, our surveillance system included a sentinel property of samples in contrast to VAERS (North America), EudraVigilance (European Union), and the Argentinian registry [44], which gather reports only if there is any AE to be reported.

According to clinical trials [3] and official registries [44], only partial information could be retrieved on the Sputnik V safety profile. Previously, multiple researchers have raised concerns about the safety of the Sputnik V vaccine [6,55,56]. Our study aimed to increase transparency regarding the safety of Sputnik V [57], because drug regulatory agencies such as in Brazil were delaying Sputnik V emergency registration: “Anvisa was unable to validate the methodology Russian studies used (...) to track and describe adverse events following vaccination” [58]. In this study, we showed that community-based surveillance via social media can provide meaningful information that could be useful, and this phenomenon should be carefully investigated. The frequencies of AEs extracted from Telegram samples in which at least one AE was reported were in line with other safety surveillance.

Mild, nonsevere AEs have usually been ignored by medical communities because they are common to all vaccines. Antivax movements have emphasized severe AEs, which have been widely discussed in social media [59] in the wider context of vaccine safety [60,61]. In the discourse on COVID-19 vaccines, the main issues were that they were developed quickly and they could compromise safety. Those issues included the fear that vaccines would alter human DNA, cause allergic reactions to vaccine ingredients, result in sudden deaths due to frailty syndrome, or cause infertility [62,63]. Wide anti-COVID-19 immunization programs promulgated a discourse in which risk (eg, the discomfort of common, but mild, AEs as well as rare, but serious AEs) and benefits (eg, efficacy in protecting from the disease) were described as “tradeoffs” of being vaccinated. Mild AEs have become an important issue for many people; moreover, they have the economic component of the potential need for sick leave. This discourse led to the formation of a public Telegram group, where users were asked to report AEs.

In this study, we demonstrated that, in the first phase of the vaccination roll-out, the AE reports were correlated (r=0.7) with vaccination volume (Figure 1). However, Telegram users tended to lose interest after a few months. It is possible that because of the prioritization of vaccine delivery, which began with public and military servants, scientists, teachers, and medical staff, these “early adopters” were more likely to post on social media and be actively involved in reporting AEs. Subsequently, users in the general population were vaccinated, and they were less involved in reporting on the Telegram platform (Figure 1). Thus, interest in COVID-19, Sputnik V, and its AEs was influenced by social context and media to much extent [64].

The results of this study showed that the number of reported AEs decreased linearly according to age (β=.05 AE per year; Figure 2). This result was dependent on biology, which was confirmed in previous clinical trials [36,37,52] and postmarketing observations [47] of other anti-COVID-19 vaccines. Telegram users older than 60 years reported significantly more systemic AEs compared with their peers in clinical trials, who tested negative for or had recovered from COVID-19 [51] (Table 4). On one hand, it is possible that people previously infected with COVID-19 were more likely to report AEs after receiving other vaccines [47]. On the other hand, self-reporting bias could be an important factor in explaining the difference between the “Moscow” clinical trial and the Telegram reports.

The safety profile of the Sputnik V vaccine includes mild AEs that are more similar to vector vaccines than to mRNA anti-COVID-19 vaccines, which was quantified by the Euclidean distance between AE frequencies (Figure 4). The Sputnik V safety profile also showed a high fever-to-fatigue ratio (Table 6) and a stronger reaction to the first dose than to the second one (Table 3), which was also analyzed in a retrospective observational study in San Marino [65].

Women reported more AEs than men (1.2-fold, P<.001; Mann-Whitney U test). This phenomenon is well recorded in other anti-COVID-19 vaccine registries [47,66] and has even been noticed among Argentinian medical staff [67], which could indicate sex-dependent vaccine reactivity. However, this result needs to be understood with caution. The Centers for Disease Control and Prevention has warned that gender bias in reporting could be more important than possible biological mechanisms [35]. The likelihood of disclosing personal information (even anonymously) is known to vary, such as according to gender [68] and social class [69]. A potential reason is that women are more likely to be interested in health, write about health on the internet, and disclose their information [68].

On Telegram, self-reports are most likely to underestimate gastric symptoms (eg, diarrhea at 0.6%). These symptoms could be a taboo effect [70], such as a response to public speaking anxiety. Alternatively, it could be easily ignored because of its high prevalence, or it could be eliminated using an over-the-counter medicine such as loperamide [45,46]. Insomnia was detected so often that it suggests an epidemiological link with the vaccine, which needs further investigation. Local AEs, such as injection site irritation, have rarely been reported. Underlying conditions of erythema/redness, which is usually one of the most common AEs in response to all injected substances including vaccines, are probably overlooked due to low subjective discomfort and lack of physical investigation by a doctor. The findings showed that their actual prevalence was probably underreported.

Our study has several limitations. First, we analyzed participatory and community-based surveillance among Russian Telegram users. Therefore, the results may be specific to the Russian population in a given stage of the pandemic and therefore should not be extrapolated to other contexts. Second, Telegram users may overlook less troublesome side effects, and the social context could influence decisions on taking part in discussions and being selective in reporting AEs [68,69]. For example, local or gastric AEs could be underreported. Third, the classifications developed in this study should not be strictly applied to other contexts. For example, pain at the injection site and pain in other parts of the body were not differentiated. Observed correlations and odds ratios do not imply causation. Fourth, we did not assess the authenticity and credibility of posts [15]; thus, incorrect information could be included in the data. Finally, because our infodemiology study focused on community research initiatives (independent and nonprofit projects, with already known strengths and weaknesses from the history of medicine [71]), our observations cannot replace real-world studies [55-57]. The symptoms reported by social media users only partially reflect their prevalence in the real world [72]. Therefore, the frequencies of symptoms should not be interpreted without considering the contexts and proportions of other symptoms (ie, fever-to-fatigue ratio), phase of the epidemic, and vaccination roll-out (ie, the number of doses administered daily and the population that is vaccinated), as willingness to report AEs satisfies typical product life-cycle temporal characteristics [73,74]).

After the Sputnik V vaccination, Russian Telegram users reported mostly pain, fever, and fatigue. The Sputnik V mild AE profile was comparable with other vector COVID-19 vaccines. Discussions on social media could provide meaningful information about the AE profile of novel vaccines. Further research on severe AEs reported on social media and their credibility is needed.