This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.
There is a limited amount of data on the safety profile of the COVID-19 vector vaccine Gam-COVID-Vac (Sputnik V). Previous infodemiology studies showed that social media discourse could be analyzed to assess the most concerning adverse events (AE) caused by drugs.
We aimed to investigate mild AEs of Sputnik V based on a participatory trial conducted on Telegram in the Russian language. We compared AEs extracted from Telegram with other limited databases on Sputnik V and other COVID-19 vaccines. We explored symptom co-occurrence patterns and determined how counts of administered doses, age, gender, and sequence of shots could confound the reporting of AEs.
We collected a unique dataset consisting of 11,515 self-reported Sputnik V vaccine AEs posted on the Telegram group, and we utilized natural language processing methods to extract AEs. Specifically, we performed multilabel classifications using the deep neural language model Bidirectional Encoder Representations from Transformers (BERT) “DeepPavlov,” which was pretrained on a Russian language corpus and applied to the Telegram messages. The resulting area under the curve score was 0.991. We chose symptom classes that represented the following AEs: fever, pain, chills, fatigue, nausea/vomiting, headache, insomnia, lymph node enlargement, erythema, pruritus, swelling, and diarrhea.
Telegram users complained mostly about pain (5461/11,515, 47.43%), fever (5363/11,515, 46.57%), fatigue (3862/11,515, 33.54%), and headache (2855/11,515, 24.79%). Women reported more AEs than men (1.2-fold,
After the Sputnik V vaccination, Russian Telegram users reported mostly pain, fever, and fatigue. The Sputnik V AE profile was comparable with other vector COVID-19 vaccines. Discussion on social media could provide meaningful information about the AE profile of novel vaccines.
The current COVID-19 pandemic is one of the most critical global health problems. The main strategies for its mitigation involve both nonpharmaceutical interventions (eg, testing and contract tracing) and up-to-date anti-COVID-19 treatments. However, the most promising intervention has been vaccines that have effectively prevented severe COVID-19 outcomes. In addition to novel messenger RNA (mRNA) vaccines, vector vaccines have been developed. One of the first was Gam-COVID-Vac (Sputnik V), which is a viral, 2-dose, vector vaccine based on 2 human adenoviruses. Each dose contains a different vector: rAd26 and rAd5. This vaccine was developed by the Gamaleya Research Institute of Epidemiology and Microbiology. Sputnik V contains a gene that encodes SARS-CoV-2’s spike (S) protein [
An increasing number of studies has analyzed English-language social media in the context of vaccinations [
Most previous studies on social media vaccine discourse have focused on the personal beliefs of users. For example, Wang et al [
The dataset analyzed in our study was collected retrospectively from the Telegram group, “Sputnik_results“ [
Originally, Telegram aimed to provide secure communication (which is very important for post-Soviet societies [
The description of the “Sputnik_results” [
In this study, we collected all messages from the “Sputnik_Results” group using Python Telegram Client telethon [
The gold standard used to identify adverse reactions is the MedDRA System Organ Class, which is applied in the European Union (EudraVigilance [
We utilized the LabelStudio data labeling tool [
Each message in our dataset could have included multiple AEs. We therefore adopted a multilabel text classification scheme. A formal definition of multilabel classification is as follows: Consider a dataset
where
and binary cross-entropy loss is used. In this case, ANN will map the probability of each class to a value between 0 and 1, and each data item could be mapped to multiple classes.
Because of the recent success of ANNs, specifically transformers, in text analysis tasks, we adopted a deep Bidirectional Encoder Representations from Transformers (BERT) architecture to perform our multilabel classification task [
We trained the BERT and LSTM models using a stratified k-fold validation scheme where
Regarding gender, age, and dose number (if available), we used counts of corresponding abbreviations and regular expression matching because the administrators of the group had provided detailed instructions for the reporting of this information.
Bidirectional Encoder Representations from Transformers (BERT) and long short-term memory (LSTM) model evaluation results.
Model | Micro-averaged aggregations | Macro-averaged aggregations | ||||
|
AUCa, mean (SD) | Precision, mean (SD) | F1, mean (SD) | Precision, mean (SD) | F1, mean (SD) | |
LSTM | 0.969 (0.002) | 0.866 (0.024) | 0.769 (0.033) | 0.514 (0.048) | 0.431 (0.042) | |
BERT | 0.991 (0.002) | 0.915 (0.016) | 0.920 (0.002) | 0.863 (0.025) | 0.858 (0.006) |
aAUC: area under the curve.
To evaluate the time relationship between the number of reports and vaccination volume, a univariate linear regression coefficient was calculated. Because the number of reports (
Reactogenicity assessment based on opt-in civic surveillance was performed to obtain results of clinical importance (similar to endpoints in trials).
The peak in the volume of self-reports corresponded with the time at which vaccinations were sped up (
Daily counts of reports of adverse events (AE) and doses administered in Russia (data according to Our World in Data [
Our analysis revealed that fever and generalized pain were the most commonly reported AEs (
Frequencies of mild adverse events extracted from the Telegram group (n=11,515).
Adverse events | n (%) | ||
|
|
||
|
Fever | 5461 (47.43) | |
|
Pain | 5363 (46.57) | |
|
Fatigue | 3862 (33.54) | |
|
Headache | 2855 (24.79) | |
|
Chills | 2651 (23.02) | |
|
Insomnia | 600 (5.21) | |
|
Lymph node enlargement | 186 (1.62) | |
|
|
||
|
Erythema/redness | 319 (2.77) | |
|
Swelling | 206 (1.79) | |
|
Pruritis | 199 (1.73) | |
|
|
||
|
Nausea/vomiting | 351 (3.05) | |
|
Diarrhea | 66 (0.57) |
Gender was reported by 3992 women and 2762 men. On average, women reported 2.5 AEs (
Age was provided by 6754 users. A linear regression analysis was performed for those who reported being at least 18 years old (minimal age of Russian registration [
AEs in response to other anti-COVID-19 vaccines have been found to depend on whether the vaccination was the first or the second dose (if applicable). For instance, AEs in response to mRNA vaccines have tended to be stronger with the second dose [
Here, we considered only reports that discussed the first and second doses separately. On average, there were 2.2 (
Comparisons of the mean numbers of adverse events (AEs) by gender and by dose using Mann-Whitney U tests.
Variable | Number of AEs, mean | ORa | |||||
|
|
|
|
||||
|
Male | 2.1 | 1.20 | <.001 | |||
|
Female | 2.5 | |||||
|
|
|
|
||||
|
First | 2.2 | 1.13 | <.001 | |||
|
Second | 1.9 |
aOR: odds ratio.
Scatterplot of the number of adverse events (AEs) reported by user vs. age. Dots indicate the mean number of AEs for a given age, while the blue line indicates the linear regression trend and shadowed area indicate its CIs.
To quantify the co-occurrence of symptoms, we calculated Spearman rank correlation coefficients between each pair of classified symptoms. We observed systemic, local, and gastric clusters (
Co-occurrence of adverse events (AEs), shown as (A) hierarchical clustering based on the correlation matrix of AE symptoms and (B) the corresponding network of AE symptoms with different communities denoted by color code.
We compared our results with 2 available datasets of AEs in response to the Sputnik V vaccine. The first one was collected in Moscow. The second one was collected in Argentina.
Mild AEs in 1029 patients older than 60 years in the phase 3 clinical trial [
We performed the following calculations to compare both datasets. To obtain
In all systemic reactions, Telegram users reported AEs significantly more often than measured in the clinical trial (
Comparisons of adverse events with the Sputnik vaccine between the Telegram and Moscow clinical trial [
Adverse event | Moscow clinical trial, n (%) | Telegram, n (%) | ORa | |
Pain | 67 (6.70) | 177 (25.65) | 3.82 | <.001 |
Headache | 30 (2.92) | 89 (12.90) | 4.42 | <.001 |
Fatigue | 31 (3.01) | 141 (20.43) | 6.78 | <.001 |
Fever | 32 (3.11) | 163 (23.62) | 7.59 | <.001 |
Nausea | 12 (1.17) | 9 (1.30) | 1.12 | .83 |
Erythema | 39 (3.79) | 15 (2.17) | 0.57 | .09 |
Diarrhea | 8 (0.78) | 3 (0.43) | 0.56 | .54 |
aOR: odds ratio for the Moscow clinical trial.
bFisher test results for the comparison between samples.
Another available dataset on AEs in response to Sputnik V was compiled from the Argentinian registry of passive AE monitoring (
We categorized gastric as the frequency of the logical function nausea OR diarrhea. We categorized site irritation as the frequency of the logical function pruritus OR erythema OR swelling. We categorized fever_pain as the frequency of the logical function fever AND (pain OR headache). We categorized fatigue_pain as the frequency of the logical function fatigue AND (pain OR headache). We categorized only_fever as the frequency of the logical function fever AND ˜(pain OR headache OR fatigue); ˜ denotes logical negation.
The comparison showed that the statistics, despite the significant differences shown in
Comparisons of adverse events with the Sputnik vaccine between the Telegram and Argentinian safety monitoring [
Adverse event | Argentinian registry, n (%) | Telegram, n (%) | ORa | |
fever_pain | 8210 (33.25) | 4142 (54.70) | 1.66 | <.001 |
fatigue_pain | 9407 (38.10) | 2998 (39.67) | 1.05 | .05 |
gastric | 1447 (5.98) | 395 (5.14) | 0.90 | .07 |
site irritation | 2306 (9.34) | 558 (7.31) | 0.80 | <.001 |
only_fever | 2065 (8.34) | 697 (9.53) | 1.11 | .02 |
aOR: odds ratio for the Argentinian registry.
bFisher test results for the comparison between samples.
Regarding vaccines registered by the EMA and FDA, lists of the frequencies of the most common adverse events are accessible; however, they vary across regulatory bodies. Thus, we chose a subset of symptoms for frequencies that were reasonably comparable (pain, headache, fatigue, fever, chills, and nausea). We built a distance (Euclidean) matrix of AEs based on clinical trial registries (EMA [
It is important to note that the Telegram users also submitted reports without any AEs at all. Thus, our surveillance system included a sentinel property of samples in contrast to VAERS (North America), EudraVigilance (European Union), and the Argentinian registry [
Adverse events in response to Sputnik V (Telegram) and other vaccines (European Medicines Agency [EMA] and Centers for Disease Control and Prevention [CDC]/Food and Drug Administration [FDA]).
Vaccine | Pain, n (%) | Headache, n (%) | Fatigue, n (%) | Fever, n (%) | Chills, n (%) | Nausea, n (%) |
AstraZeneca (EMA) | –a (54.20) | – (52.60) | – (53.10) | – (41.50) | – (31.90) | – (21.80) |
Johnson & Johnson (EMA) | – (48.60) | – (38.90) | – (38.20) | – (14.00) | – (5.00) | – (14.20) |
Johnson & Johnson (CDC; 18-59 years old) | 1193 (59.80) | 905 (44.40) | 891 (43.80) | 261 (12.80) | – (5.00) | 315 (15.50) |
Pfizer (EMA) | – (80.00) | – (50.00) | – (60.00) | – (30.00) | – (30.00) | – (5.00) |
Pfizer (CDC; 18-54 years old) | 1632 (77.80) | 1085 (51.70) | 1247 (59.40) | 331 (15.80) | 737 (35.10) | – (10.00) |
Sputnik (Telegram) | 5363 (46.57) | 2855 (24.80) | 3862 (33.54) | 5461 (47.43) | 2651 (23.02) | 351 (3.00) |
Moderna (CDC; 18-64 years old) | 9335 (90.10) | 6500 (62.80) | 7002 (67.60) | 1806 (17.40) | 5001 (48.30) | 2209 (21.30) |
Moderna (EMA) | – (92.00) | – (64.70) | – (70.00) | – (15.50) | – (45.40) | – (23.00) |
aNot reported.
Hierarchical matrix of adverse event (AE) similarity of various vaccines and reporting systems (Euclidean distance) of vaccinations investigated in the present study. Astra: AstraZeneca; CDC: Centers for Disease Control and Prevention; EMA: European Medicines Agency; FDA: Food and Drug Administration; J&J: Johnson & Johnson.
According to clinical trials [
Mild, nonsevere AEs have usually been ignored by medical communities because they are common to all vaccines. Antivax movements have emphasized severe AEs, which have been widely discussed in social media [
In this study, we demonstrated that, in the first phase of the vaccination roll-out, the AE reports were correlated (
The results of this study showed that the number of reported AEs decreased linearly according to age (
The safety profile of the Sputnik V vaccine includes mild AEs that are more similar to vector vaccines than to mRNA anti-COVID-19 vaccines, which was quantified by the Euclidean distance between AE frequencies (
Women reported more AEs than men (1.2-fold,
On Telegram, self-reports are most likely to underestimate gastric symptoms (eg, diarrhea at 0.6%). These symptoms could be a taboo effect [
Our study has several limitations. First, we analyzed participatory and community-based surveillance among Russian Telegram users. Therefore, the results may be specific to the Russian population in a given stage of the pandemic and therefore should not be extrapolated to other contexts. Second, Telegram users may overlook less troublesome side effects, and the social context could influence decisions on taking part in discussions and being selective in reporting AEs [
After the Sputnik V vaccination, Russian Telegram users reported mostly pain, fever, and fatigue. The Sputnik V mild AE profile was comparable with other vector COVID-19 vaccines. Discussions on social media could provide meaningful information about the AE profile of novel vaccines. Further research on severe AEs reported on social media and their credibility is needed.
adverse event
artificial neural network
area under the curve
Bidirectional Encoder Representations from Transformers
European Medicines Agency
Food and Drug Administration
long short-term memory
Medicines and Healthcare products Regulatory Agency
receiver operating characteristic
Vaccine Adverse Event Reporting System
The authors acknowledge the initiators and users of the Telegram group “Sputnik_results” for creation of the data analyzed and express gratitude to the editor and 3 anonymous referees. AJ and VB were partially funded by the German Research Foundation (DFG: 458528774) and Polish-German Foundation for Science (PNFN: 2019-21). Support from The Endowment Fund of St. Petersburg State University is gratefully acknowledged by AS. MK is medicine practitioner at Individual Medical Practice, Oborniki, Poland.
MK received remuneration for performing vaccinations against COVID-19 in primary care. The vaccinations did not involve Sputnik V.