Between March 23, 2020 and March 29, 2020, we enrolled a consecutive cohort of 191 patients who were admitted to the Emergency Department of the Policlinico University Hospital in Bari, Italy for COVID-19–related symptoms or because they were quarantined for previous exposure to COVID-19–positive individuals. Oropharyngeal swabs for standard SARS-CoV-2 RT-PCR analysis and venous blood samples for VivaDiag tests were simultaneously collected from each subject, and the tests were immediately performed in reference laboratories. Registries containing patients’ main clinical data, including date of symptom onset (self-reported), were created. Informed written consent was obtained from all patients. Oropharyngeal swab samples were immediately analyzed for SARS-CoV-2 by RT-PCR at the Laboratory of Molecular Epidemiology and Public Health of the Hygiene Unit of the Policlinico University Hospital (Bari, Italy), the regional reference laboratory for SARS-CoV-2 identification. The venous blood samples were analyzed at the Clinical Pathology Laboratory (Certified ISO-9001/2015; Head E. Savino) and the Institutional BioBank (Certified ISO-9001/2015; Head A. Paradiso) of the IRCCS Istituto Tumori Giovanni Paolo II (Bari, Italy). This study was approved by the Ethical Committee of the IRCCS Istituto Tumori Giovanni Paolo II, Bari (Protocol number CE 870/2020).

Nasopharyngeal/oropharyngeal swabs were subjected to nucleic acid extraction with the MagNA Pure System (Roche Diagnostics), in accordance with the manufacturer’s instructions. The presence of the E gene, RdRP gene, and N gene of the SARS-CoV-2 virus were identified by a commercial RT-PCR assay (Allplex 2019-nCoV Assay; Seegene). Samples were considered positive at molecular screening if all three genes were detected. The Centers for Disease Control and Prevention RT-PCR protocol was used to confirm the presence of SARS-CoV-2 [9]. To date [10,11], this methodology is considered the gold standard for the detection of SARS-CoV-2 infection.

The SARS-CoV-2 IgM/IgG combined antibody rapid test kit, VivaDiag, (VivaChek Biotech) is a lateral flow qualitative immunoassay used for the rapid determination of the presence or absence of both anti-SARS-CoV-2-IgM and anti-SARS-CoV-2-IgG in human specimens (whole blood, serum, and plasma). A surface antigen from SARS-CoV-2, which can specifically bind to SARS-CoV-2 antibodies (including both IgM and IgG), is conjugated to colloidal gold nanoparticles and sprayed onto conjugation pads. The rapid SARS-CoV-2 IgG/IgM combined antibody test strip has two mouse antihuman monoclonal antibodies (anti-IgG and anti-IgM) on two separate test lines.

When testing, 10-15 µL of a specimen was inserted into the sample port, and then the sample dilution buffer was added. As the specimen flowed through the device, anti-SARS-CoV-2 IgG and IgM antibodies, if present in the specimen, were bound by the SARS-CoV-2 antigens (ie, the gold colorimetric reagent fixed on the conjugate pad). As the conjugated sample continued to travel up the strip, the anti-SARS-CoV-2 IgM antibodies were bound on the M (IgM) line, and the anti-COVID-19 IgG antibodies were bound to the G (IgG) line. If the specimen did not contain SARS-CoV-2 antibodies, no labeled complexes were bound. The presence of SARS-CoV-2 IgG and IgM antibodies was indicated by a red/purple line on a specific region of the device. Each test was evaluated by two readers, and a picture was taken of the result. In case of disagreement, the picture was evaluated by a third party.

The performance of the VivaDiag tests was compared to that of the RT-PCR tests using the caret R package, which computed all the parameters needed (sensitivity, specificity, accuracy, and Cohen κ). The performance evaluation was carried out using RT-PCR as the gold standard. Both tests were performed on the same subjects. Univariate and multivariate logistic regression were performed. Age was dichotomized by using the median age as a cutoff, and the number of days after the onset of symptoms was used as a categorical variable (0-5 days, 6-8 days, 9-10 days, 11-15 days, >15 days). All analyses were carried out in R version 3.6 (The R Foundation), and results were considered to be significant when the P value was <.05.

When we compared the performance of the rapid serological test to that of RT-PCR for the detection of SARS-CoV-2 infection, our findings showed that 17.8% (34/191) of the subjects tested positive based on serum IgM/IgG expression, whereas 36.6% (70/191) of the subjects tested positive based on SARS-CoV-2 RT-PCR test results, leading to a sensitivity of 30% and a specificity of 89% for the serological test.

The clinical relevance of so-called rapid serological testing is still an open issue, since the data currently available are still scarce [13]. For this reason, we compared its performance to that of standard RT-PCR testing and analyzed performance with respect to the time of COVID-19–related symptom onset. To this end, we set up a mono-institutional consecutive cohort of patients who were tested with both assays at a single qualified laboratory.

The design of our study allowed us to specifically analyze two aspects of the open issue: (1) the concordance of the rapid serological test results with those of standard molecular testing and (2) the relationship between IgG/IgM expression and the onset of clinical symptoms.

With regard to the degree of concordance between the two tests, the results reported in Figure 1 clearly show that the precision of the VivaDiag rapid test is unsatisfactory. Notably, only 11% (21/191) of the patients that tested positive for COVID-19 based on the molecular test results also tested positive based on the serological test results. This percentage is impressively similar to the performance reported for serological tests in Spain [14] and Germany [15]. However, the first important finding from our study concerns the 6.8% (13/191) of subjects that tested negative based on RT-PCR results, but tested positive based on serological results. The two tests did not produce similar results, which is obvious for assays that are designed to analyze different aspects of COVID-19; the molecular test detects the presence of SARS-CoV-2 in samples based on specific anatomical parts of the respiratory system, while the serology test reveals the kinetics of immunoglobulins as the body reacts to viral infection. Negative serological test results in patients with a positive molecular test could mean that the patients are infected, but have not yet reached the stage of immunoglobulin reaction development. Conversely, subjects who have negative molecular test results, but also have serological test results showing the presence of specific IgG/IgM antibodies, may be recovering from COVID-19. The data shown in Figure 2 seem to confirm these assumptions, as the molecular test yielded more positive results during the early symptomatic phases of the disease in our subjects, while the serological test performed better later on (ie, 10 days after symptom appearance).

The second aspect we were able to analyze in our study was seroconversion and the kinetics of immunoglobulins with respect to the onset of COVID-19–related symptoms. Figure 3 shows the behavior of the two immunoglobulins according to symptom appearance. Interestingly, IgG and IgM did not seem to behave differently based on the number of days elapsing from symptom appearance, but they clearly and progressively increased along the course of the disease. This unexpected finding, in contrast with common knowledge concerning the kinetics of the two immunoglobulins, is supported by the results presented by Zhang et al [16] and Lou et al [17]. Both authors reported that the detectable serology markers, IgG and IgM, had similar seroconversions in COVID-19 patients, with antibody levels increasing rapidly at 6 days after exposure, and this trend occurred with a concomitant decline in viral load. Such behavior in the 6-10-day time window after symptom appearance is typically accompanied by an improvement in serological test sensitivity compared to standard molecular testing.

Very recently, Whitman et al [18] evaluated the performance of 11 SARS-CoV-2 serological assays in a multicentric study that enrolled a cohort of subjects with positive RT-PCR test results. For the serological tests, the incidence of IgG/IgM positivity in the RT-PCR SARS-CoV-2–positive samples ranged from 26.9% to 0%. In particular, the incidence of positive results with the VivaDiag test was 8.2%, which is significantly lower than the value we found in our study. However, the enrollment criteria in the Whitman et al study were very different from ours. In our study, the subjects were consecutively recruited from a single institution, and the tests were performed simultaneously on fresh biospecimens. Interestingly, Whitman et al reported that among the SARS-CoV-2 RT-PCR–positive individuals, the percent seropositivity increased with time, peaking at 81.8%-100.0% in samples taken >20 days after symptom onset. The same trend of increased seropositivity with increased time from symptom onset was observed in our cohort of subjects.

Our study had some important limitations. First, the VivaDiag test was based on the colorimetric evaluation of the IgG and IgM bands performed by the operator, thus implying that all the limitations that a qualitative inter-intra-operator evaluation produces in terms of variability are present in this study [19]. In our study, this was partially solved by resorting to double operator evaluation and taking pictures of all test results to be reanalyzed by a third-party reader in the case of first level evaluation disagreement. Therefore, our next step will be to use quantitative immunoenzymatic methods to analyze SARS-CoV-2–specific immunoglobulins [20] to overcome these issues.

Second, the neutralizing antibodies used in the VivaDiag test might cross-react with other coronavirus antigens, such as those of the SARS-CoV. The recombinant antigen utilized in VivaDiag is the receptor-binding domain of the SARS-CoV-2 spike protein, for which information on possible cross-reactivity with other coronaviruses and flu viruses has not yet been studied [13]. Further studies are urgently needed to definitively clarify this point.

Our study analyzed the clinical performance of the rapid serological test, VivaDiag, and confirmed the test’s limited applicability for the diagnosis of SARS-CoV-2 infection by comparing its performance to that of standard molecular testing. However, this rapid serological test seems to provide important information concerning individuals’ immunoreaction to the infection, and more importantly, it may detect previous exposure to the virus in currently healthy persons. A trial, recently registered in ClinicalTrial.gov (NCT04316728), will specifically address this issue by investigating the monitoring of seroconversion of COVID-19 IgG/IgM in healthy subjects who may develop COVID-19–related symptoms. In essence, our real-world results should be considered hypothesis-generating findings that warrant further examination in a controlled clinical trial in order to be confirmed [21].