Comparative evaluation of ten lateral flow immunoassays to detect SARS-CoV-2 antibodies [version 1; peer review: awaiting peer review]

Background: Rapid mobilisation from industry and academia following the outbreak of the novel coronavirus, severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), led to the development and availability of SARS-CoV-2 lateral flow immunoassays (LFAs). High quality LFAs are urgently needed at the point of care to add to currently available diagnostic tools. In this study, we provide evaluation data for ten LFAs suitable for use at the point of care. Methods: COVID-19 positive patients (N=45), confirmed by reverse transcription – quantitative polymerase chain reaction (RT-qPCR), were recruited through the International Severe Acute Respiratory and Emerging Infection Consortium Coronavirus Clinical Characterisation Consortium (ISARIC4C) study. Sera collected from patients with influenza A (N=20), tuberculosis (N=5), individuals with previous flavivirus exposure (N=21), and healthy sera (N=4), collected preOpen Peer Review Reviewer Status AWAITING PEER REVIEW Any reports and responses or comments on the article can be found at the end of the article. Page 1 of 9 Wellcome Open Research 2021, 6:18 Last updated: 29 JUN 2021


Introduction
In December 2019, an outbreak of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) emerged in Wuhan, China before spreading globally, with the World Health Organization (WHO) declaring its pandemic status in March 2020 1 . The reference standard for diagnosis of SARS-CoV-2 are reverse transcription -quantitative polymerase chain reaction (RT-qPCR) assays. However, although a successful RT-qPCR amplification confirms an infection, the peak viral load is short lived and often occurs prior to or in the initial days after symptom onset and therefore the timing of the test is crucial 2,3 .
Patients tested after several days of illness may already have decreasing viral loads and negative RT-qPCR results and other markers of infection, such as immunoglobulins (Ig) could play an adjunct role in diagnosis, particularly in cases presenting ≥10 days from onset of symptoms 4 . SARS-CoV-2 IgG and IgM can be detected in over 90% and 95% of hospitalised patients with confirmed infections respectively, with Ig presenting with simultaneous or sequential conversion 5,6 . Immunoglobulins are usually well established after 21 days of infection, but levels can be detectable at earlier timepoints 7,8 .
Global mobilisation in response to the pandemic resulted in the rapid development of lateral flow immunoassays (LFAs) for SARS-CoV-2. These assays can detect IgG, IgM, and occasionally IgA antibodies, are relatively simple to use and generate results in 10-15 minutes, making them appropriate for the point of care. LFAs identify individuals who have formerly experienced infections (with or without symptoms), to document the prevalence of infection in the population. Moreover, LFAs could also complement the information generated by RT-qPCR for the diagnosis of patients with presumptive COVID-19, detecting an early rise of antibodies at the time that viral loads have become undetectable.
Most LFAs are evaluated in-house by the manufacturer. However, the WHO recommends these evaluations should be complemented by independent evaluations of diagnostic accuracy. Here, we report an evaluation of ten SARS-COV-2 LFAs in a cohort of patients with RT-qPCR confirmed SARS-CoV-2 infections; in hospitalised patients with other conditions, and in healthy individuals.

Participants
Forty-five participants with RT-qPCR-confirmed SARS-CoV-2 infections admitted to National Health Service (NHS) hospitals in the UK, were recruited through the ISARIC4C study. RT-qPCRs were run by UK National Health Service (NHS) accredited laboratories. Patient sera were collected ≥4 days post-symptom onset and one patient provided samples at two time points. Serum samples from a further 50 patients with influenza A (N=20), tuberculosis (N=5), documented previous flavivirus exposure through vaccination or infection (N=21) or healthy sera (N=4) that had been collected pre-pandemic were included as controls.
IgG enzyme-linked immunosorbent assay (ELISA) All samples were screened by ELISA for the detection of anti-SARS-CoV-2 IgG (Omega Diagnostics, Alva, UK), as per the manufacturer's instructions.

LFAs evaluated
All LFAs were evaluated using patient sera and compared to ELISA results. The LFAs included IgG/IgM RDT 1 (ASBT Holdings Ltd, Norfolk, UK), IgG/IgM RDT 2 (ASBT Holdings Ltd, Norfolk, UK), COVID-19 Total Ab Device (Fortress Diagnostics, Antrim, UK), Rapid COVID-19 IgG/IgM Antibody Test (Suzhou Herui BioMed Co., Ltd, China), NADAL COVID-19 IgG/IgM rapid test (Nal von Minden, Moers, Germany), qSARS-CoV-2 IgG/IgM Rapid Test (Cellex, Durham, NC, USA), COVID-19 IgG/IgM Rapid Test (Nantong Egens Biotechnology, Nantong, China), NG-Test IgG-IgM COVID-19 (NG Biotech, Guipry, France), Generation one RDT prototype (Mologic, Bedfordshire, UK), and Triple Antibody RDT (Mologic, Bedfordshire, UK). All tests detect anti-SARS-CoV-2 IgG and IgM, except the Mologic LFAs which additionally detect anti-SARS-CoV-2 IgA. Serum samples for the evaluation had been stored at -80°C and were thawed to room temperature before use. LFAs were run at room temperature according to the manufacturer's instructions, as summarised in Table 1. In brief, 5-10µl serum were added to the LFA well, followed by 70-100µl of the proprietary test diluent in the same or a separate buffer well. Test lines were scored as positive or negative by two independent readers at the specified reading time. The readers were blinded to each other's results and discrepant scores were resolved by a third reader.

Statistical analysis
Data analysis was carried out in Microsoft Excel (version 2012). Sensitivity was calculated as the proportion of positive tests in convalescent sera of patients with RT-qPCR confirmed infections. Specificity was calculated as the proportion of negative tests among SARS-CoV-2 negative controls collected pre-pandemic. Further calculations of sensitivity and specificity were made when using the IgG ELISA as the reference standard. The levels of agreement were calculated using Cohen's Kappa statistic 9 .

Results
The 45 participants with confirmed SARS-CoV-2 infections had a median age of 58 years (IQR: 19) and 25 (56%) were male. Sensitivity of the LFAs ranged from 87% to 96%, with IgG ranging from 83% to 94% and IgM between 24% and 96%. Specificity ranged from 92% to 100% for both IgG and IgM (Table 2). Differences in sensitivity and specificity across the tests were not statistically different. Sensitivity of the LFAs up to 3 weeks post-symptom onset ranged from 85% to 91% and from 83% to 100% after 3 weeks post-symptom onset. All tests except Generation one RDT prototype (Mologic, Bedfordshire, UK) recorded higher sensitivity over 3 weeks post-symptom onset (Extended data:  Table 3. The greatest agreement between ELISA and a LFA was seen with the IgG/ IgM RDT 1 (ASBT Holdings Ltd), COVID-19 Total Ab Device (Fortress Diagnostics) and Rapid COVID-19 IgG/IgM Antibody Test (Suzhou Herui BioMed Co., Ltd) (kappa=0.854 for all three, corresponding to very good agreement).

Discussion
LFAs are potential tools for disease surveillance and the assessment of presence of antibodies to infection, which are rapid and easily conducted. In this study, we evaluated ten LFAs using sera from RT-qPCR confirmed SARS-CoV-2 infections and sera collected pre-pandemic. Although differences between tests were not statistically significant, the Fortress Total Ab Device had the highest overall sensitivity when compared to RT-qPCR. Across all tests, IgM had the widest sensitivity range (from 26% to 96%) while IgG LFAs had similar ranges between 83% and 94%. Specificity was high across all assays and four out of the ten LFAs had specificity ≥ 98%. The IgG ELISA's sensitivity and specificity (Omega, UK) are 95% (≥7 days) and 97% (≥10 days) postdiagnosis, respectively 11 and thus, as expected, the sensitivity and specificity of the LFAs was higher when ELISA was considered the reference standard. All LFAs had very high agreement with IgG ELISA. These data suggest LFAs can provide valuable data that is highly correlated to ELISAs, with an acknowledged small loss in sensitivity. LFAs have the advantage of being rapid and simple to run with no requirement for a laboratory or trained operators. This makes them highly suitable for low resource settings, self-testing, surveillance of the prevalence of infection or when rapid screening is required.
LFAs should have high specificity, especially in settings where infection rates are low, to avoid high numbers of false positives 12 . COVID-19 clinical presentation may be indistinguishable from other respiratory illnesses and LFAs could complement the information generated by RT-qPCR assays, with the tests combined identifying a larger number of individuals with current and previous SARS-CoV-2 infections. Moreover, with seasonal influenza likely coinciding with high COVID-19 incidence, these assays could play a significant role to differentiate SARS-CoV-2 from other viral infections and facilitate more targeted strategies for the management and quarantining of symptomatic patients.

NG-Test® IgG-IgM COVID-19 (NG
We acknowledge this validation has several limitations, as it included a small sample size, without enough power to find small differences in the performance of the assays, and therefore our findings can only be shown to be indicative of the likely findings of appropriately powered evaluations. All SARS-CoV-2 positive samples were obtained from hospitalised patients as a marker of severe COVID-19 who may be expected to have high levels of antibodies, and our findings need to be replicated in asymptomatic individuals and in patients presenting with mild symptoms and in community settings. Furthermore, all testing was run under laboratory conditions by laboratory staff and the results may vary when conducted at the point of need and self-testing by untrained individuals. Finally, we were constrained to use serum, and further evaluations are needed to evaluate their performance on whole venous or capillary blood. Test performance on capillary blood, including a comparison of self-testing and laboratory testing, would inform the potential for use of less invasive sample collection methods. The impartial evaluation of LFAs, as recommended by the WHO, can confirm the evaluations of the developer's validation data, and inform best-use settings. LFAs are a valuable tool which could be applied at the point of need in conjunction with other assays to provide a more holistic diagnosis and to monitor the prevalence of infection.
Data are available under the terms of the Creative Commons Zero "No rights reserved" data waiver (CC0 1.0 Public domain dedication).