An optimization of four SARS-CoV-2 qRT-PCR assays in a Kenyan laboratory to support the national COVID-19 rapid response teams

Background: The COVID-19 pandemic relies on real-time polymerase chain reaction (qRT-PCR) for the detection of Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2), to facilitate roll-out of patient care and infection control measures. There are several qRT-PCR assays with little evidence on their comparability. We report alterations to the developers’ recommendations to sustain the testing capability in a resource-limited setting. Methods: We used a SARS-CoV-2 positive control RNA sample to generate several 10-fold dilution series that were used for optimization and comparison of the performance of the four qRT-PCR assays: i) Charité Berlin primer-probe set, ii) European Virus Archive – GLOBAL (EVAg) primer-probe set, iii) DAAN premixed commercial kit and iv) Beijing Genomics Institute (BGI) premixed commercial kit. We adjusted the manufacturer- and protocol-recommended reaction component volumes for these assays and assessed the impact on cycle threshold (Ct) values. Results: The Berlin and EVAg E gene and RdRp assays reported mean Ct values within range of each other across the different titrations and with less than 5% difference. The DAAN premixed kit produced comparable Ct values across the titrations, while the BGI kit improved in performance following a reduction of the reaction components. Conclusion: We achieved a 2.6-fold and 4-fold increase in the number of tests per kit for the commercial kits and the primer-probe sets, respectively. All the assays had optimal performance when the primers and probes were used at 0.375X, except for the Berlin N gene assay. The DAAN kit was a reliable assay for primary screening of SARS-CoV-2 whereas the BGI kit’s performance was dependent on the volumes and concentrations of both the reaction buffer and enzyme mix. Our recommendation for SARS-CoV-2 diagnostic testing in resource-limited settings is to optimize the assays available to establish the lowest volume and suitable concentration of reagents required to produce valid results.


This article is included in the KEMRI | Wellcome
Trust gateway.
This article is included in the Coronavirus (COVID-19) collection.

Introduction
The coronavirus disease 2019 (COVID-19) pandemic that began in China 1 is caused by a novel coronavirus, named Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) 2 . It is an important public health concern due to its global spread and unexpected high mortality (of 411,680 globally as at 10 th June 2020) [https://coronavirus.jhu.edu/map.html], which is compounded by the unavailability of a treatment or vaccine to control or prevent the disease at the time of writing this paper, early in the pandemic. SARS-CoV-2 belongs to a wider group of coronaviruses that causes respiratory distress in animals, birds and humans 3 . Its genomic characterization has shown that it is distinct from severe acute respiratory syndrome coronavirus (SARS-CoV) and the Middle East respiratory syndrome (MERS) 4 . COVID-19 mainly affects the lower respiratory tract, which can result in fatal pneumonia 5 . By 10 th June 2020, there were over 7.25 million accumulated cases globally 6 and Africa accounted for 203,899 cases and 5,530 deaths. Of these, Kenya had reported 3094 cases and 89 fatalities 7 . The number of cases may be largely underestimated due to the limited capacity for testing 8 .
Highly sensitive and specific diagnostics for COVID-19 can inform efforts geared towards case detection, isolation, quarantine, contact tracing and subsequent infection control measures. Many antibody and antigen detection tests are still under validation at this time 4 . Furthermore, antibody tests provide evidence of exposure to infection and do not clearly diagnose the presence of active infections for decisions to be made on treatment and isolation. Due to these limitations, quantitative reverse transcription-PCR (qRT-PCR) remains a valuable laboratory diagnostic test for COVID-19. Progress in developing specific primers and standardized laboratory protocols for COVID-19 was made possible by the availability of SARS-CoV-2 genomes early in the epidemic 4,9,10 . The first qRT-PCR assay (Charité, Berlin) was subsequently developed by the Charité Institute of Virology, Universitätsmedizin Berlin, and it targets three regions in the SARS-CoV-2 genome, including envelope (E), nucleocapsid (N) and RNA-dependent RNA polymerase (RdRp) 11 . Subsequently, other testing kits were developed and introduced into the market: including the European Virus Archive -GLOBAL (EVAg) primer-probe set that targets the E and RdRp regions 12,13 , the DAAN kit (DAAN Gene Co. Ltd of Sun Yat-sen University) targets the ORF1ab and N coding regions 14 , and the BGI kit (BGI Genomics Co. Ltd) targets the ORF1ab region 15 .
The Kenya Medical Research Institute-Wellcome Trust Research Programme (KWTRP), Kilifi, laboratory was assigned the responsibility of providing diagnostic testing support for all Coastal counties since the outbreak started in Kenya. Currently, like many low and middle-income countries, Kenya depends on international purchases and donations for testing kits. The main limitation of this process is the delays in receiving reagents from the international manufacturers due to the global travel restrictions, resulting in an inconsistent supply of testing reagents.
To mitigate these challenges, the aforementioned assays were optimized to primarily establish the lowest volume and suitable concentration of reagents required to produce valid results. This article details the lessons learnt from using these assays early in the pandemic and presents the optimal parameters to maximize the use of the limited kits and reagents available while still maintaining assay validity.

RNA extraction
Ribonucleic acid (RNA) was extracted from the positive control, a SARS-CoV-2 heat-inactivated culture supernatant donated by Aix-Marseille University, Marseille, France) and a non-template control (nuclease-free water) using QIAamp Viral RNA Mini Kit (Qiagen). Extraction was done as per the manufacturers' instructions. The positive control RNA sample was used to generate 10-fold dilution series that were used for optimization and comparison of the performance of the four qRT-PCR assays.

Real-time PCR assays modifications
We adjusted the manufacturer-and protocol-recommended reaction component volumes for all the assays and assessed the impact on cycle threshold (Ct) values. The assays used included the Berlin targeting E, N and RdRp genes individually, European Virus Archive (EVAg) targeting E and RdRp genes individually, the DAAN kit targeting the ORF1ab and N coding regions simultaneously, and the BGI kit targeting

Amendments from Version 1
This version has been re-structured to better highlight the lowest volumes of the reagents required to increase assay throughput and still maintain diagnostic performance. The ' Abstract' section was modified to match the changes made in the methods, results and conclusions sections. In the 'Introduction' section, the manufacturers of the qRT-PCR assays were included. The objective was modified to reflect the main findings.
All sections referring to patients' samples used in the optimization assays were excluded since they were not used for the optimization experiments. In the 'Methods' section new titles were generated to ensure sequential reporting of the optimization steps and for easy comprehension. The manufacturer-recommended volumes are represented by 1X, and our titration points are now referred to by 0.5X, 0.375X and 0.25X to represent 50%. 37.5% and 25% of the manufacturerrecommended volumes respectively. The tables have been edited and merged according to the four qRT-PCR assays/kits tested, thus reducing the number of tables in the manuscript. All data on QuantiFast Multiplex RT-PCR +R Kit Master Mix were excluded, since it was tested on a smaller sample set that was not comparable to the other assays.
In the 'Results' section, we changed the titles to highlight the findings in each subsection. Figure 1 and its legend were revised, Figure 2 has been removed and replaced with Figure 3 from the initial version of the manuscript and Figure 3 is new. The order of the text was changed to match the chronology in the 'Methods' section. The percentage differences in Ct values between assays detecting the same gene have been reported.
The 'Discussion' section has been changed to highlight the findings from the dilution series of the positive control RNA and not patients' samples. Two citations have been added in reference to the BGI kit's performance.
Any further responses from the reviewers can be found at the end of the article REVISED the ORF1ab region. We titrated the primers and probes to achieve the three concentrations to be validated relative to the manufacturer-recommended primer and probes concentration of 1X. The three titration points are herein referred to as 0.5X. 0.375X and 0.25X. In all the assays, we carried out duplicate reactions of two positive RNA samples, two negative RNA samples, a non-template control and five 10-fold dilution series of the positive control RNA.

Berlin and EVAg assays titrations
The original protocol employed Superscript III One Step RT-PCR system with Platinum Taq Polymerase for both assays. These reagents were not available in our lab and thus we opted for TaqMan ® Fast Virus 1-step Master Mix (Applied Biosystems) in a 10µl total reaction volume (final working concentration of 1X). To determine the optimal concentrations and volumes of primers and probes, these were varied for both Berlin (Table 1) and EVAg (Table 2) assays while holding the TaqMan master mix, template and total reaction volumes constant.
The EVAg E and RdRp assays were later supplied as a mix of forward and reverse primers and probes (primer-probe set), so these were only tested at 0.375X and 0.25X.

BGI and DAAN kits titrations
The commercial BGI and DAAN kits have primers and probes provided as a premix in the PCR reaction mix and these were supplied in limited amounts. Therefore, the reaction mix (Liquid A) for DAAN was titrated to 0.5X, 0.375X and 0.25X. The recommended volume of the Liquid B (Hot Start Taq DNA polymerase and c-MMLV reverse transcriptase) per reaction for the DAAN kit was 3 µl (1X). Given the limited quantity provided, this was reduced to 0.5 µl (0.16X) across all the varying volumes of Liquid A (Table 3).
Prior use of the BGI assay as per the manufacturer's recommended protocol generated a high signal-to-noise ratio necessitating optimization. The recommended volume of the enzyme mix per reaction for this kit was 1.5µl (1X). However, we initially reduced this to 0.8µl (0.5X) and maintained this volume alongside varying volumes of the PCR reaction mix. We further tested two enzyme mix volumes of 0.5µl (0.33X) and 0.25µl (0.16X), while maintaining the reaction mix volume for the 0.375X concentration (Table 4).

Cycling conditions
All these assays were run on the Applied Biosystems™ 7500 Real-Time PCR System and analyzed using the 7500 software v2.3. The manufacturer's recommended qRT-PCR conditions are indicated in Table 5.
Based on the 10-fold serial dilutions of the positive control SARS-CoV-2 RNA, we established assay-specific Ct value cut-offs to determine a positive result, since the assays have different levels of signal-to-noise ratio. For the analysis of the amplification plots and subsequent data, different baseline points and thresholds were set manually as illustrated in Table 6.

Impact of titrations on the four assays
We assessed the effect of the assay modifications on the overall sensitivity of the results. We did not evaluate the performance of the assays using the recommended manufacturer's volumes, since they had been proven to work during routine testing. The focus was on deriving the smallest volume of reagents required to correctly identify a positive case.
The Berlin E and N gene assays were assessed at three titrations -0.5X, 0.375X and 0.25X. The E gene assay had comparable mean Ct values across the titrations with consistent performance. At 0.5X, all the dilution series of the positive control RNA were detected. The 0.25X titration was not notably different from 0.375X, although the former did not detect the last positive control dilution of 1:10 7 (Underlying data: Data file 1 16 ; Figure 1A). Given that 0.375X of primers and probes detected all the dilution series of the positive control in a consistent trend and with little Ct difference between replicates, we used this to set our cut-off for positivity going forward. The Berlin N gene assay was more consistent, with all the titrations detecting all dilution series of the positive control RNA (Underlying data: Data file 2 16 ; Figure 1B). However, the 0.5X titration showed a more consistent trend in amplification, with little Ct value difference between replicates and between the other titrations. Consequently, we settled for this titration volume for subsequent testing.
The EVAg E and RdRp gene (primer-probe set) assays were assessed at two titrations -0.375X and 0.25X. For the E gene assay, both titrations detected all the dilution series of the positive control RNA. There was no notable difference between the two titrations (Underlying data: Data file 3 16 ; Figure 1C). We settled for 0.375X as our optimal volume of the primer-probe mix since all the dilution series of the positive control were  detected in a consistent trend and with little Ct difference between replicates. For the RdRp gene assay, the 0.25X titration did not detect the last dilution point (1:10 7 ) whereas, the 0.375X titration was more consistent and detected all the dilutions series (Underlying data: Data file 4 16 ; Figure 1D). Thus, the 0.375X volume was chosen for subsequent testing.
The BGI and DAAN premixed kits supplied conducted about 50 and 96 tests per kit, respectively. In the dual-gene target DAAN assay, three titrations -0.5X, 0.375X and 0.25X were assessed. All the replicates of the dilution series of the positive control RNA had little Ct value differences (less than 1) across the titrations for both the N and ORF1ab genes ( Figure 1E and 1F). The 0.5X and 0.375X titrations detected all the positive control RNA dilution series, while the 0.25X volume failed to detect the last positive control dilution of 1:10 7 for both gene targets (Underlying data: Data file 5 and 6 16 ). Consequently, we settled for 0.375X titration for subsequent runs, yielding 252 tests per kit.
The BGI kit produced inconsistent detection results between COVID-19 patients' sample batches (data not shown) when we used 0.5X of the recommended reaction mix. Over 70% of the samples tested were positive (Figures 2A and 2B), leading to a suspicion of false positive amplifications or likely contamination. A confirmatory test with Berlin E and N genes assay did not yield the equivalent number of positives. The titration of the reaction to 0.375X and scaling down the enzyme mix Table 5. Quantitative reverse transcription-PCR (qRT-PCR) cycling conditions for detection of SARS-CoV-2 RNA using four assays.
Step   Figure 1H). Consequently, we settled for the Mix 2 combination (Table 4), where the reaction mix was at 0.375X and the enzyme volume was at 0.16X, yielding 132 tests per kit.

Intra-gene assay performance
The positive control RNA dilution series were used to assess the efficiency of the assays in detecting the same gene targets.
We settled for the 0.375X titration to compare the performance of the assays.
The Berlin and EVAg E gene assays reported mean Ct values within range of each other and with less than 5% difference ( Figure 3A). The Berlin and EVAg RdRp gene assays showed a similar trend ( Figure 3B) with even a lower percentage difference between the mean Ct values being reported (<3%). The N gene assays had up to 13% difference in mean Ct values reported for the dilution series of the positive control. However, the DAAN N gene assay appeared to be more sensitive in detecting the virus since it had lower Ct values than the Berlin N gene assay ( Figure 3C). The DAAN ORF1ab assay also had better sensitivity than the BGI ORF1ab assay ( Figure 3D), although the mean Ct values had a maximum difference of 8.1%. Overall, there were Ct variations across these assays for the serially diluted positive controls ( Figure 3E). The RdRp gene assays appeared to have lower Ct values than the rest of the assays.

Discussion
Our experience from performing over 15,500 tests with limited resources has allowed us to develop a series of adjustments to the primer-probe sets (Charité Berlin and EVAg) and commercial kits (BGI and DAAN) to optimize their use in SARS-CoV-2 testing. This study reports the performance of these assays following modifications on the recommended reaction volumes. Our findings suggest that the reduction in the manufacturers' recommended volumes still allowed for detection of the virus. The 0.375X titration was the optimal volume for all the primers and probes for the gene-specific assays, and therefore recommended for resource-limited settings. The exception was the Berlin N gene assay which worked optimally at 0.5X.
The Berlin RdRp assay generated low slope amplification curves that were characteristic of low-specificity primers. The BGI kit was the only assay whose enzyme volumes were adjusted to maximise the number of samples processed and mitigate the occurrence of false positives. The sensitivity of this assay improved when lower enzyme volumes were used as described above even though a recent publication indicated the impeccable sensitivity of the BGI kit when used according to the recommended volumes 17 . However, according to Public Health England, the false positives in this kit could have been attributed to batch issues linked to different lot numbers 18 . We determined that this assay was more reliable when paired with a confirmatory test from another gene target assay. The DAAN kit was efficient in detection of SARS-CoV-2 RNA, and it had the advantage over the other assays -the dual-gene target for the virus and a human gene internal control that evaluated the integrity of the sample tested and the reliability of the PCR results.
The E gene assays proved more reliable and consistent in detecting true positives. Generally, when comparing the intra-gene assay performance, we expected a variation in Ct values owing to primer design, priming efficiency and master mix differences (salt and pH).
The limitation of this study is the small sample size. These tests were conducted in the early days of the epidemic in Kenya when we had a limited supply of PCR testing kits versus a high number of samples to be tested. Consequently, we leveraged on what was available to determine the optimal parameters in our setting. We acknowledge that some of the findings cannot be generalized, nevertheless, the findings from this study enabled us to maximise the use of the limited kits and reagents available while still maintaining assay performance.

Conclusions
We achieved approximately a 2.6-fold and 4-fold increase in the number of tests per kit for the commercial premixed kits and primer-probe sets, respectively, by adjusting the manufacturer's recommendations on volumes following careful optimization in our laboratory. This enabled us to continuously conduct and support testing in the Coastal region of Kenya and address the challenge of inconsistencies in the supply of testing reagents. We highlight the challenges encountered in the use of the early batches of the BGI kit that we noted was prone to false positives, but this was mitigated by diluting the reagent volumes and by including an additional confirmatory assay. Due to the nature of the qPCR assay, any kit may lead to false positives and thus in addition to negative controls, a dilution series of the positive controls, a confirmatory test and a set threshold must all be included to report a positive

Introduction:
The authors should consider citing Altamimi et al (doi:1002/jmv_26900) that compared the performance of 17 SARS-CoV-2 kits that included BGI and DAAN. In the paper, the authors evaluated the kits in the format recommended by the manufacturer. BJI and DAAN were among the best performers.

Methods:
In Tables 1-5, shouldn't the QuantiFast multiplex RT-PCR Master mix not be included as highlighted?

Component Volume
Nuclease-free water 4.5 4x TaqMan® Fast Virus 1-Step Master Mix/ QuantiFast 2.5 Multiplex RT-PCR +R Kit Master Mix ○ The paper is silent on the performance of the recommended manufacturer's volumes. On page 6 (highlighted section), neat is indicated as 50%. Author's should consider including data on 100%, which is the manufacturer's recommended volume. reagents used. Unfortunately, the QuantiFast Multiplex RT-PCR +R Kit Master Mix was only used in a limited set of assays. Consequently, we excluded these results to eliminate the confusion and misrepresentation. The upscaling of testing in the Coastal counties and the country at large coupled with the crippled supply chain due to the COVID pandemic meant rapid depletion of kits faster than we could replenish our stocks. This prompted us to optimize the kits in a bid to maximize the number of tests we could perform while still maintaining the integrity of the assay. As a result, we did not test the recommended manufacturer's volumes during this optimization.

Discussion
As previously mentioned, this manuscript was first published on 7 th July 2020 and the referenced paper was not out then. However, we have mentioned this in the latest version (L218-220 with hidden tracked changes). It is possible that the kit was improved by the manufacturer considering it was released around 2020. We have documented that this kit's performance improved after the modifications were made. We believe that batch-to-batch variation based on the lot numbers would have resulted in our findings. This has been supported by an assessment done on the kit by Public Health England (see new citation 18 in the bibliography). The manuscript describes the optimisation of real-time quantitative reverse transcription PCR (qRT-PCR) protocols for SARS-CoV-2 detection with objective of increasing the reactions that could be conducted per kit (or primer-probe set) over that of the manufacturer's recommended protocols. The authors used a set of positive and negative patient samples (four in total) and a 10fold dilution series (up to 10 7 ) of extracted virus culture to evaluate assay performance. On the whole two primer/probe sets and two premixed kits were evaluated.
The rationale for this type of assay optimisation is obvious given the widespread shortage of COVID-19 diagnostic reagents across the world because of disruptions in the supply chain.
However, I have a number of general and specific recommendation to improve the manuscript.
In its present form, the optimisation steps are not reported in a clear and sequential format for easy comprehension. Although the processes may have been conducted in disparate steps, it would improve the clarity of the manuscript if the reporting is structured around common themes. An example could be: (1) optimisation to establish lowest concentration of reagents that produce consistent diagnostic performance (usually measured by comparison of the sensitivity and specificity against a reference, which in this case, could be the manufacturers recommended protocol; (2) determination of the combination of gene targets for optimal scoring and confirmation of positive samples; and (3) workflow optimisation to increase assay throughput.
The use of non-standard terminologies to describe the dilutions results in unnecessary complexity for example "we tested three titration points across the four RT-PCR assays as follows: neat (50% of the manufacturer's recommendations), 75% of the neat and 50% of the neat". This should be simplified by designating the reagent concentration in the standard protocol as either 1X or 100%. Then the subsequent dilutions investigated would fractions of the standard and designated as .75X, .5X, .25X as the case may be.
In reviewing the supporting files, some evaluations were conducted with a dilutions series extending only up to 10 4 and others to 10 7 . It is difficult to understand how was the cut off for the Berlin RdRP assay was determined to be 31, given that the limit of detection (LOD) was not reached in the evaluation shown in data files 5 & 6 where the lowest dilution of 10 4 was detected with a CT of 26-27. Figure 1 is particularly difficult to follow given the amount of information contained. This is not helped by ignoring the chronological order in the explanatory text which begins with graph 'D' instead of 'A'.
A large part of the discussion section is devoted to reporting optimisation steps (which would fit better under methods) instead of highlighting the implications and potential impact of these changes, for example, "For quality control during testing, we included in every assay a negative template control, two positive extraction controls (a neat and 1 in 10 dilution) and 2 negative controls placed randomly (but not near each other) across the wells of a 96-well format plate".
Overall, the manuscript will benefit from reporting the optimisation steps in a structured format, simplifying the 'nomenclature' used in describing the dilutions, reviewing the discussions to remove description of the methods, explaining the rationale for the cut-off CTs for the various assays (gene targets) in the absence of determining the LOD and by simplifying Figure 1.

Is the description of the method technically sound? Partly
Are sufficient details provided to allow replication of the method development and its use by others? Yes If any results are presented, are all the source data underlying the results available to ensure full reproducibility? Partly Are the conclusions about the method and its performance adequately supported by the findings presented in the article? Partly Competing Interests: No competing interests were disclosed.
I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

Author Response 24 Feb 2022
Khadija Said, KEMRI-Wellcome Trust Research Programme, Kilifi, Kenya Thank you for your review and comments. Please find appended the responses.
a) Response to comment 1: The manuscript has been restructured and centered around establishing the lowest volumes of the reagents required to generate consistent diagnostic performance while increasing assay throughput. For a clear narrative, the four assays have been introduced in the 'Introduction' section (L25-32 in the hidden tracked version of the manuscript). The optimization method has been initially defined in the 'Real-time PCR assay modifications' section (L57-60) within the 'Methods' section. The adjustments that make up our optimization steps have been explained in two sections based on the type of assayprimer-probe sets (L63-71) and commercial kits (L82-96).
b) Response to comment 2: Noted and modified as suggested. Herein, we refer to the manufacturer-recommended volumes at 1X, and our titration points are referred to by 0.5X, 0.375X and 0.25X to represent 50%. 37.5% and 25% of the manufacturer-recommended volumes respectively. c) Response to comment 3: Included the relevant data for Berlin RdRp comprising all the dilution series tested up to 10 7 . Refer to Updated Data File 9. However, due to issues with Berlin RdRp curves we excluded the Ct cut-off of this assay in Table 6. d) Response to comment 4: The plot in Figure 1 was replaced by a more succinct one. Ensured the chronological order of the accompanying plots and texts was maintained.
e) Response to comment 5: For this part of the discussion section, the idea was to report our experience and emphasize on the importance of including controls in the qRT-PCR assays especially during optimization. However, following the restructuring of the 'Discussion' section this was excluded and the message adheres to the context.