COVID-19 Diagnostic Use Cases
March 20, 2020
In December 2019, astute physicians noted an unknown pneumonia in Wuhan, China. It was soon recognized as a new pathogen, a previously undetected coronavirus, reminiscent of SARS-CoV (cause of SARS) and MERS-CoV (cause of MERS). The WHO has named the new coronaviral disease COVID-19, while the International Committee on the Taxonomy of Viruses has designated the novel coronavirus SARS-CoV-2, since it is a close relative of the previous SARS virus (SARS-CoV). Recent studies indicate that the initial zoonotic transmission to humans was likely from the natural host, bats, mediated by pangolins and snakes (Wang Y, et al.).
The virus has spread remarkably fast with over 200,000 cases of COVID-19 reported in more than 150 countries as of this writing. It continues to spread through human-to-human transmission. Based on a report from the National Health Commission of China, transmission occurs through respiratory aspirates, droplets, aerosols, contacts, and feces. The mean time for incubation is 6.4 days (range 0-24 days). Nosocomial transmission is a major problem in COVID-19. In China, unexpectedly, a large portion of nosocomial transmissions occurred through contacts between clinicians and visitors with no or mild symptoms of COVID-19. Similarly, pre-symptomatic transmission occurred through social gatherings.
There has been a report that viral loads are high even in infected persons without symptoms (Wang Y, et al.). There has been a great deal of effort to develop and deploy tests that can be useful in the detection, control and management of SARS-CoV-2 infections. The current massive test development effort is reminiscent of the West African Ebola outbreak in 2014, but much more global in nature. Then, tests were needed for a number of different uses such as triage, diagnosis, confirmation, and cause of death. During the Ebola outbreak, on the behalf of Grand Challenges Canada, we compiled a list of 10 test Use Cases for Ebola tests with substantially different target use and performance requirements (e.g., site of use, personal protective equipment (PPE) requirements, sensitivity, specificity, etc.). In some situations, the tests meant for one application were of quite limited use in another application, whereas others were more broadly useful in a variety of settings. This was due to the fact that tests that required relatively low clinical performance (e.g., in a site with a current epidemic) were not useful where higher clinical performance was needed (e.g., in a site without known current infections); but typically a test designed for a high performance need was also useful in a site of lower need; that is, high clinical performance tends to be the lowest common denominator for multiple applications. Unfortunately, many organizations developed low clinical performance tests for the Ebola epidemic that were not useful in the post-epidemic world. Other product requirements also differed across Use Cases, such as site of testing, procedures for sample collection, types of samples that could be employed, and impact of testing while wearing PPE. We see similar issues developing in the COVID-19 epidemic, which led us to provide this Halteres Newsletter concerning six SARS-CoV-2 testing Use Cases, highlighting the comparison, contrast and implications for each. Additional information concerning these Use Cases and others - such as surveillance and post infection immunity - can be found at the linked Use Case Tables.
So far, most testing organizations interested in addressing the COVID-19 epidemic are looking at at-risk populations such as travelers from epidemic regions, contacts of infected persons, healthcare and other emergency professionals and persons with suspicious fever and a dry cough. As the epidemic expands, we will see additional populations to be tested and sites for them to be tested within. We believe that there is a great need for more point of care testing, even though we recognize the challenges for preferred biomarkers (e.g., RNA, antigens, IgM, IgG, metabolites, cells) within useful assay formats (e.g., PCR devices versus immunoassays without instruments). It is astonishing to see how many companies are already involved in most of these types of SARS-CoV-2 assays (FIND). Home sample collection and transport to a lab is useful, but true point of care tests would offer many advantages as presented below. Also, we wanted to expand our thinking to the possible future where COVID-19 becomes an entrenched and recurrent problem. We hope that this does not occur; however, if we don’t consider this possibility now, the diagnostics community might very well fail to develop tests that serve the long-term needs of our global healthcare and surveillance systems.
In this document we use the term “endemic” perhaps somewhat loosely. We are not differentiating between “regularly found within a population or location” and “eliminated from a population or location and reintroduced”. Please bear with us.
Prior to developing assays and systems, it is essential that test developers understand the details of Use Cases for SARS-CoV-2 testing including who will be tested, by whom, the site of testing, using which samples, under what conditions, and what the minimum acceptable clinical performance requirements are likely to be. The most important component of a Use Case is the intended use; that is, what is the clinical decision that will be enabled with the test and in what subject population. Given the early phase and novelty of the COVID-19 epidemic, many things that are known in other disease states are not generally known, such as the presence of virus in body compartments and fluids over time, the breadth of host responses, and the best clinical samples to use (Wang W, et al.). For the Ebola Use Cases, we presented information concerning the availability and potential utility of specific biomarkers. Given the early nature of this disease, we will only briefly address those issues in the Conclusions section. Multiple sample types are used today: nasal and nasopharyngeal swabs, sputum, bronchial lavage, urine, feces, and blood. As we learn more, some sample types will become preferred, while others are discontinued. We address this to a greater extent in the Use Case Tables. We present here approximate clinical performance needs and price targets but have not yet supported these estimates for a SARS-CoV-2 test with robust health economic models.
As far as we know, the scenarios we present concerning triage and confirmation testing are not in common use except for very broad screening tests such as taking a person’s temperature then sending nasal swabs for RNA testing. We believe that there are tests available, or near the market, that can be used as more useful triage tests that could enable the options presented here.
The intended use is to determine if a symptomatic individual in an epidemic setting has a reasonable likelihood of a current infection warranting temporary isolation pending confirmation testing (Use Case 4). For Group 1 Sites, use examples include locations with known infections such as cruise ships, hospitals, assisted living centers and quarantine facilities. These are locations where multiple people are at least temporary residents (days). Group 2 sites include public health clinics, primary care facilities, urgent care clinics, and emergency departments, where symptomatic individuals will visit temporally unless they are found to be positive. Note that primary care as used here includes community-level settings in low to middle income country (LMIC) regions referred to as Level I and Level 0. These two groups of sites offer different issues for testing. In Group 1 sites, residents can be isolated and confirmation testing can be conducted remotely; however, the Group 2 sites are more problematic since unless the confirmation testing can be conducted locally and rapidly, it will not be possible to keep patients on site long enough to isolate them to prevent potential transmissions. If the Group 2 site were associated with a quarantine location (e.g., emergency department in a hospital) it would be possible to quarantine triaged patients until confirmation-testing results were received within a few hours. In any event, a triage test that permits the majority of symptomatic patients to leave if they have a negative result (low probability of a false negative result; high sensitivity and high negative predictive value) would limit the need to isolate and test for confirmation of infection. On the other hand, the test requires an acceptable false positive rate (high specificity and high positive predictive value) so that few people are unnecessarily quarantined. Even in an epidemic setting, it is likely that most persons with respiratory disease symptoms do not have COVID-19, likely >99%. The balance of clinical performance characteristics and test requirements should be analyzed and justified within an impact model including economic parameters (e.g., hospital and testing costs) and clinical measures (e.g., QALYs). Please contact us for details if you are interested.
The triage test would need to be in a point of care format. Given the sites of use, an instrumentfree solution would be desirable, but a small processing/reading instrument would also be possible for most sites. In either case, a test or system designed to meet CLIA waiver requirements is essential. It is also highly desirable to use information and communication technologies (ICT) to permit test results to be reported remotely, for example, to national health authorities or the WHO. End user price targets for the test are likely to be <$25 for the US, but considerably less for low and middle-income countries (LMICs). For comparison, prices for Ebola PCR-based tests used during the 2018 outbreak in the Democratic Republic of Congo ranged from $10-$79 (estimates) depending on whether they were used in batch mode (20 tests) or as single tests (Cnops, et al.).
We must remember that healthcare professionals today send samples out and wait for results. In any scenario where a test will be performed at the point of care, it is possible the trained professionals will be wearing substantial PPE. In this case, they must be able to perform the test while gloved, gowned and masked, which has implications for the type of sample that can be obtained (e.g., finger prick blood or nasal swabs while wearing gloves), how it can be processed (e.g., micro-capillary manipulation observation while wearing a mask), and the operator interfaces on the devices used (e.g., touch screen while wearing gloves).
The potential use of a triage test in an epidemic is tied to the availability of an acceptable confirmation test. Without rapid availability of confirmatory testing results, quarantine restrictions, even if temporary, will have significant, perhaps unrealistic, impact on the sites and the individuals tested. Also, healthcare professionals will need to maintain PPE requirements until the individual is confirmed to be negative. If the confirmation test can be performed locally, the second set of sites with temporary visiting capacity would benefit greatly. Until there are rapid point of care confirmation tests available, there are likely to be a large number of sample send outs for lab tests, presenting logistical challenges
The intended use is to determine if a symptomatic individual in an endemic or epidemic setting has a reasonable likelihood of a current infection warranting temporary isolation pending confirmation testing (Use Case 4). It is not clear that the endemic scenario will become a reality. Is Wuhan now an endemic site? Or, like SARS-CoV, will SARS-CoV-2 disappear? Although no one can say, for the purpose of discussing the implications for test design, we will assume that COVID-19 will remain a recurring threat, perhaps akin to seasonal flu. We will not address the epidemic scenario again here, but we assume that a test that works in the endemic setting will work in the epidemic setting, if the PPE encumbrances are considered.
The sites of testing are the same as for Use Case 1, except that there will likely be a much broader set of sites that will wish to test. It is also likely that more testing will occur in Group 2 sites (temporary visits) than the Group 1 sites (residents or inpatients). Regulatory requirements, for example the need for CLIA waiver, will again be important considerations. Most of the individuals presenting with respiratory symptoms at the sites are less likely to be infected with SARS-CoV-2 than with other more common respiratory pathogens such as common cold or flu viruses than persons presenting in Use Case 1. The need for high sensitivity remains, but there is now a need for higher specificity since in a low prevalence (e.g., <1%) environment, the majority of positive results could be false positives. For example, with a specificity of 95%, five false positives would be expected for each true positive person. At a 99% specificity, the ratio of false positives to true positives would be 1 to 1, a far more acceptable result. Price targets for the test are likely to be less than for epidemic settings, <$15 for the US, but considerably less for LMICs.
It is likely that isolation will become a major annoyance for persons tested, healthcare professionals, and facilities, especially when PPE and patient isolation is required during the clinical encounter and until the test results are received. Therefore, confirmation testing would need to be very rapid. In fact, it might not be practical to use triage testing in some settings like emergency departments. See Use Case 5 as an alternative. Use Case 3: Triage of at-risk pre-symptomatic and symptom
The intended use is to determine if an at-risk individual with or without symptoms in an endemic setting has a reasonable likelihood of a current infection warranting temporary isolation pending confirmation testing. The sites for testing symptomatic individuals remain the same as Use Case 2; however, the site for testing pre-symptomatic persons is far more problematic. General testing of the population is costly and unacceptable for a variety of reasons including access, adherence, awareness, training and cost. In an endemic setting, occasionally a cluster of new symptomatic cases could appear and initiate a broad testing protocol of recent contacts. A point of care triage test could be used to process and identify infected, pre-symptomatic persons for temporary isolation pending rapid confirmation testing, potentially even at ports of entry or at large public gatherings. Send-out testing taking days would probably not be useful except possibly within confined populations. With very high sensitivity and specificity triage testing (e.g., >99%), it would be possible to eliminate confirmation testing altogether and initiate extended quarantine immediately upon test completion. Such a triage test would be nearly indistinguishable from a diagnostic test (Use Case 5). Price targets for the test are likely to be ~$25 for the US since there would be a premium value for identifying pre-symptomatic patients, but considerably less for LMICs.
The intended use is to confirm that an individual is currently infected with SARS-CoV-2 after positive results from triage testing. Sites of testing would be dependent upon the sites of triage testing. Where the patients are residents or inpatients, patients can remain confined for a few days enabling sample send out; however, in sites of temporary visits, local confirmation testing is preferred, for instance, in a qualified lab near an emergency room or primary care facility. Rapid turnaround of two hours or less would be preferable. It is possible that a mobile lab could be employed. The test could be conducted by a trained laboratorian, but it is preferable that a trained nurse perform testing.
Sensitivity and specificity goals are less than for a diagnostic test, since after triage testing the population will be greatly enriched for infected persons. If the triage test had specificity of 95%, only one in 20 persons would be false positive in the triaged population to be confirmed. Price could be as high as $100 for testing, depending in part (rightly or wrongly) on the assay technology used, but preferably lower. Again, if the triage test were of high enough clinical performance it would not be necessary to use a confirmation test.
The intended use is to diagnose a symptomatic individual with a SARS-CoV-2 infection in an endemic or an epidemic setting. Sites include emergency rooms, urgent care clinics, hospitals, and primary healthcare facilities. The test could be CLIA-waivable for a minimally trained healthcare worker or a moderately complex system for a small lab performed by a laboratorian. The turnaround time needs to be no more than one hour, but preferably 15 minutes or less.
Sensitivity and specificity will need to be quite high (≥ 99%). A false negative result, particularly in the elderly, could result in high morbidity and mortality rates, while increasing the risk of transmissions. False positive results would lead to unnecessary and costly quarantine or hospitalization. Cross reactivity with other respiratory pathogens would be highly problematic. Price targets should be $30-$50 in the US, probably less in LMICs.
The intended use is to diagnose an individual with other upper or lower respiratory infections, such as Influenza A, Influenza B, RSV, pneumonia, bronchitis or SARS-CoV-2 infection in a COVID-19 endemic or epidemic setting. Optimally, the test would also include the four coronaviruses associated with upper respiratory infections and the common cold: HCoV 229E, NL63, OC43, HKU1. The Use Case testing sites and users are the same as Use Case 5, but in this case will require additional high-performing tests run simultaneously. A test result indicating that a person is not infected with SARS-CoV-2 but is infected with an alternative respiratory pathogen would have a great advantage. Even in the absence of a therapy for SARS-CoV-2, this would lead to great comfort for both the patient and the caregiver, although a flu diagnosis shouldn’t be treated lightly. Note that if the test is designed for an endemic setting, it should also be useful in the epidemic setting, since the performance needs are higher for an endemic setting. Acceptable test costs are likely somewhat more than Use Case 5 at $30-$120, depending on the complexity of the test and the assay technology used.
There is much to consider before embarking upon test development. In particular, we have found it beneficial to consider the full breadth of Use Cases for the envisioned test and from them to deduce the set of requirements that cover the majority of the Use Cases that will be of the greatest clinical and economic value. Typically, it is impractical to cover all of the Use Cases with one test. In the case of tests for SARS-CoV-2 infection, we investigated triage, confirmation and diagnostic testing. Triage testing is completely dependent upon the characteristics of the confirmation test. If the individuals to be tested are in populations confined for at least a few days, send out confirmation testing is a reasonable option. On the other hand, send out confirmation is impractical for other settings unless tested individuals will be self-isolated at home. In either case, the increased time to diagnosis, the need for PPE for healthcare professionals, and the isolation of patients until testing results are problematic.
Acceptable clinical performance for triage testing depends upon the characteristics of the local population. In an epidemic setting, false negative results are a major concern due to the potential to miss infections leading to possible resultant morbidity, mortality and continued transmissions. False positive results could be tolerated if they lead to unnecessary isolation of persons with mild symptoms, whereas if the individual has severe symptoms, they are in need of care no matter what the cause. In contrast, under endemic conditions with a very low rate of current infection, the false positive rate is of substantially greater concern and could initiate a large-scale unnecessary response to limit spread of a new pocket of cases. Rapid and inexpensive confirmation testing would prevent overreaction. Under the best of circumstances, the triage test would have sensitivity and specificity above 99%, in which case it might not require confirmation testing and is then probably the same as a diagnostic test. The impact of lower clinical performance should be subjected to substantial health economic modeling to determine the impact of the performance versus value tradeoffs to determine the minimum specifications that would be acceptable.
At the time of writing, there is widespread concern about the availability of SARS-CoV-2 testing. The majority of tests available or in development are viral RNA-based (e.g., rtPCR, LAMP); however, several immunoassays for SARS-CoV-2 antigens, IgM and IgG are also already on or near the market, some in rapid diagnostic test (RDT) formats (FIND). We speculate that many of these are based upon SARS-CoV antigens and not SARS-CoV-2 specific proteins. Sequencing (NGS) tests are also available or under development and will continue to provide much needed information concerning viral variation over space and time. In the near term, we believe NGS tests should be the “gold standard” that other tests are measured against until we have sufficient data to know that other RNA or antigen tests were designed properly. Test assay capture and labeling reagent design is always a difficult problem when new pathogens are involved.
Although most SARS-CoV-2 testing for diagnosis and confirmation testing today is in central laboratories, we need to continue the effort to provide point of care systems with sufficient performance for local confirmation and diagnostic testing far more quickly. Triage testing could be based upon combined clinical presentation and basic clinical test patterns (e.g., CBC and metabolic tests) for COVID-19 versus other respiratory diseases, and should be thoroughly investigated as one option to provide far greater availability and lower cost than current testing, which can be followed with more complex and expensive confirmation tests like RNA assays. Testing costs have not been a substantial concern so far since there is such a push to provide testing of any kind. But with time, costs will become a major concern, and those that consider costs now will become preferred providers over time, assuming equivalent performance and ease of use.
We recommend that test developers use our considerations as a start to understand the full set of Use Cases that could be valuable and their implications for the design of the right tests. Once Use Cases are analyzed, health economic models and target product profiles can be used to justify and describe the test and system requirements. Let’s not make the mistakes of the past and provide less than what is really needed to detect, control and manage COVID-19, and with time, eradicate SARS-CoV-2 altogether.
Use Case Considerations
We present six Use Cases for SARS-CoV-2 infection testing here:
- Triage of symptomatic individuals in an epidemic setting
- Triage of symptomatic individuals in endemic setting
- Triage of at-risk pre-symptomatic and symptomatic individuals in endemic settings
- Confirmation testing
- Diagnosis of symptomatic individuals in endemic or epidemic settings
- Differential diagnosis in endemic or epidemic settings