Important work published in ASCO JCO Precision Oncology suggest technical factors to be the major source of discordant ctDNA results
One major area of concern regarding wide use of liquid biopsy for therapy selection in precision medicine has been the discordance between the genetic analysis of tissue biopsy (the accepted “gold standard”) and mutation detection in plasma. While the need for liquid biopsy is clear in cases of insufficient tissue availability or the need for ongoing monitoring, if the mutation analysis from plasma is not reliable and trustworthy wider adoption will be certainly hampered.
Several publications give cause for concern – and uncertainty
Several publications (see references 1, 2 and 3 below) showed sharp differences between tissue genotypes and mutations detected in plasma; for example, in Kuderer et al. only 22% of the variants they detected in tissue (10 of 45) were detected in plasma.
Apart from technical factors, a few biological factors were attributed to the cause of false- negative (FN) and false-positive (FP) results in plasma. For the former, FNs were attributed to potentially be from ‘non-shedding’ tumors; these tumors are estimated in frequency to be found 20% to 30% of the time, (see reference 4). For FPs, white blood cell DNA can undergo clonal hematopoiesis, where its affect in plasma is called clonal hematopoiesis of indeterminate potential or CHIP.
Naturally in tissue, inherent tumor heterogeneity can be the source of FN in tissue genotyping. Particularly relevant with development of resistance while under therapy, tissue sampling issue may not capture the entire genetic variation of the entire tumor whereas liquid biopsy has that potential.
The aforementioned references, with discordance rates as high as 55% (reference 3), has caused caution around wide adoption of liquid biopsy. An influential report from ASCO/CAP (reference 5) “has put a damper on what some view as an overenthusiasm around liquid biopsy tests.” (GenomeWeb Premium, subscription required.)
A plasma ctDNA sequencing ‘bakeoff’ previewed in 2017
Over the course of 2017 and 2018, a group from AstraZeneca presented posters at various conferences (including the annual American Association for Cancer Research in 2017 and at an influential FDA/AACR workshop later that Fall) showing preliminary results of what they called a ‘Plasma-Seq Bakeoff’.
They took 24 samples from cancer patients to four different liquid biopsy test providers, and then comparing the results to each other as well as to matched tumor / normal tissue sequencing data. Of the 24 samples, 7 were breast cancer samples, 12 were lung cancer samples, 4 were ovarian cancer samples and 1 was prostate cancer sample; as the researchers wanted to challenge the test providers there were 8 Stage I samples and 13 Stage II samples; thus 21/24 samples were early stage. In addition, while the >8 mL of plasma from each patient was processed identically (collection in K2EDTA tubes, double- spun within 4 hours of collection), only 2 mL were provided to each test provider.
True positives were defined as a tumor tissue mutation matching a plasma mutation, or two plasma mutations matching each other regardless of the tissue data.
The Main findings of Stetson et al JCO Precision Oncology 2019 (see reference 6)
The researchers found ‘substantial variability among the ctDNA assays’, with sensitivities ranging from 38% to 89% and Positive Predictive Values (PPVs) ranging from 36% to 80%. The FP’s identified were novel mutations absent from somatic mutation databases. Of the 56 unique variants identified by all four ctDNA assays, a full 68% (41/56) were from technical discordance, rather than a biological source (e.g. the aforementioned CHIP).
Importantly, their analysis included the measure of minor allele frequency (MAF) from the plasma samples, and as BAM files were provided back to the researchers from the test providers a detailed and consistent bioinformatic analysis be undertaken.
They provide a striking figure (Figure 1) where the variant concordance with TP, FP and FN are all plotted across four different vendors, and the vast majority of FPs and FNs are less than 1% MAF.
Here is the Supplementary Figure 1 from the Supplemental Information available online and the accompanying legend.
Supplementary Fig. 1 | Variant tile plot. True positives (TP, green), false negatives (FN, orange), false positives (FP, red) not detected (blank) and not reported (gray) are plotted one variant per line. Not detected are variants not found in a vendors’ assay.
If you look across all the red, you’ll note the majority of these FPs are well below 1% in measured MAF, with a few at 1.5% or 2%. We have written before about the high cost of FPs and FNs, and this is a serious issue.
The authors comment: “50% (22 of 44) of all TP somatic variants had a VAF of less than 1%, underscoring the importance of analytically validated assays with sensitivity below
1%.” (Stetson et al. Reference 6)
How sensitive is your assay for liquid biopsy?
A relevant accompanying editorial, “Does Testing Error Underlie Liquid Biopsy Discordance?”, concludes with this:
“As the use of plasma NGS becomes increasingly widespread in cancer care, there remains a clear need for concordance studies such as this one. Future studies would ideally focus on actionable variants and would be limited to advanced cancer. In addition, we have found that orthogonal benchmarking against an established assay, such as digital droplet polymerase chain reaction, is a powerful way to establish a reference point for such analyses.”
At Sysmex Inostics we have both enhanced digital PCR and NGS-based liquid biopsy for testing patient samples, and a worldwide laboratory presence with clinically-validated assay sensitivity (depending on the particular mutations and assay technology) down to 0.03%.
- Thress K., Barrett J.C. et al. Lung Cancer 90(3):509-15 (2015). EGFR mutation detection in ctDNA from NSCLC patient plasma: A cross-platform comparison of leading technologies to support the clinical development of AZD9291. PMID:26494259
- Kuderer N.M., Blau C.A. et JAMA Oncol. 3(7):996-998 (2017). Comparison of 2 Commercially Available Next-Generation Sequencing Platforms in Oncology. PMID:27978570
- Jovelet C., Lacroix L. et a Clin Cancer Res 22(12):2960-2968 (2016). Circulating Cell-Free Tumor DNA Analysis of 50 Genes by Next-Generation Sequencing in the Prospective MOSCATO Trial. PMID:26758560
- Abbosh C. and Swanton C et Nat Rev Clin Oncol 12:1344-1356 (2017). Early stage NSCLC – challenges to implementing ctDNA-based screening and MRD detection. PMID:29968853
- Merker J. and Turner N.C. et al. J Clin Oncol 36(16):1631-1641 (2018). Circulating Tumor DNA Analysis in Patients With Cancer: American Society of Clinical Oncology and College of American Pathologists Joint Review. PMID:29504847
- Stetson D. and Dougherty B.A. JCO Precision Oncol (2019). Orthogonal Comparison of Four Plasma NGS Tests With Tumor Suggests Technical Factors are a Major Source of Assay Discordance. https://ascopubs.org/doi/10.1200/PO.18.00191