Positive predictive value metrics for multicancer detection tests

Abstract

Introduction

Multicancer detection (MCD) tests generally target selected cancer sites, but may also have some detection ability at other sites; they additionally typically give a predicted cancer signal origin (CSO) for positive tests. Positive predictive value (PPV) is a widely used metric assessing performance of single-cancer screening tests. The aim here was to define PPVs in a number of ways in the MCD context, create a prototypical MCD test, and show the qualitative and quantitative properties of these PPV metrics.

Methods

PPVs were defined based on which subjects were included in the numerator and denominator. PPV_ALL, PPV_T, and PPV_CSO each have as denominator all subjects with positive screens, and as numerators all subjects with any cancer diagnosis, diagnosis at any targeted site, and diagnosis at the predicted CSO site, respectively. Predicted CSO site-specific PPVs were defined similarly, except with denominator subjects with a given cancer site as predicted CSO. MCD performance data taken from a case-control study and population incidence rates were used to determine sensitivity, specificity, CSO accuracy, and cancer prevalence values of a prototypical MCD test for which PPV were calculated. A range of sensitivity rates for nontargeted cancer sites were assessed.

Results

PPV_ALL increased and PPV_T decreased as sensitivity for nontargeted sites increased. Predicted site-specific PPVs depended primarily on cancer site CSO accuracy, not on site population prevalence.

Conclusions

PPVs can be defined in multiple ways for MCD tests. Their usefulness depends on the clinical context. Companies offering MCD tests can consider discussing the clinical usefulness of various PPVs with clinicians to decide what metrics to include on test reports.

Keywords

Cancer screening multicancer detection tests positive predictive value cancer signal origin

Introduction

In the past several years a new paradigm in cancer screening has emerged, namely, that of multicancer detection (MCD) tests.^1,2 These tests, which utilize diverse underlying technologies, have the ability to detect multiple types of cancers simultaneously, and may be used to screen asymptomatic persons for cancer. A major issue with the use of MCD tests for screening is managing the diagnostic work-up of positive MCD screens, as well as interpreting their reported results.^3,4 MCD tests report a binary positive/negative finding for a detected cancer signal; most also report a primary (and sometimes secondary) cancer signal origin (CSO) prediction for the likely organ site of the cancer. Additionally, MCD tests generally specify a group of cancers that the test is designed to detect, so-called “targeted” cancer sites, and reporting of a CSO prediction is restricted to these sites. However, there is evidence that many MCDs also have the ability to detect, with varying levels of sensitivity, tumors from nontargeted sites.^5–7 Positive MCD test results among those with nontargeted cancers complicate the clinical decision-making process, in that these individuals will always have incorrect primary (and secondary) CSO predictions, and clinicians would have to decide on how and whether to continue the work-up. Given these considerations, interpretation of MCD test results and the subsequent diagnostic work-up is much more complex than that of traditional single-cancer screening tests.

With standard screening for a single cancer, such as low-dose computed tomography for lung cancer or mammography for breast cancer, a basic metric available to clinicians managing a positive screen result in a patient is the positive predictive value (PPV) of the test in the setting of interest.^8,9 In the single cancer screening setting, the PPV is defined as the proportion of positive screens for which the screened-for cancer is diagnosed (within a given timeframe). In single cancer-site screening where disease prevalence is low, the PPV is generally relatively low, usually below 10%. However, PPVs for MCDs have the potential to be higher, since the prevalence of a large group of cancers would collectively be greater than that of a single cancer. PPV is generally related to disease prevalence, and increases with such prevalence, all other things being equal.

Due to increased complexity of MCD results, there are many possible ways to define the PPV in the MCD context. For example, PPVs could be defined based on the proportion of positive screens for which any targeted cancer is diagnosed, or for which any cancer (targeted or not targeted) is diagnosed. For those MCDs with CSO predictions, the possible types of PPVs are multiplied. The PPV could be defined as the proportion of positive screens for which the primary predicted CSO cancer is diagnosed. Alternatively, it could be defined as the proportion of positive screens with a given predicted primary CSO (e.g. lung) for which the predicted cancer type is diagnosed, or any type is diagnosed. Some PPV values have been reported for MCDs, generally as the proportion of all positive screens with any subsequent cancer diagnosis, regardless of whether the site was targeted or not.^10,11

In this article, we first define PPVs of possible interest in the MCD context and propose clinical scenarios in which they may be useful, that is, use cases. Next, to illustrate the properties of these PPVs with a concrete example, we use data from a case-control study evaluating a precursor to the Grail Galleri MCD test to develop a prototypical MCD test with similar performance characteristics (e.g. sensitivity, specificity, and CSO prediction accuracy values).¹² Finally, we compute values of the defined PPVs for this prototypical MCD test in an average-risk screening population and assess the effects of various parameters on these PPVs.

Materials and methods

Defining PPVs

Table 1 shows definitions of the different PPVs of interest with an MCD screening test. PPVs are categorized by the denominator and numerator of the proportion that defines a PPV. The denominator can be either all positive screens, or all positive screens with a given cancer site (e.g. lung) as the primary predicted CSO. The numerator can be either the number of positive screens with any cancer diagnosis, a cancer diagnosis at any targeted cancer site, or a cancer diagnosis at the primary predicted CSO site. The diagnosis is required to be within a given timeframe of the positive screen (typically equal to the screening interval). By targeted sites, we define those as sites that are possible choices for the CSO prediction algorithm. For MCD tests without a CSO predicter, targeted sites could be defined as those listed in the labeling of the test as sites that it can detect.

Table 1.

Definitions of types of PPVs.

PPV type—denominator	PPV type—numerator	Abbreviation	Definition	Use cases
All positive screens	Cancer diagnosis at any site	PPV_All	Proportion of all positive screens that have diagnosis of cancer at any site	Shared-decision making prior to undertaking screening; clinical decision making after receiving test result for MCDs w/o CSO
	Cancer diagnosis at any targeted site	PPV_T	Proportion of all positive screens that have diagnosis of cancer at any targeted site	Shared-decision making prior to undertaking screening; clinical decision making after receiving test result for MCDs w/o CSO
	Cancer diagnosis at primary prediction site	PPV_CSO	Proportion of all positive screens that have diagnosis of cancer at the primary predicted site	Shared-decision making prior to undertaking screening
Positive screens with specific primary CSO prediction site (e.g. lung)	Cancer diagnosis at any site	PPV_Site:All	Of all positive screens with the specific primary predicted site, the proportion with cancer diagnosis at any site	Clinical decision making after receiving test result
	Cancer diagnosis at any targeted site	PPV_Site:T	Of all positive screens with the specific primary predicted site, the proportion with diagnosis of cancer at any targeted site	Clinical decision making after receiving test result
	Cancer diagnosis at the specific primary prediction site	PPV_Site:Site	Of all positive screens with the specific primary predicted site, the proportion with diagnosis of cancer at that site	Clinical decision making after receiving test result

CSO: cancer signal origin; MCD: multicancer detection; PPV: positive predictive value.

Use cases

Table 1 also shows example use cases for the PPVs. For those PPVs with a denominator of all positive screens (e.g. PPV_ALL and PPV_T), a use case is for counseling or shared-decision making prior to undertaking screening. These PPVs give individuals potentially undergoing screening an idea of the implications of receiving a positive result on an MCD test. For the PPVs with denominator of a specific predicted TOO (e.g. PPV_Site:Site), a use case would be clinical decision making about the diagnostic work-up, involving both the physician and patient, after receipt of a positive MCD test result with the given predicted CSO. Additionally, for those MCD tests without a TOO prediction, clinical decision making after a test result would also be a use case for PPV_ALL or PPV_T.

Computing PPVs for a prototypical MCD test

If adequate data were available from a large prospective screening study of an MCD test, then PPVs for that test could be estimated directly. However, such data are not available. Therefore, we utilized data from a case-control study to construct a prototypical MCD test from which the various PPVs could be calculated. The Circulating Cell-free Genome Atlas (CCGA) study evaluated the performance of an MCD test that was an early version of the Grail Galleri test.¹² The study evaluated samples from individuals with over 20 types of cancer, taken at the time of diagnosis, as well as cancer-free controls. The primary outcomes were test performance (i.e. sensitivity and specificity) and CSO prediction across the range of cancer types. A prespecified secondary analysis examined test performance in a subset of 12 cancer types for which prior studies had indicated high rates of cancer signal detection.

Calculating the defined PPVs for a prototypical MCD test requires specifying the test's cancer site-specific sensitivity and primary CSO prediction accuracy rates, as well as its specificity. These were taken from the CCGA study. Calculating PPVs from case-control data also requires estimates of cancer prevalence in the appropriate population; annual incidence rates obtained from population-based data for the appropriate screening age-group (50–74) were used to estimate prevalence.¹³

For the choice of targeted cancer sites for our prototypical MCD test, we used the 12 prespecified cancer types of the CCGA study: anal, bladder, colorectal, esophageal, head and neck, liver, lung, lymphoma, ovarian, pancreatic, plasma cell, and stomach. For the other cancer sites outside of these 12, there were adequate data from the CCGA study on sensitivity and CSO accuracy for some sites, including breast and prostate, but little or none for other sites. For the purpose of illustrating the properties of the PPVs, we chose to not directly use the CCGA study data for any site outside of those 12, but instead evaluated a range of aggregate sensitivity values for the nontargeted sites (i.e. all sites outside the above 12), using a plausible range of 5% to 40%.

There was no information published from the CCGA study on the distribution of predicted primary CSOs for false positives and limited information on predicted CSOs for nontargeted sites; these affect the site-specific PPVs. Therefore, as a default assumption, we assumed that these were proportional to the detection rates for the targeted cancer sites. For example, if lung cancers represented 20% of the detected cancers among the targeted cancer sites, then 20% of the false positives were assumed to have a primary predicted CSO of lung cancer. We used a similar assumption for the primary predicted CSOs for nontargeted sites, and for incorrect predictions among the targeted sites. However, as a sensitivity analysis, we varied the distribution of predicted CSOs among false positives to assess its effect on the various PPVs. Specifically, for selected cancer sites, we doubled or halved that proportion relative to strict proportionality, and changed the proportions for the other sites accordingly, leaving their relative proportions unchanged.

More details on calculating the PPVs are given in the Appendix. As part of these calculations, we also computed a corresponding confusion matrix for positive screens, showing the actual versus predicted cancer tissue of origin sites. Table 2 shows the parameter values of the prototypical MCD test, derived from the CCGA study, that were used in the analysis. Note the calculated PPV values are not meant to represent those of any version of the Grail Galleri test, but only to show the properties of PPVs for a MCD test with plausible performance characteristics.

Table 2.

Incidence, sensitivity, specificity, detection rate, and CSO accuracy by cancer site—base case analysis.

	Incidence rate (% of all cancers\| % of all targeted cancers)^a	Sensitivity, %^d	Detection rate (% of all targeted cancers)^c	Primary CSO accuracy, %
Targeted sites: cancer type
Anal	6 (1.4\|0.6)	75	5 (1.5)	78
Bladder	40 (9.3\|3.8)	50	20 (6.6)	88
Colorectal	81 (18.8\|7.6)	79	64 (21.0)	99
Esophageal	12 (2.8\|1.1)	73	9 (2.9)	88
Head & neck	32 (7.4\|3.0)	82	26 (8.6)	93
Liver	27 (6.3\|2.5)	91	24 (8.1)	93
Lung	119 (27.6\|11.1)	67	79 (26.2)	92
Lymphoma	34 (7.9\|3.2)	56	19 (6.3)	99
Ovarian	12 (2.8\|1.1)	75	9 (3.0)	76
Pancreatic	29 (6.7\|2.7)	77	22 (7.4)	84
Plasma cell	20 (4.6\|1.9)	70	14 (4.6)	100
Stomach	19 (4.4\|1.8)	64	12 (4.0)	88
All	431 (40.5)	71^b	303 (100)	93^b
Nontargeted sites	632 (59.5)	Range 5–40	Range 32–253	NA
All sites	1063 (100 \| 100)	—	Range 335–556	NA
	Proportion of screens	False positive rate, %		—
No cancer	98.94	0.5	NA	NA

Per 100,000 per year. For ovarian, incidence rate is for total population, not just women.

Weighted average, by incidence for sensitivity and by detection rate for CSO accuracy.

Detection rate is incidence rate times sensitivity (units are per 100,000 per screening round).

Sensitivity was estimated as the average of early stage (I–II) and late-stage (III–IV) sensitivity.

CSO: cancer signal origin.

Results

Figure 1 shows the values of three PPVs, namely, PPV_All, PPV_T, and PPV_CSO1, for the prototypical MCD test. PPV are plotted across a range of aggregated sensitivity values for nontargeted sites (SE_NT). PPV_T and PPV_CSO both decreased as SE_NT increased, by around 7 to8 percentage points across the range (5%–40%) of SE_NT values. In contrast, PPV_All increased as SE_NT increased, going from 40% (SE_NT= 5%) to 53% (SE_NT= 40%). The proportion of all detected cancers that were targeted decreased as SE_NT increased, from 91% (SE_NT= 5%) to 55% (SE_NT= 40%).

Figure 2 shows the confusion matrix, with SE_NT set at 20%. Of 10,000 total positive screens, there were 5406 false positives and 10,000–5406 = 4594 true positives, giving PPV_All of 45.9%. Along the diagonal line with the numbers of correct primary CSO predictions, there are 2984 cases, giving a PPV_CSO of 29.8%.

For context, one can consider the higher level of all screens (negative or positive) in a population, not only positive screens. As SE_NT increases, the number of positive screens increases, from 830 (SE_NT= 5%) to 1050 (SE_NT= 40%) per 100,000, due to increased sensitivity for nontargeted cancers. The number of true positives for targeted cancers (the numerator of PPV_T) remains the same, at 304 per 100,000, but due to the increased number of positive screens, which comprise the denominator of PPV_T, PPV_T decreased from 36.6% (304/830) to 28.9% (304/1050) over this SE_NT range.

Figure 3 shows values for the cancer site-specific PPVs (at fixed SE_NT= 20%). For all three PPVs (PPV_Site:All, PPV_Site:T, PPV_Site:Site), there was a near-linear, modestly increasing relationship between the site's PPV value and its primary CSO accuracy, with an overall PPV range across sites of about 5%. The PPV was essentially independent of the cancer site's prevalence. For example, the relatively low in prevalence liver cancer had slightly higher PPVs than the much more prevalent lung cancer. Increasing the level of SE_NT led to decreases in PPV_Site:Site and PPV_Site:T, but increases in PPV_Site:All for all cancer sites (data not shown). In Figure 2, looking at the row for head & neck cancer as the predicted CSO, there were 887 screens of which 456 were false positives, leaving 887–456 = 431 true positives, and PPV_HN:All= 431/887 = 48.6%. There were 284 screens with the correct CSO prediction (diagonal cell), giving PPV_HN:HN = 284/887 = 32.0%.

Figure 4 shows values of PPV_Site:T for two examples when a given cancer site has its proportion of predicted CSOs for false positives either halved or doubled from the default assumption of proportionality. In Figure 4(a), the predicted CSO proportion for CRC among false positives, originally 21%, is either halved (to 10.5%) or doubled (to 42%). The resulting PPV for CRC (PPV_CRC:T) increases substantially when the proportion is halved and decreases substantially when the proportion is doubled. In contrast, the PPVs for the other cancer sites decrease when the proportion for CRC is halved and increase when it is doubled. The changes for the other cancer sites are smaller than those for CRC, but still greater than minimal. Figure 4(b) shows a similar graph except the cancer site with the changed proportions is a rarer cancer, namely liver. Here, PPV_Liver:T shows similar patterns as PPV_CRC:T, with substantial increases when its predicted CSO proportion for false positives is halved, and substantial decreases when that proportion is doubled. However, the effect on the PPVs of the other cancer sites is much less than with CRC.

Discussion

In this article, we have defined PPVs in a variety of ways in the context of MCDs and have illustrated the basic properties of these metrics. Defining PPVs in these multiple ways underscores the greater level of complexity in interpreting positive MCD screens as compared to positive screens for a single cancer.

We distinguished here between the PPV for targeted cancer sites (PPV_T) and that for any cancer site (PPV_ALL). Depending on the aggregate sensitivity of the MCD for nontargeted sites, as well as the proportion of incident cancers that these sites comprise, the differences in these two PPVs may be minimal or considerable. Importantly, PPV_ALL increases with nontargeted sensitivity, while PPV_TRG decreases. There are limited data in the literature on the sensitivity of MCDs for nontargeted sites, but most MCDs seem to have at least some ability to detect cancer at nontargeted sites.

There is not agreement in the literature as to whether it is desirable or not to detect a nontargeted cancer. A consensus lexicon from an industry group on screening terminology in the MCD context defined a false positive as a positive screen in someone without a “cancer that the test purports to detect” (i.e. without a targeted cancer).¹⁴ This implies that they believe detecting a nontargeted cancer is not a desired outcome. Others have used our PPV_ALL definition to define the PPV for MCDs, implying that it is a desired outcome.^10,11

The magnitude of the differences between PPV_Site:All, PPV_Site:T, and PPV_Site:Site for a given predicted TOO site has implications for the diagnostic work-up. If, say, PPV_Site:All is over twice the level of PPV_Site:T, this indicates that if a cancer is present, then under half the time would it be one that could be specified as a primary or secondary CSO. Similarly, PPV_Site:All being substantially greater than PPV_Site:Site indicates a high likelihood, given that a cancer is present, that it would not be at the primary CSO. Therefore, in these situations, finding the cancer may involve a long and complex diagnostic odyssey. In contrast, if all three PPV metrics are relatively close in value, the likelihood that a cancer, if present, is at the primary CSO is high.

We have shown here, under the default assumption that predicted CSOs among false positives are distributed proportionally to detected cancers, that predicted CSO-specific PPVs are primarily dependent on the accuracy of the primary CSO prediction for the given cancer type, and not on the prevalence of that type in the screened population. This is different from the situation with single screening tests, where, for equal sensitivity and specificity, PPV increases with prevalence. However, we have also shown that changes to the default assumption, where the predicted CSO for a given cancer site is over- or under-represented among false positives, can substantially modify that cancer site's PPV.

Calculating and disseminating estimates of the PPV metrics presented here may be challenging for several reasons. For the standard single-cancer screening tests, estimates of PPV are generally available from the literature. With MCD tests, with several already available for population use, and likely many more available in the next few years, PPV estimates must be specific for the given MCD being used. Starting out, estimates of overall PPV (PPV_ALL or PPV_T) may be available, which could get more precise over time with continued use of the MCD. However, having estimates of the predicted site-specific PPVs available at the start, or even years down the road, may be problematic. First, these require large amounts of data, especially for rarer cancers, since, as mentioned above, they are very sensitive to changes in the distribution of predicted CSOs among false positives. They also depend on the population being screened. Second, some entity must appropriately analyze the data, calculate the PPVs, point estimates along with measures of precision, and make these available to the general clinical community. Whether the MCD companies would have the resources and desire to do this is not clear, although demand by clinical stakeholders could influence how a company chooses to organize test reports. Alternatively, independent researchers could perform the required studies, with or without the cooperation of the companies. However, this may have less impact in clinical contexts, being disassociated from test reports. For the prototypical MCD test assessed here, the range of site-specific PPVs across cancer types under the default proportionality assumption for false positives was relatively modest, about 5%. However, it is likely that this assumption will not always hold in practice, and without it the range of PPVs could be much wider.

With screening using MCD tests still an emerging field, researchers, and practicing clinicians must come to a consensus on what types of PPV metrics are informative, with a clear understanding of the effects of underlying variables such as cancer prevalence and distribution of TOO predictions on these metrics. MCD developers should respond with standardized reporting so that PPV metrics are comparable across assays with diverse numbers and types of targeted cancers. Clinicians should be aware of the meaning, implication and utility of reported PPVs and use these metrics in helping to select an MCD test, and advising on a diagnostic work-up.

Conclusion

In the context of MCDs, PPVs can be defined in various ways, with different use cases. It is important to assess useful PPV metrics and begin appropriate data generation and publication to ensure MCD test performance is understandable and actionable for clinicians and patients.

Supplemental Material

sj-docx-1-msc-10.1177_09691413261421478 - Supplemental material for Positive predictive value metrics for multicancer detection tests

Supplemental material, sj-docx-1-msc-10.1177_09691413261421478 for Positive predictive value metrics for multicancer detection tests by Paul F Pinsky, Elyse M LeeVan, Christos Patriotis and Wendy S Rubinstein in Journal of Medical Screening

Footnotes

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Paul F Pinsky

Supplemental material

Supplemental material for this article is available online.

References

Rubinstein

Patriotis

Dickerherber

, et al. Cancer screening with multicancer detection tests: a translational science review. CA Cancer J Clin 2024; 74: 368–382.

Guerra

Sharma

Castillo

. Multi-cancer early detection: the new frontier in cancer early detection. Annu Rev Med 2024; 75: 67–81.

Post

Braun

Etzioni

, et al. Multicancer early detection tests: an overview of early results from prospective clinical studies and opportunities for oncologists. JCO Oncol Pract 2023; 19: 1111–1115.

Minasian

Pinsky

Katki

, et al. Study design considerations for trials to evaluate multicancer early detection assays for clinical utility. JNCI 2023; 115: 250–257.

Wood

Pinsky

Novotny

, et al. Performance of multiple multi-cancer detection tests using a large independent reference set (Alliance A212102). JNCI 2026. 10.1093/jnci/djag001

Nguyen

LHD

Nguyen

THH

, et al. Prospective validation study: a noninvasive circulating tumor DNA-based assay for simultaneous early detection of multiple cancers in asymptomatic adults. BMC Med 2025; 23: 90.

Schrag

Beer

McDonnell

, et al. Blood-based tests for multicancer early detection (PATHFINDER): a prospective cohort study. Lancet 2023; 402: 1251–1260.

Kerlikowske

Grady

Barclay

, et al. Positive predictive value of screening mammography by age and family history of breast cancer. JAMA 1993; 270: 2444–2450.

The National Lung Screening Trial Research Team. Results of initial low-dose computed tomographic screening for lung cancer. NEJM 2013; 368: 1980–1991.

10.

Wade

Nevitt

Liu

, et al. Multi-cancer early detection tests for general population screening: a systematic literature review. Health Technol Assess 2025; 29. 10.3310/DLMT1294

11.

Hoffman

Wolf

AMD

Raoof

, et al. Multicancer early detection testing: guidance for primary care discussions with patients. Cancer 2025; 131: e35823.

12.

Klein

Richards

Cohn

, et al. Clinical validation of a targeted methylation-based multi-cancer early detection test using an independent validation set. Ann Oncol 2021; 32: 1167–1177.

13.

National Cancer Institute. Surveillance, Epidemiology and End Results Program. www.SEER.CANCER.gov (accessed August 3^rd, 2025).

14.

Clarke

Mitchell

Putcha

, et al. “Lexicon for blood-based early detection and screening” BLOODPAC consensus document. Clin Trans Sci 2024; 17: e70016.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.02 MB