10-year performance of four models of breast cancer risk: a validation study

Mary Beth Terry, Yuyan Liao, Alice S. Whittemore, Nicole Leoce, Richard Buchsbaum, Nur Zeinomar, Gillian S. Dite, Wendy K. Chung, Julia A. Knight, Melissa C. Southey, Roger L. Milne, David Goldgar, Graham G. Giles, Sue Anne McLachlan, Michael L. Friedlander, Prue C. Weideman, Gord Glendon, Stephanie Nesci, Irene L. Andrulis, Esther M. John & 5 others Kelly Anne Phillips, Mary B. Daly, Saundra S. Buys, John L. Hopper, Robert J. MacInnis

Research output: Contribution to journalArticleResearchpeer-review

Abstract

Background: Independent validation is essential to justify use of models of breast cancer risk prediction and inform decisions about prevention options and screening. Few independent validations had been done using cohorts for common breast cancer risk prediction models, and those that have been done had small sample sizes and short follow-up periods, and used earlier versions of the prediction tools. We aimed to validate the relative performance of four commonly used models of breast cancer risk and assess the effect of limited data input on each one's performance. Methods: In this validation study, we used the Breast Cancer Prospective Family Study Cohort (ProF-SC), which includes 18 856 women from Australia, Canada, and the USA who did not have breast cancer at recruitment, between March 17, 1992, and June 29, 2011. We selected women from the cohort who were 20–70 years old and had no previous history of bilateral prophylactic mastectomy or ovarian cancer, at least 2 months of follow-up data, and information available about family history of breast cancer. We used this selected cohort to calculate 10-year risk scores and compare four models of breast cancer risk prediction: the Breast and Ovarian Analysis of Disease Incidence and Carrier Estimation Algorithm model (BOADICEA), BRCAPRO, the Breast Cancer Risk Assessment Tool (BCRAT), and the International Breast Cancer Intervention Study model (IBIS). We compared model calibration based on the ratio of the expected number of breast cancer cases to the observed number of breast cancer cases in the cohort, and on the basis of their discriminatory ability to separate those who will and will not have breast cancer diagnosed within 10 years as measured with the concordance statistic (C-statistic). We did subgroup analyses to compare the performance of the models at 10 years in BRCA1 or BRCA2 mutation carriers (ie, BRCA-positive women), tested non-carriers and untested participants (ie, BRCA-negative women), and participants younger than 50 years at recruitment. We also assessed the effect that limited data input (eg, restriction of the amount of family history and non-genetic information included) had on the models' performance. Findings: After median follow-up of 11·1 years (IQR 6·0–14·4), 619 (4%) of 15 732 women selected from the ProF-SC cohort study were prospectively diagnosed with breast cancer after recruitment, of whom 519 (84%) had histologically confirmed disease. BOADICEA and IBIS were well calibrated in the overall validation cohort, whereas BRCAPRO and BCRAT underpredicted risk (ratio of expected cases to observed cases 1·05 [95% CI 0·97–1·14] for BOADICEA, 1·03 [0·96–1·12] for IBIS, 0·59 [0·55–0·64] for BRCAPRO, and 0·79 [0·73–0·85] for BRCAT). The estimated C-statistics for the complete validation cohort were 0·70 (95% CI 0·68–0·72) for BOADICEA, 0·71 (0·69–0·73) for IBIS, 0·68 (0·65–0·70) for BRCAPRO, and 0·60 (0·58–0·62) for BCRAT. In subgroup analyses by BRCA mutation status, the ratio of expected to observed cases for BRCA-negative women was 1·02 (95% CI 0·93–1·12) for BOADICEA, 1·00 (0·92–1·10) for IBIS, 0·53 (0·49–0·58) for BRCAPRO, and 0·97 (0·89–1·06) for BCRAT. For BRCA-positive participants, BOADICEA and IBIS were well calibrated, but BRCAPRO underpredicted risk (ratio of expected to observed cases 1·17 [95% CI 0·99–1·38] for BOADICEA, 1·14 [0·96–1·35] for IBIS, and 0·80 [0·68–0·95] for BRCAPRO). We noted similar patterns of calibration for women younger than 50 years at recruitment. Finally, BOADICEA and IBIS predictive scores were not appreciably affected by limiting input data to family history for first-degree and second-degree relatives. Interpretation: Our results suggest that models that include multigenerational family history, such as BOADICEA and IBIS, have better ability to predict breast cancer risk, even for women at average or below-average risk of breast cancer. Although BOADICEA and IBIS performed similarly, further improvements in the accuracy of predictions could be possible with hybrid models that incorporate the polygenic risk component of BOADICEA and the non-family-history risk factors included in IBIS. Funding: US National Institutes of Health, National Cancer Institute, Breast Cancer Research Foundation, Australian National Health and Medical Research Council, Victorian Health Promotion Foundation, Victorian Breast Cancer Research Consortium, Cancer Australia, National Breast Cancer Foundation, Queensland Cancer Fund, Cancer Councils of New South Wales, Victoria, Tasmania, and South Australia, and Cancer Foundation of Western Australia.

Original languageEnglish
Pages (from-to)504-517
Number of pages14
JournalThe Lancet Oncology
Volume20
Issue number4
DOIs
Publication statusPublished - 1 Apr 2019

Cite this

Terry, M. B., Liao, Y., Whittemore, A. S., Leoce, N., Buchsbaum, R., Zeinomar, N., ... MacInnis, R. J. (2019). 10-year performance of four models of breast cancer risk: a validation study. The Lancet Oncology, 20(4), 504-517. https://doi.org/10.1016/S1470-2045(18)30902-1
Terry, Mary Beth ; Liao, Yuyan ; Whittemore, Alice S. ; Leoce, Nicole ; Buchsbaum, Richard ; Zeinomar, Nur ; Dite, Gillian S. ; Chung, Wendy K. ; Knight, Julia A. ; Southey, Melissa C. ; Milne, Roger L. ; Goldgar, David ; Giles, Graham G. ; McLachlan, Sue Anne ; Friedlander, Michael L. ; Weideman, Prue C. ; Glendon, Gord ; Nesci, Stephanie ; Andrulis, Irene L. ; John, Esther M. ; Phillips, Kelly Anne ; Daly, Mary B. ; Buys, Saundra S. ; Hopper, John L. ; MacInnis, Robert J. / 10-year performance of four models of breast cancer risk : a validation study. In: The Lancet Oncology. 2019 ; Vol. 20, No. 4. pp. 504-517.
@article{9678d5495ab94159b441a7675b958fcb,
title = "10-year performance of four models of breast cancer risk: a validation study",
abstract = "Background: Independent validation is essential to justify use of models of breast cancer risk prediction and inform decisions about prevention options and screening. Few independent validations had been done using cohorts for common breast cancer risk prediction models, and those that have been done had small sample sizes and short follow-up periods, and used earlier versions of the prediction tools. We aimed to validate the relative performance of four commonly used models of breast cancer risk and assess the effect of limited data input on each one's performance. Methods: In this validation study, we used the Breast Cancer Prospective Family Study Cohort (ProF-SC), which includes 18 856 women from Australia, Canada, and the USA who did not have breast cancer at recruitment, between March 17, 1992, and June 29, 2011. We selected women from the cohort who were 20–70 years old and had no previous history of bilateral prophylactic mastectomy or ovarian cancer, at least 2 months of follow-up data, and information available about family history of breast cancer. We used this selected cohort to calculate 10-year risk scores and compare four models of breast cancer risk prediction: the Breast and Ovarian Analysis of Disease Incidence and Carrier Estimation Algorithm model (BOADICEA), BRCAPRO, the Breast Cancer Risk Assessment Tool (BCRAT), and the International Breast Cancer Intervention Study model (IBIS). We compared model calibration based on the ratio of the expected number of breast cancer cases to the observed number of breast cancer cases in the cohort, and on the basis of their discriminatory ability to separate those who will and will not have breast cancer diagnosed within 10 years as measured with the concordance statistic (C-statistic). We did subgroup analyses to compare the performance of the models at 10 years in BRCA1 or BRCA2 mutation carriers (ie, BRCA-positive women), tested non-carriers and untested participants (ie, BRCA-negative women), and participants younger than 50 years at recruitment. We also assessed the effect that limited data input (eg, restriction of the amount of family history and non-genetic information included) had on the models' performance. Findings: After median follow-up of 11·1 years (IQR 6·0–14·4), 619 (4{\%}) of 15 732 women selected from the ProF-SC cohort study were prospectively diagnosed with breast cancer after recruitment, of whom 519 (84{\%}) had histologically confirmed disease. BOADICEA and IBIS were well calibrated in the overall validation cohort, whereas BRCAPRO and BCRAT underpredicted risk (ratio of expected cases to observed cases 1·05 [95{\%} CI 0·97–1·14] for BOADICEA, 1·03 [0·96–1·12] for IBIS, 0·59 [0·55–0·64] for BRCAPRO, and 0·79 [0·73–0·85] for BRCAT). The estimated C-statistics for the complete validation cohort were 0·70 (95{\%} CI 0·68–0·72) for BOADICEA, 0·71 (0·69–0·73) for IBIS, 0·68 (0·65–0·70) for BRCAPRO, and 0·60 (0·58–0·62) for BCRAT. In subgroup analyses by BRCA mutation status, the ratio of expected to observed cases for BRCA-negative women was 1·02 (95{\%} CI 0·93–1·12) for BOADICEA, 1·00 (0·92–1·10) for IBIS, 0·53 (0·49–0·58) for BRCAPRO, and 0·97 (0·89–1·06) for BCRAT. For BRCA-positive participants, BOADICEA and IBIS were well calibrated, but BRCAPRO underpredicted risk (ratio of expected to observed cases 1·17 [95{\%} CI 0·99–1·38] for BOADICEA, 1·14 [0·96–1·35] for IBIS, and 0·80 [0·68–0·95] for BRCAPRO). We noted similar patterns of calibration for women younger than 50 years at recruitment. Finally, BOADICEA and IBIS predictive scores were not appreciably affected by limiting input data to family history for first-degree and second-degree relatives. Interpretation: Our results suggest that models that include multigenerational family history, such as BOADICEA and IBIS, have better ability to predict breast cancer risk, even for women at average or below-average risk of breast cancer. Although BOADICEA and IBIS performed similarly, further improvements in the accuracy of predictions could be possible with hybrid models that incorporate the polygenic risk component of BOADICEA and the non-family-history risk factors included in IBIS. Funding: US National Institutes of Health, National Cancer Institute, Breast Cancer Research Foundation, Australian National Health and Medical Research Council, Victorian Health Promotion Foundation, Victorian Breast Cancer Research Consortium, Cancer Australia, National Breast Cancer Foundation, Queensland Cancer Fund, Cancer Councils of New South Wales, Victoria, Tasmania, and South Australia, and Cancer Foundation of Western Australia.",
author = "Terry, {Mary Beth} and Yuyan Liao and Whittemore, {Alice S.} and Nicole Leoce and Richard Buchsbaum and Nur Zeinomar and Dite, {Gillian S.} and Chung, {Wendy K.} and Knight, {Julia A.} and Southey, {Melissa C.} and Milne, {Roger L.} and David Goldgar and Giles, {Graham G.} and McLachlan, {Sue Anne} and Friedlander, {Michael L.} and Weideman, {Prue C.} and Gord Glendon and Stephanie Nesci and Andrulis, {Irene L.} and John, {Esther M.} and Phillips, {Kelly Anne} and Daly, {Mary B.} and Buys, {Saundra S.} and Hopper, {John L.} and MacInnis, {Robert J.}",
year = "2019",
month = "4",
day = "1",
doi = "10.1016/S1470-2045(18)30902-1",
language = "English",
volume = "20",
pages = "504--517",
journal = "Lancet Oncology",
issn = "1470-2045",
publisher = "Elsevier",
number = "4",

}

Terry, MB, Liao, Y, Whittemore, AS, Leoce, N, Buchsbaum, R, Zeinomar, N, Dite, GS, Chung, WK, Knight, JA, Southey, MC, Milne, RL, Goldgar, D, Giles, GG, McLachlan, SA, Friedlander, ML, Weideman, PC, Glendon, G, Nesci, S, Andrulis, IL, John, EM, Phillips, KA, Daly, MB, Buys, SS, Hopper, JL & MacInnis, RJ 2019, '10-year performance of four models of breast cancer risk: a validation study' The Lancet Oncology, vol. 20, no. 4, pp. 504-517. https://doi.org/10.1016/S1470-2045(18)30902-1

10-year performance of four models of breast cancer risk : a validation study. / Terry, Mary Beth; Liao, Yuyan; Whittemore, Alice S.; Leoce, Nicole; Buchsbaum, Richard; Zeinomar, Nur; Dite, Gillian S.; Chung, Wendy K.; Knight, Julia A.; Southey, Melissa C.; Milne, Roger L.; Goldgar, David; Giles, Graham G.; McLachlan, Sue Anne; Friedlander, Michael L.; Weideman, Prue C.; Glendon, Gord; Nesci, Stephanie; Andrulis, Irene L.; John, Esther M.; Phillips, Kelly Anne; Daly, Mary B.; Buys, Saundra S.; Hopper, John L.; MacInnis, Robert J.

In: The Lancet Oncology, Vol. 20, No. 4, 01.04.2019, p. 504-517.

Research output: Contribution to journalArticleResearchpeer-review

TY - JOUR

T1 - 10-year performance of four models of breast cancer risk

T2 - a validation study

AU - Terry, Mary Beth

AU - Liao, Yuyan

AU - Whittemore, Alice S.

AU - Leoce, Nicole

AU - Buchsbaum, Richard

AU - Zeinomar, Nur

AU - Dite, Gillian S.

AU - Chung, Wendy K.

AU - Knight, Julia A.

AU - Southey, Melissa C.

AU - Milne, Roger L.

AU - Goldgar, David

AU - Giles, Graham G.

AU - McLachlan, Sue Anne

AU - Friedlander, Michael L.

AU - Weideman, Prue C.

AU - Glendon, Gord

AU - Nesci, Stephanie

AU - Andrulis, Irene L.

AU - John, Esther M.

AU - Phillips, Kelly Anne

AU - Daly, Mary B.

AU - Buys, Saundra S.

AU - Hopper, John L.

AU - MacInnis, Robert J.

PY - 2019/4/1

Y1 - 2019/4/1

N2 - Background: Independent validation is essential to justify use of models of breast cancer risk prediction and inform decisions about prevention options and screening. Few independent validations had been done using cohorts for common breast cancer risk prediction models, and those that have been done had small sample sizes and short follow-up periods, and used earlier versions of the prediction tools. We aimed to validate the relative performance of four commonly used models of breast cancer risk and assess the effect of limited data input on each one's performance. Methods: In this validation study, we used the Breast Cancer Prospective Family Study Cohort (ProF-SC), which includes 18 856 women from Australia, Canada, and the USA who did not have breast cancer at recruitment, between March 17, 1992, and June 29, 2011. We selected women from the cohort who were 20–70 years old and had no previous history of bilateral prophylactic mastectomy or ovarian cancer, at least 2 months of follow-up data, and information available about family history of breast cancer. We used this selected cohort to calculate 10-year risk scores and compare four models of breast cancer risk prediction: the Breast and Ovarian Analysis of Disease Incidence and Carrier Estimation Algorithm model (BOADICEA), BRCAPRO, the Breast Cancer Risk Assessment Tool (BCRAT), and the International Breast Cancer Intervention Study model (IBIS). We compared model calibration based on the ratio of the expected number of breast cancer cases to the observed number of breast cancer cases in the cohort, and on the basis of their discriminatory ability to separate those who will and will not have breast cancer diagnosed within 10 years as measured with the concordance statistic (C-statistic). We did subgroup analyses to compare the performance of the models at 10 years in BRCA1 or BRCA2 mutation carriers (ie, BRCA-positive women), tested non-carriers and untested participants (ie, BRCA-negative women), and participants younger than 50 years at recruitment. We also assessed the effect that limited data input (eg, restriction of the amount of family history and non-genetic information included) had on the models' performance. Findings: After median follow-up of 11·1 years (IQR 6·0–14·4), 619 (4%) of 15 732 women selected from the ProF-SC cohort study were prospectively diagnosed with breast cancer after recruitment, of whom 519 (84%) had histologically confirmed disease. BOADICEA and IBIS were well calibrated in the overall validation cohort, whereas BRCAPRO and BCRAT underpredicted risk (ratio of expected cases to observed cases 1·05 [95% CI 0·97–1·14] for BOADICEA, 1·03 [0·96–1·12] for IBIS, 0·59 [0·55–0·64] for BRCAPRO, and 0·79 [0·73–0·85] for BRCAT). The estimated C-statistics for the complete validation cohort were 0·70 (95% CI 0·68–0·72) for BOADICEA, 0·71 (0·69–0·73) for IBIS, 0·68 (0·65–0·70) for BRCAPRO, and 0·60 (0·58–0·62) for BCRAT. In subgroup analyses by BRCA mutation status, the ratio of expected to observed cases for BRCA-negative women was 1·02 (95% CI 0·93–1·12) for BOADICEA, 1·00 (0·92–1·10) for IBIS, 0·53 (0·49–0·58) for BRCAPRO, and 0·97 (0·89–1·06) for BCRAT. For BRCA-positive participants, BOADICEA and IBIS were well calibrated, but BRCAPRO underpredicted risk (ratio of expected to observed cases 1·17 [95% CI 0·99–1·38] for BOADICEA, 1·14 [0·96–1·35] for IBIS, and 0·80 [0·68–0·95] for BRCAPRO). We noted similar patterns of calibration for women younger than 50 years at recruitment. Finally, BOADICEA and IBIS predictive scores were not appreciably affected by limiting input data to family history for first-degree and second-degree relatives. Interpretation: Our results suggest that models that include multigenerational family history, such as BOADICEA and IBIS, have better ability to predict breast cancer risk, even for women at average or below-average risk of breast cancer. Although BOADICEA and IBIS performed similarly, further improvements in the accuracy of predictions could be possible with hybrid models that incorporate the polygenic risk component of BOADICEA and the non-family-history risk factors included in IBIS. Funding: US National Institutes of Health, National Cancer Institute, Breast Cancer Research Foundation, Australian National Health and Medical Research Council, Victorian Health Promotion Foundation, Victorian Breast Cancer Research Consortium, Cancer Australia, National Breast Cancer Foundation, Queensland Cancer Fund, Cancer Councils of New South Wales, Victoria, Tasmania, and South Australia, and Cancer Foundation of Western Australia.

AB - Background: Independent validation is essential to justify use of models of breast cancer risk prediction and inform decisions about prevention options and screening. Few independent validations had been done using cohorts for common breast cancer risk prediction models, and those that have been done had small sample sizes and short follow-up periods, and used earlier versions of the prediction tools. We aimed to validate the relative performance of four commonly used models of breast cancer risk and assess the effect of limited data input on each one's performance. Methods: In this validation study, we used the Breast Cancer Prospective Family Study Cohort (ProF-SC), which includes 18 856 women from Australia, Canada, and the USA who did not have breast cancer at recruitment, between March 17, 1992, and June 29, 2011. We selected women from the cohort who were 20–70 years old and had no previous history of bilateral prophylactic mastectomy or ovarian cancer, at least 2 months of follow-up data, and information available about family history of breast cancer. We used this selected cohort to calculate 10-year risk scores and compare four models of breast cancer risk prediction: the Breast and Ovarian Analysis of Disease Incidence and Carrier Estimation Algorithm model (BOADICEA), BRCAPRO, the Breast Cancer Risk Assessment Tool (BCRAT), and the International Breast Cancer Intervention Study model (IBIS). We compared model calibration based on the ratio of the expected number of breast cancer cases to the observed number of breast cancer cases in the cohort, and on the basis of their discriminatory ability to separate those who will and will not have breast cancer diagnosed within 10 years as measured with the concordance statistic (C-statistic). We did subgroup analyses to compare the performance of the models at 10 years in BRCA1 or BRCA2 mutation carriers (ie, BRCA-positive women), tested non-carriers and untested participants (ie, BRCA-negative women), and participants younger than 50 years at recruitment. We also assessed the effect that limited data input (eg, restriction of the amount of family history and non-genetic information included) had on the models' performance. Findings: After median follow-up of 11·1 years (IQR 6·0–14·4), 619 (4%) of 15 732 women selected from the ProF-SC cohort study were prospectively diagnosed with breast cancer after recruitment, of whom 519 (84%) had histologically confirmed disease. BOADICEA and IBIS were well calibrated in the overall validation cohort, whereas BRCAPRO and BCRAT underpredicted risk (ratio of expected cases to observed cases 1·05 [95% CI 0·97–1·14] for BOADICEA, 1·03 [0·96–1·12] for IBIS, 0·59 [0·55–0·64] for BRCAPRO, and 0·79 [0·73–0·85] for BRCAT). The estimated C-statistics for the complete validation cohort were 0·70 (95% CI 0·68–0·72) for BOADICEA, 0·71 (0·69–0·73) for IBIS, 0·68 (0·65–0·70) for BRCAPRO, and 0·60 (0·58–0·62) for BCRAT. In subgroup analyses by BRCA mutation status, the ratio of expected to observed cases for BRCA-negative women was 1·02 (95% CI 0·93–1·12) for BOADICEA, 1·00 (0·92–1·10) for IBIS, 0·53 (0·49–0·58) for BRCAPRO, and 0·97 (0·89–1·06) for BCRAT. For BRCA-positive participants, BOADICEA and IBIS were well calibrated, but BRCAPRO underpredicted risk (ratio of expected to observed cases 1·17 [95% CI 0·99–1·38] for BOADICEA, 1·14 [0·96–1·35] for IBIS, and 0·80 [0·68–0·95] for BRCAPRO). We noted similar patterns of calibration for women younger than 50 years at recruitment. Finally, BOADICEA and IBIS predictive scores were not appreciably affected by limiting input data to family history for first-degree and second-degree relatives. Interpretation: Our results suggest that models that include multigenerational family history, such as BOADICEA and IBIS, have better ability to predict breast cancer risk, even for women at average or below-average risk of breast cancer. Although BOADICEA and IBIS performed similarly, further improvements in the accuracy of predictions could be possible with hybrid models that incorporate the polygenic risk component of BOADICEA and the non-family-history risk factors included in IBIS. Funding: US National Institutes of Health, National Cancer Institute, Breast Cancer Research Foundation, Australian National Health and Medical Research Council, Victorian Health Promotion Foundation, Victorian Breast Cancer Research Consortium, Cancer Australia, National Breast Cancer Foundation, Queensland Cancer Fund, Cancer Councils of New South Wales, Victoria, Tasmania, and South Australia, and Cancer Foundation of Western Australia.

UR - http://www.scopus.com/inward/record.url?scp=85063574910&partnerID=8YFLogxK

U2 - 10.1016/S1470-2045(18)30902-1

DO - 10.1016/S1470-2045(18)30902-1

M3 - Article

VL - 20

SP - 504

EP - 517

JO - Lancet Oncology

JF - Lancet Oncology

SN - 1470-2045

IS - 4

ER -

Terry MB, Liao Y, Whittemore AS, Leoce N, Buchsbaum R, Zeinomar N et al. 10-year performance of four models of breast cancer risk: a validation study. The Lancet Oncology. 2019 Apr 1;20(4):504-517. https://doi.org/10.1016/S1470-2045(18)30902-1