Evaluation of models to predict BRCA germline mutations

The selection of candidates for BRCA germline mutation testing is an important clinical issue yet it remains a significant challenge. A number of risk prediction models have been developed to assist in pretest counselling. We have evaluated the performance and the inter-rater reliability of four of these models (BRCAPRO, Manchester, Penn and the Myriad-Frank). The four risk assessment models were applied to 380 pedigrees of families who had undergone BRCA1/2 mutation analysis. Sensitivity, specificity, positive and negative predictive values, likelihood ratios and area under the receiver operator characteristic (ROC) curve were calculated for each model. Using a greater than 10% probability threshold, the likelihood that a BRCA test result was positive in a mutation carrier compared to the likelihood that the same result would be expected in an individual without a BRCA mutation was 2.10 (95% confidence interval (CI) 1.66-2.67) for Penn, 1.74 (95% CI 1.48-2.04) for Myriad, 1.35 (95% CI 1.19-1.53) for Manchester and 1.68 (95% CI 1.39-2.03) for BRCAPRO. Application of these models, therefore, did not rule in BRCA mutation carrier status. Similar trends were observed for separate BRCA1/2 performance measures except BRCA2 assessment in the Penn model where the positive likelihood ratio was 5.93. The area under the ROC curve for each model was close to 0.75. In conclusion, the four models had very little impact on the pre-test probability of disease; there were significant clinical barriers to using some models and risk estimates varied between experts. Use of models for predicting BRCA mutation status is not currently justified for populations such as that evaluated in the current study.