RADIFUSION: A multi-radiomics deep learning based breast cancer risk prediction model using sequential mammographic images with image attention and bilateral asymmetry refinement

Objective.Breast cancer is a significant public health concern, and early detection is critical for triaging high-risk patients. Sequential screening mammograms can provide important spatiotemporal information about changes in breast tissue over time, which may be useful for breast cancer risk prediction models.Approach.In this study, we propose a deep learning architecture called radiomics fused gated attention (RADIFUSION) that utilizes sequential mammograms and incorporates a linear image attention mechanism, radiomics features, a new gating mechanism to combine different mammographic views, and bilateral asymmetry-based finetuning for breast cancer risk assessment. We evaluate our model on a screening dataset called the Cohort of Screen-Aged Women, consisting of 8723 patients altogether.Main results.Based on results obtained on the independent testing set consisting of 1749 women, our approach achieved slightly better performance compared to other state-of-the-art models with area under the receiver operating characteristic curves (AUCs) of 0.905, 0.872 and 0.866 in the three respective metrics of 1 year AUC, 2 year AUC and 3 year AUC. Our study highlights the importance of incorporating various deep learning mechanisms, such as image attention, radiomics features, a gating mechanism, and bilateral asymmetry-based fine-tuning, to improve the accuracy of breast cancer risk assessment. We also demonstrate that our model's performance was enhanced by leveraging spatiotemporal information from sequential mammograms.Significance.Our findings suggest that RADIFUSION can provide clinicians with a useful tool for breast cancer risk assessment.