TY - JOUR
T1 - Surformer
T2 - An interpretable pattern-perceptive survival transformer for cancer survival prediction from histopathology whole slide images
AU - Wang, Zhikang
AU - Gao, Qian
AU - Yi, Xiaoping
AU - Zhang, Xinyu
AU - Zhang, Yiwen
AU - Zhang, Daokun
AU - Lio`, Pietro
AU - Bain, Christopher A.
AU - Bassed, Richard
AU - Li, Shanshan
AU - Guo, Yuming
AU - Imoto, Seiya
AU - Yao, Jianhua
AU - Daly, Roger J.
AU - Song, Jiangning
N1 - Funding Information:
This work was supported by Major and Seed Inter-Disciplinary Research (IDR) projects awarded by Monash University and a Grant from the International Joint Usage/Research Center, Institute of Medical Science, The University of Tokyo ( K23-2074 ).
Publisher Copyright:
© 2023 The Author(s)
PY - 2023/11
Y1 - 2023/11
N2 - Background and Objective: High-resolution histopathology whole slide images (WSIs) contain abundant valuable information for cancer prognosis. However, most computational pathology methods for survival prediction have weak interpretability and cannot explain the decision-making processes reasonably. To address this issue, we propose a highly interpretable neural network termed pattern-perceptive survival transformer (Surformer) for cancer survival prediction from WSIs. Methods: Notably, Surformer can quantify specific histological patterns through bag-level labels without any patch/cell-level auxiliary information. Specifically, the proposed ratio-reserved cross-attention module (RRCA) generates global and local features with the learnable prototypes (pglobal, plocals) as detectors and quantifies the patches correlative to each plocal in the form of ratio factors (rfs). Afterward, multi-head self&cross-attention modules proceed with the computation for feature enhancement against noise. Eventually, the designed disentangling loss function guides multiple local features to focus on distinct patterns, thereby assisting rfs from RRCA in achieving more explicit histological feature quantification. Results: Extensive experiments on five TCGA datasets illustrate that Surformer outperforms existing state-of-the-art methods. In addition, we highlight its interpretation by visualizing rfs distribution across high-risk and low-risk cohorts and retrieving and analyzing critical histological patterns contributing to the survival prediction. Conclusions: Surformer is expected to be exploited as a useful tool for performing histopathology image data-driven analysis and gaining new insights for interpreting the associations between such images and patient survival states.
AB - Background and Objective: High-resolution histopathology whole slide images (WSIs) contain abundant valuable information for cancer prognosis. However, most computational pathology methods for survival prediction have weak interpretability and cannot explain the decision-making processes reasonably. To address this issue, we propose a highly interpretable neural network termed pattern-perceptive survival transformer (Surformer) for cancer survival prediction from WSIs. Methods: Notably, Surformer can quantify specific histological patterns through bag-level labels without any patch/cell-level auxiliary information. Specifically, the proposed ratio-reserved cross-attention module (RRCA) generates global and local features with the learnable prototypes (pglobal, plocals) as detectors and quantifies the patches correlative to each plocal in the form of ratio factors (rfs). Afterward, multi-head self&cross-attention modules proceed with the computation for feature enhancement against noise. Eventually, the designed disentangling loss function guides multiple local features to focus on distinct patterns, thereby assisting rfs from RRCA in achieving more explicit histological feature quantification. Results: Extensive experiments on five TCGA datasets illustrate that Surformer outperforms existing state-of-the-art methods. In addition, we highlight its interpretation by visualizing rfs distribution across high-risk and low-risk cohorts and retrieving and analyzing critical histological patterns contributing to the survival prediction. Conclusions: Surformer is expected to be exploited as a useful tool for performing histopathology image data-driven analysis and gaining new insights for interpreting the associations between such images and patient survival states.
KW - Survival analysis
KW - multiple in- stance learning
KW - whole slide image
KW - deep learning interpretation
UR - http://www.scopus.com/inward/record.url?scp=85167570944&partnerID=8YFLogxK
U2 - 10.1016/j.cmpb.2023.107733
DO - 10.1016/j.cmpb.2023.107733
M3 - Article
AN - SCOPUS:85167570944
SN - 0169-2607
VL - 241
JO - Computer Methods and Programs in Biomedicine
JF - Computer Methods and Programs in Biomedicine
M1 - 107733
ER -