Surformer: An interpretable pattern-perceptive survival transformer for cancer survival prediction from histopathology whole slide images

Zhikang Wang, Qian Gao, Xiaoping Yi, Xinyu Zhang, Yiwen Zhang, Daokun Zhang, Pietro Lio`, Christopher A. Bain, Richard Bassed, Shanshan Li, Yuming Guo, Seiya Imoto, Jianhua Yao, Roger J. Daly, Jiangning Song

Research output: Contribution to journalArticleResearchpeer-review

3 Citations (Scopus)

Abstract

Background and Objective: High-resolution histopathology whole slide images (WSIs) contain abundant valuable information for cancer prognosis. However, most computational pathology methods for survival prediction have weak interpretability and cannot explain the decision-making processes reasonably. To address this issue, we propose a highly interpretable neural network termed pattern-perceptive survival transformer (Surformer) for cancer survival prediction from WSIs. Methods: Notably, Surformer can quantify specific histological patterns through bag-level labels without any patch/cell-level auxiliary information. Specifically, the proposed ratio-reserved cross-attention module (RRCA) generates global and local features with the learnable prototypes (pglobal, plocals) as detectors and quantifies the patches correlative to each plocal in the form of ratio factors (rfs). Afterward, multi-head self&cross-attention modules proceed with the computation for feature enhancement against noise. Eventually, the designed disentangling loss function guides multiple local features to focus on distinct patterns, thereby assisting rfs from RRCA in achieving more explicit histological feature quantification. Results: Extensive experiments on five TCGA datasets illustrate that Surformer outperforms existing state-of-the-art methods. In addition, we highlight its interpretation by visualizing rfs distribution across high-risk and low-risk cohorts and retrieving and analyzing critical histological patterns contributing to the survival prediction. Conclusions: Surformer is expected to be exploited as a useful tool for performing histopathology image data-driven analysis and gaining new insights for interpreting the associations between such images and patient survival states.

Original languageEnglish
Article number107733
Number of pages12
JournalComputer Methods and Programs in Biomedicine
Volume241
DOIs
Publication statusPublished - Nov 2023

Keywords

  • Survival analysis
  • multiple in- stance learning
  • whole slide image
  • deep learning interpretation

Cite this