Analyzing high resolution whole slide images (WSIs) with regard to information across multiple scales poses a significant challenge in digital pathology. Multi-instance learning (MIL) is a common solution for working with high resolution images by classifying bags of objects (i.e. sets of smaller image patches). However, such processing is typically performed at a single scale (e.g., 20x magnification) of WSIs, disregarding the vital inter-scale information that is key to diagnoses by human pathologists. In this study, we propose a novel cross-scale MIL algorithm to explicitly aggregate inter-scale relationships into a single MIL network for pathological image diagnosis. The contribution of this paper is three-fold: (1) A novel cross-scale MIL (CS-MIL) algorithm that integrates the multi-scale information and the inter-scale relationships is proposed; (2) A toy dataset with scale-specific morphological features is created and released to examine and visualize differential cross-scale attention; (3) Superior performance on both in-house and public datasets is demonstrated by our simple cross-scale MIL strategy. The official implementation is publicly available at https://github.com/hrlblab/CS-MIL.
@article{arxiv.2304.00216,
title = {Cross-scale Multi-instance Learning for Pathological Image Diagnosis},
author = {Ruining Deng and Can Cui and Lucas W. Remedios and Shunxing Bao and R. Michael Womick and Sophie Chiron and Jia Li and Joseph T. Roland and Ken S. Lau and Qi Liu and Keith T. Wilson and Yaohong Wang and Lori A. Coburn and Bennett A. Landman and Yuankai Huo},
journal= {arXiv preprint arXiv:2304.00216},
year = {2024}
}