Zero-Shot Whole Slide Image Retrieval in Histopathology Using Embeddings of Foundation Models

Saghir Alfasly; Ghazal Alabtah; Sobhan Hemati; Krishna Rani Kalari; H. R. Tizhoosh

Zero-Shot Whole Slide Image Retrieval in Histopathology Using Embeddings of Foundation Models

Image and Video Processing 2024-09-13 v2 Artificial Intelligence Computer Vision and Pattern Recognition

Authors: Saghir Alfasly , Ghazal Alabtah , Sobhan Hemati , Krishna Rani Kalari , H. R. Tizhoosh

Abstract

We have tested recently published foundation models for histopathology for image retrieval. We report macro average of F1 score for top-1 retrieval, majority of top-3 retrievals, and majority of top-5 retrievals. We perform zero-shot retrievals, i.e., we do not alter embeddings and we do not train any classifier. As test data, we used diagnostic slides of TCGA, The Cancer Genome Atlas, consisting of 23 organs and 117 cancer subtypes. As a search platform we used Yottixel that enabled us to perform WSI search using patches. Achieved F1 scores show low performance, e.g., for top-5 retrievals, 27% +/- 13% (Yottixel-DenseNet), 42% +/- 14% (Yottixel-UNI), 40%+/-13% (Yottixel-Virchow), 41%+/-13% (Yottixel-GigaPath), and 41%+/-14% (GigaPath WSI).

Keywords

whole slide image retrieval foundation model histopathology zero-shot learning Yottixel TCGA

Cite

@article{arxiv.2409.04631,
  title  = {Zero-Shot Whole Slide Image Retrieval in Histopathology Using Embeddings of Foundation Models},
  author = {Saghir Alfasly and Ghazal Alabtah and Sobhan Hemati and Krishna Rani Kalari and H. R. Tizhoosh},
  journal= {arXiv preprint arXiv:2409.04631},
  year   = {2024}
}

Comments

This paper will be updated with more results

Zero-Shot Whole Slide Image Retrieval in Histopathology Using Embeddings of Foundation Models

Abstract

Keywords

Cite

Comments

Related papers