Pathological diagnosis is vital for determining disease characteristics, guiding treatment, and assessing prognosis, relying heavily on detailed, multi-scale analysis of high-resolution whole slide images (WSI). However, existing large vision-language models (LVLMs) are limited by input resolution constraints, hindering their efficiency and accuracy in pathology image analysis. To overcome these issues, we propose two innovative strategies: the mixed task-guided feature enhancement, which directs feature extraction toward lesion-related details across scales, and the prompt-guided detail feature completion, which integrates coarse- and fine-grained features from WSI based on specific prompts without compromising inference speed. Leveraging a comprehensive dataset of 490K samples from diverse pathology tasks, we trained the pathology-specialized LVLM, OmniPath. Extensive experiments demonstrate that this model significantly outperforms existing methods in diagnostic accuracy and efficiency, providing an interactive, clinically aligned approach for auxiliary diagnosis in a wide range of pathology applications.
@article{arxiv.2412.09521,
title = {Efficient and Comprehensive Feature Extraction in Large Vision-Language Model for Pathology Analysis},
author = {Shengxuming Zhang and Weihan Li and Tianhong Gao and Jiacong Hu and Haoming Luo and Xiuming Zhang and Jing Zhang and Mingli Song and Zunlei Feng},
journal= {arXiv preprint arXiv:2412.09521},
year = {2025}
}