Pathology reports contain useful information such as the main involved organ, diagnosis, etc. These information can be identified from the free text reports and used for large-scale statistical analysis or serve as annotation for other modalities such as pathology slides images. However, manual classification for a huge number of reports on multiple tasks is labor-intensive. In this paper, we have developed an automatic text classifier based on BERT and we propose a human-centric metric to evaluate the model. According to the model confidence, we identify low-confidence cases that require further expert annotation and high-confidence cases that are automatically classified. We report the percentage of low-confidence cases and the performance of automatically classified cases. On the high-confidence cases, the model achieves classification accuracy comparable to pathologists. This leads a potential of reducing 80% to 98% of the manual annotation workload.
@article{arxiv.1911.01226,
title = {Human-centric Metric for Accelerating Pathology Reports Annotation},
author = {Ruibin Ma and Po-Hsuan Cameron Chen and Gang Li and Wei-Hung Weng and Angela Lin and Krishna Gadepalli and Yuannan Cai},
journal= {arXiv preprint arXiv:1911.01226},
year = {2019}
}
Comments
Machine Learning for Health (ML4H) at NeurIPS 2019 - Extended Abstract