Early identification of high-risk ICU patients is crucial for directing limited medical resources. We introduce ALFIA (Adaptive Layer Fusion with Intelligent Attention), a modular, attention-based architecture that jointly trains LoRA (Low-Rank Adaptation) adapters and an adaptive layer-weighting mechanism to fuse multi-layer semantic features from a BERT backbone. Trained on our rigorous cw-24 (CriticalWindow-24) benchmark, ALFIA surpasses state-of-the-art tabular classifiers in AUPRC while preserving a balanced precision-recall profile. The embeddings produced by ALFIA's fusion module, capturing both fine-grained clinical cues and high-level concepts, enable seamless pairing with GBDTs (CatBoost/LightGBM) as ALFIA-boost, and deep neuro networks as ALFIA-nn, yielding additional performance gains. Our experiments confirm ALFIA's superior early-warning performance, by operating directly on routine clinical text, it furnishes clinicians with a convenient yet robust tool for risk stratification and timely intervention in critical-care settings.
@article{arxiv.2506.04924,
title = {Predicting ICU In-Hospital Mortality Using Adaptive Transformer Layer Fusion},
author = {Han Wang and Ruoyun He and Guoguang Lao and Ting Liu and Hejiao Luo and Changqi Qin and Hongying Luo and Junmin Huang and Zihan Wei and Lu Chen and Yongzhi Xu and Ziqian Bi and Junhao Song and Tianyang Wang and Chia Xin Liang and Xinyuan Song and Huafeng Liu and Junfeng Hao and Chunjie Tian},
journal= {arXiv preprint arXiv:2506.04924},
year = {2025}
}