Computation and Language · Computer Science
Never Lost in the Middle: Mastering Long-Context Question Answering with Position-Agnostic Decompositional Training
Junqing He, Kunhao Pan, Xiaoqun Dong, Zhuoyang Song +6
2024-08-15
Computation and Language · Computer Science
Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization
Cheng-Yu Hsieh, Yung-Sung Chuang, Chun-Liang Li, Zifeng Wang +7
2024-07-04
Computation and Language · Computer Science
Positional Biases Shift as Inputs Approach Context Window Limits
Blerta Veseli, Julian Chibane, Mariya Toneva, Alexander Koller
2025-08-12
Computation and Language · Computer Science
Pause-Tuning for Long-Context Comprehension: A Lightweight Approach to LLM Attention Recalibration
James Begin, Namit Agrawal, Eshan Singh, Yicheng Fu +3
2025-03-03
Computation and Language · Computer Science
What Works for 'Lost-in-the-Middle' in LLMs? A Study on GM-Extract and Mitigations
Mihir Gupte, Eshan Dixit, Muhammad Tayyab, Arun Adiththan
2025-11-19
Computation and Language · Computer Science
Context Length Alone Hurts LLM Performance Despite Perfect Retrieval
Yufeng Du, Minyang Tian, Srikanth Ronanki, Subendhu Rongali +6
2025-10-08
Computation and Language · Computer Science
Evaluating Multilingual Long-Context Models for Retrieval and Reasoning
Ameeta Agrawal, Andy Dang, Sina Bagheri Nezhad, Rhitabrat Pokharel +1
2024-10-15
Computation and Language · Computer Science
Efficient Solutions For An Intriguing Failure of LLMs: Long Context Window Does Not Mean LLMs Can Analyze Long Sequences Flawlessly
Peyman Hosseini, Ignacio Castro, Iacopo Ghinassi, Matthew Purver
2024-12-23
Computation and Language · Computer Science
FACT: Examining the Effectiveness of Iterative Context Rewriting for Multi-fact Retrieval
Jinlin Wang, Suyuchen Wang, Ziwen Xia, Sirui Hong +3
2024-10-29
Computation and Language · Computer Science
Training With "Paraphrasing the Original Text" Teaches LLM to Better Retrieve in Long-context Tasks
Yijiong Yu, Yongfeng Huang, Zhixiao Qi, Zhe Zhou
2024-12-13
Computation and Language · Computer Science
Lost in the Middle: How Language Models Use Long Contexts
Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape +3
2023-11-22
Computation and Language · Computer Science
FltLM: An Intergrated Long-Context Large Language Model for Effective Context Filtering and Understanding
Jingyang Deng, Zhengyang Shen, Boyang Wang, Lixin Su +5
2024-10-10
Computation and Language · Computer Science
GPT-5 vs Other LLMs in Long Short-Context Performance
Nima Esmi, Maryam Nezhad-Moghaddam, Fatemeh Borhani, Asadollah Shahbahrami +2
2026-02-25
Artificial Intelligence · Computer Science
Lost-in-Distance: Impact of Contextual Proximity on LLM Performance in Graph Tasks
Hamed Firooz, Maziar Sanjabi, Wenlong Jiang, Xiaoling Zhai
2025-01-03
Computation and Language · Computer Science
A MapReduce Approach to Effectively Utilize Long Context Information in Retrieval Augmented Language Models
Gongbo Zhang, Zihan Xu, Qiao Jin, Fangyi Chen +7
2024-12-23
Computation and Language · Computer Science
Hidden in the Haystack: Smaller Needles are More Difficult for LLMs to Find
Owen Bianchi, Mathew J. Koretsky, Maya Willey, Chelsea X. Alvarado +6
2025-12-18
Artificial Intelligence · Computer Science
Escaping the Context Bottleneck: Active Context Curation for LLM Agents via Reinforcement Learning
Xiaozhe Li, Tianyi Lyu, Yizhao Yang, Liang Shan +5
2026-04-14
Computation and Language · Computer Science
Context Shapes LLMs Retrieval-Augmented Fact-Checking Effectiveness
Pietro Bernardelle, Stefano Civelli, Kevin Roitero, Gianluca Demartini
2026-02-25
Computation and Language · Computer Science
Long-context LLMs Struggle with Long In-context Learning
Tianle Li, Ge Zhang, Quy Duc Do, Xiang Yue +1
2024-06-13