Egocentric memory is widely used in embodied intelligence, but it may be insufficient for comprehensive spatial-temporal reasoning. Inspired by human recall from both field and observer perspectives, we introduce EgoExoMem, the first benchmark for cross-view memory reasoning over synchronized egocentric and exocentric videos. EgoExoMem contains 2.6K high-quality MCQs across eight temporal, spatial, and cross-view QA types. To support dual-view retrieval, we propose E2-Select, a training-free frame selection method for synchronized ego-exo videos. It combines relevance-based budget allocation with per-view k-DPP sampling to handle view asymmetry and cross-view temporal consistency. Experiments show that ego and exo views provide complementary memory cues, while existing MLLMs remain far from solving the benchmark: the best model reaches only 55.3%. E2-Select achieves state-of-the-art performance of 58.2% over frame-selection and RAG-based memory baselines. Further analysis reveals systematic view-preference conflicts between question framing and answer grounding, underscoring the novelty and challenge of cross-view memory reasoning.
@article{arxiv.2605.18734,
title = {EgoExoMem: Cross-View Memory Reasoning over Synchronized Egocentric and Exocentric Videos},
author = {Ruiping Liu and Junwei Zheng and Yufan Chen and Di Wen and Shaofang Quan and Chengzhi Wu and Jiaming Zhang and Kailun Yang and Kunyu Peng and Rainer Stiefelhagen},
journal= {arXiv preprint arXiv:2605.18734},
year = {2026}
}
Comments
The source code and dataset can be found at https://github.com/RuipingL/EgoExoMem