Existing recommendation systems either rely on user interaction logs, such as online shopping history for shopping recommendations, or focus on text signals. However, item-based histories are not always accessible, and are not generalizable for multimodal recommendation. We hypothesize that a user's visual history -- comprising images from daily life -- can offer rich, task-agnostic insights into their interests and preferences, and thus be leveraged for effective personalization. To this end, we propose VisualLens, a novel framework that leverages multimodal large language models (MLLMs) to enable personalization using task-agnostic visual history. VisualLens extracts, filters, and refines a spectrum user profile from the visual history to support personalized recommendation. We created two new benchmarks, Google-Review-V and Yelp-V, with task-agnostic visual histories, and show that VisualLens improves over state-of-the-art item-based multimodal recommendations by 5-10% on Hit@3, and outperforms GPT-4o by 2-5%. Further analysis shows that VisualLens is robust across varying history lengths and excels at adapting to both longer histories and unseen content categories.
@article{arxiv.2411.16034,
title = {VisualLens: Personalization through Task-Agnostic Visual History},
author = {Wang Bill Zhu and Deqing Fu and Kai Sun and Yi Lu and Zhaojiang Lin and Seungwhan Moon and Kanika Narang and Mustafa Canim and Yue Liu and Anuj Kumar and Xin Luna Dong},
journal= {arXiv preprint arXiv:2411.16034},
year = {2025}
}