Affordance is crucial for intelligent robots in the context of object manipulation. In this paper, we argue that affordance should be task-/instruction-dependent, which is overlooked by many previous works. That is, different instructions can lead to different manipulation regions and directions even for the same object. According to this observation, we present a new dataset comprising fifteen thousand object-instruction-affordance triplets. All scenes in the dataset are from an egocentric viewpoint, designed to approximate the perspective of a human-like robot. Furthermore, we investigate how to enable large multimodal models (LMMs) to serve as affordance predictors by implementing a ``search against verifiers'' pipeline. An LMM is asked to progressively predict affordances, with the output at each step being verified by itself during the iterative process, imitating a reasoning process. Experiments show that our method not only unlocks new instruction-oriented affordance prediction capabilities, but also achieves outstanding performance broadly.
@article{arxiv.2508.17922,
title = {Egocentric Instruction-oriented Affordance Prediction via Large Multimodal Model},
author = {Bokai Ji and Jie Gu and Xiaokang Ma and Chu Tang and Jingmin Chen and Guangxia Li},
journal= {arXiv preprint arXiv:2508.17922},
year = {2025}
}