English

TVWorld: Foundations for Remote-Control TV Agents

Computer Vision and Pattern Recognition 2026-01-21 v1 Artificial Intelligence Computation and Language

Abstract

Recent large vision-language models (LVLMs) have demonstrated strong potential for device control. However, existing research has primarily focused on point-and-click (PnC) interaction, while remote-control (RC) interaction commonly encountered in everyday TV usage remains largely underexplored. To fill this gap, we introduce \textbf{TVWorld}, an offline graph-based abstraction of real-world TV navigation that enables reproducible and deployment-free evaluation. On this basis, we derive two complementary benchmarks that comprehensively assess TV-use capabilities: \textbf{TVWorld-N} for topology-aware navigation and \textbf{TVWorld-G} for focus-aware grounding. These benchmarks expose a key limitation of existing agents: insufficient topology awareness for focus-based, long-horizon TV navigation. Motivated by this finding, we propose a \emph{Topology-Aware Training} framework that injects topology awareness into LVLMs. Using this framework, we develop \textbf{TVTheseus}, a foundation model specialized for TV navigation. TVTheseus achieves a success rate of 68.3%68.3\% on TVWorld-N, surpassing strong closed-source baselines such as Gemini 3 Flash and establishing state-of-the-art (SOTA) performance. Additional analyses further provide valuable insights into the development of effective TV-use agents.

Cite

@article{arxiv.2601.13142,
  title  = {TVWorld: Foundations for Remote-Control TV Agents},
  author = {Zhantao Ma and Quanfeng Lu and Shuai Zhong and Dahai Yu and Ping Luo and Michael K. Ng},
  journal= {arXiv preprint arXiv:2601.13142},
  year   = {2026}
}
R2 v1 2026-07-01T09:10:45.969Z