Detecting actions as they occur is essential for applications like video surveillance, autonomous driving, and human-robot interaction. Known as online action detection, this task requires classifying actions in streaming videos, handling background noise, and coping with incomplete actions. Transformer architectures are the current state-of-the-art, yet the potential of recent advancements in computer vision, particularly vision-language models (VLMs), remains largely untapped for this problem, partly due to high computational costs. In this paper, we introduce TOAD: a Text-driven Online Action Detection architecture that supports zero-shot and few-shot learning. TOAD leverages CLIP (Contrastive Language-Image Pretraining) textual embeddings, enabling efficient use of VLMs without significant computational overhead. Our model achieves 82.46% mAP on the THUMOS14 dataset, outperforming existing methods, and sets new baselines for zero-shot and few-shot performance on the THUMOS14 and TVSeries datasets.
@article{arxiv.2501.13518,
title = {Text-driven Online Action Detection},
author = {Manuel Benavent-Lledo and David Mulero-Pérez and David Ortiz-Perez and Jose Garcia-Rodriguez},
journal= {arXiv preprint arXiv:2501.13518},
year = {2026}
}
Comments
Published in Integrated Computer-Aided Engineering