English
Related papers

Related papers: Masked Diffusion Vision-Language Models for Tempor…

200 papers

While large language models (LLMs) excel at static scientific reasoning, they struggle to model the temporal structure of dynamic physical processes. We present EvoMD-LLM (Evolutionary Molecular Dynamics Large Language Model), a framework…

Artificial Intelligence · Computer Science 2026-05-29 Zhichen Tang , Zhengzheng Dang , Yulin Chen , Jixin Wu , Haiwen Li , Yanming Wang

Point-Level temporal action localization (PTAL) aims to localize actions in untrimmed videos with only one timestamp annotation for each action instance. Existing methods adopt the frame-level prediction paradigm to learn from the sparse…

Computer Vision and Pattern Recognition · Computer Science 2020-12-16 Chen Ju , Peisen Zhao , Ya Zhang , Yanfeng Wang , Qi Tian

Recently, temporal action localization (TAL) has garnered significant interest in information retrieval community. However, existing supervised/weakly supervised methods are heavily dependent on extensive labeled temporal boundaries and…

Computer Vision and Pattern Recognition · Computer Science 2025-04-01 Yupeng Hu , Han Jiang , Hao Liu , Kun Wang , Haoyu Tang , Liqiang Nie

We propose TAL-Net, an improved approach to temporal action localization in video that is inspired by the Faster R-CNN object detection framework. TAL-Net addresses three key shortcomings of existing approaches: (1) we improve receptive…

Computer Vision and Pattern Recognition · Computer Science 2018-04-23 Yu-Wei Chao , Sudheendra Vijayanarasimhan , Bryan Seybold , David A. Ross , Jia Deng , Rahul Sukthankar

Although vision models such as Contrastive Language-Image Pre-Training (CLIP) show impressive generalization performance, their zero-shot robustness is still limited under Out-of-Distribution (OOD) scenarios without fine-tuning. Instead of…

Computer Vision and Pattern Recognition · Computer Science 2024-05-30 Zhuo Huang , Chang Liu , Yinpeng Dong , Hang Su , Shibao Zheng , Tongliang Liu

Masked Diffusion Language Models (MDLMs) have recently emerged as a promising alternative to Autoregressive Language Models (ARLMs), leveraging a denoising objective that, in principle, should enable more uniform context utilisation. In…

Machine Learning · Computer Science 2025-11-27 Julianna Piskorz , Cristina Pinneri , Alvaro Correia , Motasem Alfarra , Risheek Garrepalli , Christos Louizos

Prompt learning has achieved great success in efficiently exploiting large-scale pre-trained models in natural language processing (NLP). It reformulates the downstream tasks as the generative pre-training ones to achieve consistency, thus…

Computer Vision and Pattern Recognition · Computer Science 2023-12-18 Ning Liao , Bowen Shi , Xiaopeng Zhang , Min Cao , Junchi Yan , Qi Tian

Video temporal grounding (VTG), which localizes the start and end times of a queried event in an untrimmed video, is a key test of whether multimodal large language models (MLLMs) understand not only what happens but also when it happens.…

Computer Vision and Pattern Recognition · Computer Science 2026-05-22 Dazhao Du , Liao Duan , Jian Liu , Tao Han , Yujia Zhang , Eric Liu , Xi Chen , Song Guo

Weakly-Supervised Temporal Action Localization (WSTAL) aims to localize actions in untrimmed videos with only video-level labels. Currently, most state-of-the-art WSTAL methods follow a Multi-Instance Learning (MIL) pipeline: producing…

Computer Vision and Pattern Recognition · Computer Science 2022-08-24 Meng Cao , Can Zhang , Long Chen , Mike Zheng Shou , Yuexian Zou

Temporal Action Localization (TAL) has experienced remarkable success under the supervised learning paradigm. However, existing TAL methods are rooted in the closed set assumption, which cannot handle the inevitable unknown actions in…

Computer Vision and Pattern Recognition · Computer Science 2022-03-11 Wentao Bao , Qi Yu , Yu Kong

Natural Language Video Localization (NLVL), grounding phrases from natural language descriptions to corresponding video segments, is a complex yet critical task in video understanding. Despite ongoing advancements, many existing solutions…

Computer Vision and Pattern Recognition · Computer Science 2024-01-17 Chongzhi Zhang , Mingyuan Zhang , Zhiyang Teng , Jiayi Li , Xizhou Zhu , Lewei Lu , Ziwei Liu , Aixin Sun

We present our solution to the BinEgo-360 Challenge at ICCV 2025, which focuses on temporal action localization (TAL) in multi-perspective and multi-modal video settings. The challenge provides a dataset containing panoramic, third-person,…

Computer Vision and Pattern Recognition · Computer Science 2025-12-15 Anh-Kiet Duong , Petra Gomez-Krämer

Large Language Models (LLMs) and Vision-Language Models (VLMs) have emerged as promising candidates for end-to-end autonomous driving. However, these models typically face challenges in inference latency, action precision, and…

Computer Vision and Pattern Recognition · Computer Science 2026-02-25 Jiaru Zhang , Manav Gagvani , Can Cui , Juntong Peng , Ruqi Zhang , Ziran Wang

Temporal Action Localization (TAL) in untrimmed video is important for many applications. But it is very expensive to annotate the segment-level ground truth (action class and temporal boundary). This raises the interest of addressing TAL…

Computer Vision and Pattern Recognition · Computer Science 2018-12-18 Zheng Shou , Hang Gao , Lei Zhang , Kazuyuki Miyazawa , Shih-Fu Chang

Real-world videos often contain overlapping events and complex temporal dependencies, making multimodal interaction modeling particularly challenging. We introduce DEL, a framework for dense semantic action localization, aiming to…

Computer Vision and Pattern Recognition · Computer Science 2025-07-01 Mona Ahmadian , Amir Shirian , Frank Guerin , Andrew Gilbert

Human perception of events is intrinsically tied to distinguishing between completed (perfect and telic) and ongoing (durative) actions, a process mediated by both linguistic structure and visual cues. In this work, we introduce the…

Computer Vision and Pattern Recognition · Computer Science 2025-06-03 Olga Loginova , Sofía Ortega Loguinova

Video-language pre-trained models have shown remarkable success in guiding video question-answering (VideoQA) tasks. However, due to the length of video sequences, training large-scale video-based models incurs considerably higher costs…

Computer Vision and Pattern Recognition · Computer Science 2023-08-17 Guangyi Chen , Xiao Liu , Guangrun Wang , Kun Zhang , Philip H. S. Torr , Xiao-Ping Zhang , Yansong Tang

Dense video captioning aims to interpret and describe all temporally localized events throughout an input video. Recent state-of-the-art methods leverage large language models (LLMs) to provide detailed moment descriptions for video data.…

Computer Vision and Pattern Recognition · Computer Science 2026-01-07 Wei-Yuan Cheng , Kai-Po Chang , Chi-Pin Huang , Fu-En Yang , Yu-Chiang Frank Wang

Temporal action localization is an important step towards video understanding. Most current action localization methods depend on untrimmed videos with full temporal annotations of action instances. However, it is expensive and…

Computer Vision and Pattern Recognition · Computer Science 2020-01-23 Ashraful Islam , Richard J. Radke

Multimodal Large Language Models (MLLMs) have significantly improved performance across various image-language applications. Recently, there has been a growing interest in adapting image pre-trained MLLMs for video-related tasks. However,…

Computer Vision and Pattern Recognition · Computer Science 2024-09-06 Mingze Gao , Jingyu Liu , Mingda Li , Jiangtao Xie , Qingbin Liu , Bo Zhao , Xi Chen , Hui Xiong