English

PLOT-TAL: Prompt Learning with Optimal Transport for Few-Shot Temporal Action Localization

Computer Vision and Pattern Recognition 2025-07-25 v2 Machine Learning

Abstract

Few-shot temporal action localization (TAL) methods that adapt large models via single-prompt tuning often fail to produce precise temporal boundaries. This stems from the model learning a non-discriminative mean representation of an action from sparse data, which compromises generalization. We address this by proposing a new paradigm based on multi-prompt ensembles, where a set of diverse, learnable prompts for each action is encouraged to specialize on compositional sub-events. To enforce this specialization, we introduce PLOT-TAL, a framework that leverages Optimal Transport (OT) to find a globally optimal alignment between the prompt ensemble and the video's temporal features. Our method establishes a new state-of-the-art on the challenging few-shot benchmarks of THUMOS'14 and EPIC-Kitchens, without requiring complex meta-learning. The significant performance gains, particularly at high IoU thresholds, validate our hypothesis and demonstrate the superiority of learning distributed, compositional representations for precise temporal localization.

Keywords

Cite

@article{arxiv.2403.18915,
  title  = {PLOT-TAL: Prompt Learning with Optimal Transport for Few-Shot Temporal Action Localization},
  author = {Edward Fish and Andrew Gilbert},
  journal= {arXiv preprint arXiv:2403.18915},
  year   = {2025}
}

Comments

Accepted to ICCVWS

R2 v1 2026-06-28T15:36:04.782Z