Prior benchmarks have analyzed models' answers to questions about videos in order to measure visual compositional reasoning. Action Genome Question Answering (AGQA) is one such benchmark. AGQA provides a training/test split with balanced answer distributions to reduce the effect of linguistic biases. However, some biases remain in several AGQA categories. We introduce AGQA 2.0, a version of this benchmark with several improvements, most namely a stricter balancing procedure. We then report results on the updated benchmark for all experiments.
@article{arxiv.2204.06105,
title = {AGQA 2.0: An Updated Benchmark for Compositional Spatio-Temporal Reasoning},
author = {Madeleine Grunde-McLaughlin and Ranjay Krishna and Maneesh Agrawala},
journal= {arXiv preprint arXiv:2204.06105},
year = {2022}
}
Comments
7 pages, 2 figures, 7 tables, update to AGQA arXiv:2103.16002