Agile Modeling: From Concept to Classifier in Minutes
Abstract
The application of computer vision to nuanced subjective use cases is growing. While crowdsourcing has served the vision community well for most objective tasks (such as labeling a "zebra"), it now falters on tasks where there is substantial subjectivity in the concept (such as identifying "gourmet tuna"). However, empowering any user to develop a classifier for their concept is technically difficult: users are neither machine learning experts, nor have the patience to label thousands of examples. In reaction, we introduce the problem of Agile Modeling: the process of turning any subjective visual concept into a computer vision model through a real-time user-in-the-loop interactions. We instantiate an Agile Modeling prototype for image classification and show through a user study (N=14) that users can create classifiers with minimal effort under 30 minutes. We compare this user driven process with the traditional crowdsourcing paradigm and find that the crowd's notion often differs from that of the user's, especially as the concepts become more subjective. Finally, we scale our experiments with simulations of users training classifiers for ImageNet21k categories to further demonstrate the efficacy.
Cite
@article{arxiv.2302.12948,
title = {Agile Modeling: From Concept to Classifier in Minutes},
author = {Otilia Stretcu and Edward Vendrow and Kenji Hata and Krishnamurthy Viswanathan and Vittorio Ferrari and Sasan Tavakkol and Wenlei Zhou and Aditya Avinash and Enming Luo and Neil Gordon Alldrin and MohammadHossein Bateni and Gabriel Berger and Andrew Bunner and Chun-Ta Lu and Javier A Rey and Giulia DeSalvo and Ranjay Krishna and Ariel Fuxman},
journal= {arXiv preprint arXiv:2302.12948},
year = {2023}
}