On Optimizing Human-Machine Task Assignments
Human-Computer Interaction
2015-09-28 v1 Computer Vision and Pattern Recognition
Abstract
When crowdsourcing systems are used in combination with machine inference systems in the real world, they benefit the most when the machine system is deeply integrated with the crowd workers. However, if researchers wish to integrate the crowd with "off-the-shelf" machine classifiers, this deep integration is not always possible. This work explores two strategies to increase accuracy and decrease cost under this setting. First, we show that reordering tasks presented to the human can create a significant accuracy improvement. Further, we show that greedily choosing parameters to maximize machine accuracy is sub-optimal, and joint optimization of the combined system improves performance.
Cite
@article{arxiv.1509.07543,
title = {On Optimizing Human-Machine Task Assignments},
author = {Andreas Veit and Michael Wilber and Rajan Vaish and Serge Belongie and James Davis and Vishal Anand and Anshu Aviral and Prithvijit Chakrabarty and Yash Chandak and Sidharth Chaturvedi and Chinmaya Devaraj and Ankit Dhall and Utkarsh Dwivedi and Sanket Gupte and Sharath N. Sridhar and Karthik Paga and Anuj Pahuja and Aditya Raisinghani and Ayush Sharma and Shweta Sharma and Darpana Sinha and Nisarg Thakkar and K. Bala Vignesh and Utkarsh Verma and Kanniganti Abhishek and Amod Agrawal and Arya Aishwarya and Aurgho Bhattacharjee and Sarveshwaran Dhanasekar and Venkata Karthik Gullapalli and Shuchita Gupta and Chandana G and Kinjal Jain and Simran Kapur and Meghana Kasula and Shashi Kumar and Parth Kundaliya and Utkarsh Mathur and Alankrit Mishra and Aayush Mudgal and Aditya Nadimpalli and Munakala Sree Nihit and Akanksha Periwal and Ayush Sagar and Ayush Shah and Vikas Sharma and Yashovardhan Sharma and Faizal Siddiqui and Virender Singh and Abhinav S. and Anurag. D. Yadav},
journal= {arXiv preprint arXiv:1509.07543},
year = {2015}
}
Comments
HCOMP 2015 Work in Progress