Related papers: When is a crowd wise?
Reputations provide a powerful mechanism to sustain cooperation, as individuals cooperate with those of good social standing. But how should moral reputations be updated as we observe social behavior, and when will a population converge on…
In this paper we study the effect on crowd worker efficiency and effectiveness of the dominance of one class in the data they process. We aim at understanding if there is any positive or negative bias in workers seeing many negative…
Prior studies on the effectiveness of professional jury consultants in predicting juror proclivities have yielded mixed results, and few have rigorously evaluated consultant performance against chance under controlled conditions. This study…
Automated scene analysis has been a topic of great interest in computer vision and cognitive science. Recently, with the growth of crowd phenomena in the real world, crowded scene analysis has attracted much attention. However, the visual…
We evaluate the folk wisdom that algorithmic decision rules trained on data produced by biased human decision-makers necessarily reflect this bias. We consider a setting where training labels are only generated if a biased decision-maker…
Background: Wisdom is a superordinate construct that embraces perspective taking, reflectiveness, prosocial orientation, reflective empathetic action, and intellectual humility. Unlike conventional models of reasoning that are rigidly bound…
Crowdsourcing is beginning to be used for policymaking. The wisdom of crowds [Surowiecki 2005], and crowdsourcing [Brabham 2008], are seen as new avenues that can shape all kinds of policy, from transportation policy [Nash 2009] to urban…
This paper discusses a crowdsourcing based method that we designed to quantify the importance of different attributes of a dataset in determining the outcome of a classification problem. This heuristic, provided by humans acts as the…
The wisdom of the crowd is a valuable asset in today's society. It is not only important in predicting elections but also plays an essential role in marketing and the financial industry. Having a trustworthy source of opinion can make…
Complex systems show the capacity to aggregate information and to display coordinated activity. In the case of social systems the interaction of different individuals leads to the emergence of norms, trends in political positions, opinions,…
The principle of rewarding a crowd for surprisingly common answers has been used in the literature for designing a number of truthful information elicitation mechanisms. A related method has also been proposed in the literature for better…
Recently, there has been a burst in the number of research projects on human computation via crowdsourcing. Multiple choice (or labeling) questions could be referred to as a common type of problem which is solved by this approach. As an…
We propose a simple model to explore an educational phenomenon where the correct answer emerges from group discussion. We construct our model based on several plausible assumptions: (i) We tend to follow peers' opinions. However, if a…
We revisit DeGroot learning to examine the robustness of social learning in dynamic networks -- networks that evolve randomly over time. Dynamics have double-edged effects depending on social structure: while they can foster consensus and…
Opinion evolution and judgment revision are mediated through social influence. Based on a large crowdsourced in vitro experiment (n=861), it is shown how a consensus model can be used to predict opinion evolution in online collective…
Decisions are often made by heterogeneous groups of individuals, each with distinct initial biases and access to information of different quality. We show that in large groups of independent agents who accumulate evidence the first to…
We investigate the behavior of experts who seek to make predictions with maximum impact on an audience. At a known future time, a certain continuous random variable will be realized. A public prediction gradually converges to the outcome,…
Microtask crowdsourcing has enabled dataset advances in social science and machine learning, but existing crowdsourcing schemes are too expensive to scale up with the expanding volume of data. To scale and widen the applicability of…
With the increasing size and frequency of mass events, the study of crowd disasters and the simulation of pedestrian flows have become important research areas. Yet, even successful modeling approaches such as those inspired by Newtonian…
Various measures can be used to estimate bias or unfairness in a predictor. Previous work has already established that some of these measures are incompatible with each other. Here we show that, when groups differ in prevalence of the…