Related papers: A statistical model for aggregating judgments by i…
How should one combine noisy information from diverse sources to make an inference about an objective ground truth? This frequently recurring, normative question lies at the core of statistics, machine learning, policy-making, and everyday…
We present a new method for probabilistic elicitation of expert knowledge using binary responses of human experts assessing simulated data from a statistical model, where the parameters are subject to uncertainty. The binary responses…
Complex systems show the capacity to aggregate information and to display coordinated activity. In the case of social systems the interaction of different individuals leads to the emergence of norms, trends in political positions, opinions,…
We consider two-alternative elections where voters' preferences depend on a state variable that is not directly observable. Each voter receives a private signal that is correlated to the state variable. Voters may be "contingent" with…
We propose a new probabilistic graphical model that jointly models the difficulties of questions, the abilities of participants and the correct answers to questions in aptitude testing and crowdsourcing settings. We devise an active…
We study the problem of eliciting and aggregating probabilistic information from multiple agents. In order to successfully aggregate the predictions of agents, the principal needs to elicit some notion of confidence from agents, capturing…
Opinion surveys can contain closed questions to which respondents can give multiple answers. We propose to model these data as networks in which vertices are eligible items and arcs are respondents. This representation opens up the…
Conformal prediction methods are statistical tools designed to quantify uncertainty and generate predictive sets with guaranteed coverage probabilities. This work introduces an innovative refinement to these methods for classification…
Collective intelligence, which aggregates the shared information from large crowds, is often negatively impacted by unreliable information sources with the low quality data. This becomes a barrier to the effective use of collective…
A statistical estimation model with qualitative input provides a mechanism to fuse human intuition in the form of qualitative information into a statistical model. We investigate the statistical properties of this model and devise a…
An important question when eliciting opinions from experts is how to aggregate the reported opinions. In this paper, we propose a pooling method to aggregate expert opinions. Intuitively, it works as if the experts were continuously…
We proposed a probabilistic approach to joint modeling of participants' reliability and humans' regularity in crowdsourced affective studies. Reliability measures how likely a subject will respond to a question seriously; and regularity…
In a crowd forecasting system, aggregation is an algorithm that returns aggregated probabilities for each question based on the probabilities provided per question by each individual in the crowd. Various aggregation methods have been…
We study a model of a population making a binary decision based on information spreading within the population, which is fully connected or covering a square grid. We assume that a fraction of the population wants to make the choice of the…
In this paper we explore the application of methods for classical judgment aggregation in pooling probabilistic opinions on logically related issues. For this reason, we first modify the Boolean judgment aggregation framework in the way…
We consider the task of aggregating beliefs of severalexperts. We assume that these beliefs are represented as probabilitydistributions. We argue that the evaluation of any aggregationtechnique depends on the semantic context of this task.…
We propose a method for combining probabilistic outputs of classifiers to make a single consensus class prediction when no further information about the individual classifiers is available, beyond that they have been trained for the same…
Recommender systems leverage product and community information to target products to consumers. Researchers have developed collaborative recommenders, content-based recommenders, and (largely ad-hoc) hybrid systems. We propose a unified…
Aggregating signals from a collection of noisy sources is a fundamental problem in many domains including crowd-sourcing, multi-agent planning, sensor networks, signal processing, voting, ensemble learning, and federated learning. The core…
We consider the problem of information aggregation in federated decision making, where a group of agents collaborate to infer the underlying state of nature without sharing their private data with the central processor or each other. We…