Related papers: Incentive-Compatible Forecasting Competitions
Peer review (e.g., grading assignments in Massive Open Online Courses (MOOCs), academic paper review) is an effective and scalable method to evaluate the products (e.g., assignments, papers) of a large number of agents when the number of…
Standard conformal prediction offers a marginal guarantee on coverage, but for prediction sets to be truly useful, they should ideally ensure coverage conditional on each test point. Unfortunately, it is impossible to achieve exact,…
Prediction markets are often used as mechanisms to aggregate information about a future event, for example, whether a candidate will win an election. The event is typically assumed to be exogenous. In reality, participants may influence the…
We study allocation problems without monetary transfers where agents have correlated types, i.e., hold private information about one another. Such peer information is relevant in various settings, including science funding, allocation of…
The classical Prophet Inequality arises from a fundamental problem in optimal-stopping theory. In this problem, a gambler sees a finite sequence of independent, non-negative random variables. If he stops the sequence at any time, he…
Predictive models are often introduced to decision-making tasks under the rationale that they improve performance over an existing decision-making policy. However, it is challenging to compare predictive performance against an existing…
We demonstrate that the forecasting combination puzzle is a consequence of the methodology commonly used to produce forecast combinations. By the combination puzzle, we refer to the empirical finding that predictions formed by combining…
Because high-quality data is like oxygen for AI systems, effectively eliciting information from crowdsourcing workers has become a first-order problem for developing high-performance machine learning algorithms. Two prevalent paradigms,…
We present a model of a forecaster who must predict the future value of a variable that depends on an exogenous state and on the intervention of a policy-maker. We investigate the incentives of the forecaster to acquire costly private…
Abstaining classifiers have the option to refrain from providing a prediction for instances that are difficult to classify. The abstention mechanism is designed to trade off the classifier's performance on the accepted data while ensuring a…
In many areas of industry and society, e.g., energy, healthcare, logistics, agents collect vast amounts of data that they deem proprietary. These data owners extract predictive information of varying quality and relevance from data…
Peer prediction is a method to promote contributions of information by users in settings in which there is no way to verify the quality of responses. In multi-task peer prediction, the reports from users across multiple tasks are used to…
We investigate the performance and sampling variability of estimated forecast combinations, with particular attention given to the combination of forecast distributions. Unknown parameters in the forecast combination are optimized according…
Consider two forecasters, each making a single prediction for a sequence of events over time. We ask a relatively basic question: how might we compare these forecasters, either online or post-hoc, while avoiding unverifiable assumptions on…
Conformal predictors are machine learning algorithms that output prediction sets that have a guarantee of marginal validity for finite samples with minimal distributional assumptions. This is a property that makes conformal predictors…
Recommender systems are being employed across an increasingly diverse set of domains that can potentially make a significant social and individual impact. For this reason, considering fairness is a critical step in the design and evaluation…
Since the rise of fair machine learning as a critical field of inquiry, many different notions on how to quantify and measure discrimination have been proposed in the literature. Some of these notions, however, were shown to be mutually…
Optimistic rollups are a popular and promising method of increasing the throughput capacity of their underlying chain. These methods rely on economic incentives to guarantee their security. We present a model of optimistic rollups that…
The primary contribution of this paper resides in devising constant-factor approximation guarantees for revenue maximization in two-sided matching markets, under general pairwise rewards. A major distinction between our work and…
As recommender systems are being designed and deployed for an increasing number of socially-consequential applications, it has become important to consider what properties of fairness these systems exhibit. There has been considerable…