Related papers: Hedging and Approximate Truthfulness in Traditiona…
Prediction markets are useful for estimating probabilities of claims whose truth will be revealed at some fixed time -- this includes questions about the values of real-world events (i.e. statistical uncertainty), and questions about the…
Scoring rules measure the deviation between a probabilistic forecast and reality. Strictly proper scoring rules have the property that for any forecast, the mathematical expectation of the score of a forecast p by the lights of p is…
Benchmarking the capabilities of AI systems, including Large Language Models (LLMs) and Vision Models, typically ignores the impact of uncertainty in the underlying ground truth answers from experts. This ambiguity is not just limited to…
A common form of competition is one where judges grade contestants' performances which are then compiled to determine the final ranking of the contestants. Unlike in another common form of competition where two contestants play a…
An algorithm that outputs predictions about the state of the world will almost always be designed with the implicit or explicit goal of outputting accurate predictions (i.e., predictions that are likely to be true). In addition, the rise of…
This paper studies how insurers can chose which claims to investigate for fraud. Given a prediction model, typically only claims with the highest predicted propability of being fraudulent are investigated. We argue that this can lead to…
Algorithmic predictions are increasingly informing societal resource allocations by identifying individuals for targeting. Policymakers often build these systems with the assumption that by gathering more observations on individuals, they…
Reinforcement Learning from Human Feedback significantly enhances Natural Language Processing by aligning language models with human expectations. A critical factor in this alignment is the strength of reward models used during training.…
Can LLMs accurately adjust their confidence when facing opposition? Building on previous studies measuring calibration on static fact-based question-answering tasks, we evaluate Large Language Models (LLMs) in a dynamic, adversarial debate…
Despite extensive theoretical research on proportionality in approval-based multiwinner voting, its impact on which committees and candidates can be selected in practice remains poorly understood. We address this gap by (i) analyzing the…
We study the design of effort-maximizing grading schemes between agents with private abilities. Assuming agents derive value from the information their grade reveals about their ability, we find that more informative grading schemes induce…
This paper reports experimental data describing the dynamics of three key information-sharing outcomes: quantity of information shared, falsification and accuracy. The experimental design follows a formal model predicting that cooperative…
The role of skill (fitness) and luck (randomness) as driving forces on the dynamics of resource accumulation in a myriad of systems have long puzzled scientists. Fueled by undisputed inequalities that emerge from actual competitions, there…
In public discussions of the quality of forecasts, attention typically focuses on the predictive performance in cases of extreme events. However, the restriction of conventional forecast evaluation methods to subsets of extreme observations…
The setting of the classic prophet inequality is as follows: a gambler is shown the probability distributions of $n$ independent, non-negative random variables with finite expectations. In their indexed order, a value is drawn from each…
A seller wants to sell a good to a set of bidders using a credible mechanism. We show that when the seller has private information about her cost, it is impossible for a static mechanism to achieve the optimal revenue. In particular, even…
Systematically biased forecasts are typically interpreted as evidence of forecasters' irrationality and/or asymmetric loss. In this paper we propose an alternative explanation: when forecasts inform policy decisions, and the resulting…
Online information ecosystems are now central to our everyday social interactions. Of the many opportunities and challenges this presents, the capacity for artificial agents to shape individual and collective human decision-making in such…
We reveal a contradiction in measure-theoretic probability. The contradiction is an "equation" $1/2 = 0$ with its two sides representing probabilities. Unlike known paradoxes in mathematics, the revealed contradiction cannot be explained…
This paper gives game-theoretic versions of several results on "merging of opinions" obtained in measure-theoretic probability and algorithmic randomness theory. An advantage of the game-theoretic versions over the measure-theoretic results…