Related papers: Elicitability
In order to build reliable and trustworthy NLP applications, models need to be both fair across different demographics and explainable. Usually these two objectives, fairness and explainability, are optimized and/or examined independently…
I consider decision-making constrained by considerations of morality, rationality, or other virtues. The decision maker (DM) has a true preference over outcomes, but feels compelled to choose among outcomes that are top-ranked by some…
This paper offers a step towards research infrastructure, which makes data from experimental economics efficiently usable for analysis of web data. We believe that regularities of human behavior found in experimental data also emerge in…
In many settings, an effective way of evaluating objects of interest is to collect evaluations from dispersed individuals and to aggregate these evaluations together. Some examples are categorizing online content and evaluating student…
For analysis of a high-dimensional dataset, a common approach is to test a null hypothesis of statistical independence on all variable pairs using a non-parametric measure of dependence. However, because this approach attempts to identify…
We consider crowdsourcing problems where the users are asked to provide evaluations for items; the user evaluations are then used directly, or aggregated into a consensus value. Lacking an incentive scheme, users have no motive in making…
Participation incentives is a well-known issue inhibiting randomized controlled trials (RCTs) in medicine, as well as a potential cause of user dissatisfaction for RCTs in online platforms. We frame this issue as a non-standard…
This paper develops a prior-free model of data-driven decision making in which the decision maker observes the entire distribution of signals generated by a known experiment under an unknown distribution of the state variable and evaluates…
An agent acquires a costly flexible signal before making a decision. We explore to what degree knowledge of the agent's information costs helps predict her behavior. We establish an impossibility result: learning costs alone generate no…
Consider the relationship between a regulator (the principal) and an experimenter (the agent) such as a pharmaceutical company. The pharmaceutical company wishes to sell a drug for profit, whereas the regulator wishes to allow only…
A sender with private preferences would like to influence a receiver's action by providing information through a statistical test. The technology for information production is controlled by a monopolist intermediary, who offers a menu of…
Machine learning algorithms enable advanced decision making in contemporary intelligent systems. Research indicates that there is a tradeoff between their model performance and explainability. Machine learning models with higher performance…
Algorithms are often used to produce decision-making rules that classify or evaluate individuals. When these individuals have incentives to be classified a certain way, they may behave strategically to influence their outcomes. We develop a…
In centralized mechanisms and platforms, participants do not fully observe each others' type reports. Hence, if there is a deviation from the promised mechanism, participants may be unable to detect it. We formalize a notion of auditabilty…
Explainable systems expose information about why certain observed effects are happening to the agents interacting with them. We argue that this constitutes a positive flow of information that needs to be specified, verified, and balanced…
What is the purpose of pre-analysis plans, and how should they be designed? We model the interaction between an agent who analyzes data and a principal who makes a decision based on agent reports. The agent could be the manufacturer of a…
A randomized trial and an analysis of observational data designed to emulate the trial sample observations separately, but have the same eligibility criteria, collect information on some shared baseline covariates, and compare the effects…
In this paper, we study belief elicitation about an uncertain future event, where the reports will affect a principal's decision. We study two problems that can arise in this setting: (1) Agents may have an interest in the outcome of the…
Our goal is for agents to optimize the right reward function, despite how difficult it is for us to specify what that is. Inverse Reinforcement Learning (IRL) enables us to infer reward functions from demonstrations, but it usually assumes…
An analyst observes the frequency with which a decision maker (DM) takes actions, but not the frequency conditional on payoff-relevant states. We ask when the analyst can rationalize the DM's choices as if the DM first learns something…