Related papers: A Sharp Test for the Judge Leniency Design
This paper studies a potential outcome model with a continuous or discrete outcome, a discrete multi-valued treatment, and a discrete multi-valued instrument. We derive sharp, closed-form testable implications for a class of restrictions on…
We develop a step-by-step guide to leniency (a.k.a. judge or examiner instrument) designs, drawing on recent econometric literatures. The unbiased jackknife instrumental variables estimator (UJIVE) is purpose-built for leveraging exogenous…
We present the Judge Reliability Harness, an open source library for constructing validation suites that test the reliability of LLM judges. As LLM based scoring is widely deployed in AI benchmarks, more tooling is needed to efficiently…
This paper considers inference in a linear instrumental variable regression model with many potentially weak instruments, in the presence of heterogeneous treatment effects. I first show that existing test procedures, including those that…
Matching is a commonly used causal inference study design in observational studies. Through matching on measured confounders between different treatment groups, valid randomization inferences can be conducted under the no unmeasured…
Fisherian randomization inference is often dismissed as testing an uninteresting and implausible hypothesis: the sharp null of no effects whatsoever. We show that this view is overly narrow. Many randomization tests are also valid under a…
This paper provides a solution to the evaluation of treatment effects in selective samples when neither instruments nor parametric assumptions are available. We provide sharp bounds for average treatment effects under a conditional…
This paper considers the estimation of treatment effects in randomized experiments with complex experimental designs, including cases with interference between units. We develop a design-based estimation theory for arbitrary experimental…
We develop randomization-based tests for heterogeneous treatment effects in the presence of network interference. Leveraging the exposure mapping framework, we study a broad class of null hypotheses that represent various forms of constant…
In this paper I derive a set of testable implications for econometric models defined by three assumptions: (i) the existence of strictly exogenous discrete instruments, (ii) restrictions on how the instruments affect adoption of a finite…
Observational studies provide invaluable opportunities to draw causal inference, but they may suffer from biases due to pretreatment difference between treated and control units. Matching is a popular approach to reduce observed covariate…
We develop sensitivity analyses for weak nulls in matched observational studies while allowing unit-level treatment effects to vary. The methods may be applied to studies using any optimal without-replacement matching algorithm. In contrast…
We study settings in which a researcher has an instrumental variable (IV) and seeks to evaluate the effects of a counterfactual policy that alters treatment assignment, such as a directive encouraging randomly assigned judges to release…
We propose and study a general method for construction of consistent statistical tests on the basis of possibly indirect, corrupted, or partially available observations. The class of tests devised in the paper contains Neyman's smooth…
A sensitivity analysis in an observational study tests whether the qualitative conclusions of an analysis would change if we were to allow for the possibility of limited bias due to confounding. The design sensitivity of a hypothesis test…
We consider the classical problem of estimating the mixing distribution of binomial mixtures, but under trial heterogeneity and smoothness. This problem has been studied extensively when the trial parameter is homogeneous, but not under the…
End-to-end models reach state-of-the-art performance for speech recognition, but global soft attention is not monotonic, which might lead to convergence problems, to instability, to bad generalisation, cannot be used for online streaming,…
This paper studies the validity of nonparametric tests used in the regression discontinuity design. The null hypothesis of interest is that the average treatment effect at the threshold in the so-called sharp design equals a pre-specified…
Despite extensive research on adversarial training strategies to improve robustness, the decisions of even the most robust deep learning models can still be quite sensitive to imperceptible perturbations, creating serious risks when…
This paper considers the problem of design-based inference for the average treatment effect in finely stratified experiments. Here, by "design-based'' we mean that the only source of uncertainty stems from the randomness in treatment…