相关论文: Game-theoretic statistics and safe anytime-valid i…
In the past two decades, AB testing has proliferated to optimise products in digital domains. Traditional AB tests use fixed-horizon testing, determining the sample size of the experiment and continuing until the experiment has concluded.…
In many multi-agent systems, agents interact repeatedly and are expected to settle into stable, rational behavior over time. Yet in practice, behavior often drifts, and detecting such deviations in real time remains an open challenge. We…
In a Monte-Carlo test, the observed dataset is fixed, and several resampled or permuted versions of the dataset are generated in order to test a null hypothesis that the original dataset is exchangeable with the resampled/permuted ones.…
Researchers in explainable artificial intelligence have developed numerous methods for helping users understand the predictions of complex supervised learning models. By contrast, explaining the $\textit{uncertainty}$ of model outputs has…
Game-theoretic upper expectations are joint (global) probability models that mathematically describe the behaviour of uncertain processes in terms of supermartingales; capital processes corresponding to available betting strategies.…
The assessment of safety is an important aspect of the evaluation of new therapies in clinical trials, with analyses of adverse events being an essential part of this. Standard methods for the analysis of adverse events such as the…
The usual way of testing probability forecasts in game-theoretic probability is via construction of test martingales. The standard assumption is that all forecasts are output by the same forecaster. In this paper I will discuss possible…
Experimentation involves risk. The investigator expends time and money in the pursuit of data that supports a hypothesis. In the end, the investigator may find that all of these costs were for naught and the data fail to reject the null.…
In this paper, we address the problem of testing exchangeability of a sequence of random variables, $X_1, X_2,\cdots$. This problem has been studied under the recently popular framework of testing by betting. But the mapping of testing…
Confidence sequences, anytime p-values (called p-processes in this paper), and e-processes all enable sequential inference for composite and nonparametric classes of distributions at arbitrary stopping times. Examining the literature, one…
Given a random sample from a random variable $T$ which is bounded from above, $T\le\tau$ a.s., we define processes that are positive supermartingales if $E(T)\ge\mu$. Such processes are called test martingales. Tests of the supermartingale…
Generalist robot manipulation policies are becoming increasingly capable, but are limited in evaluation to a small number of hardware rollouts. This strong resource constraint in real-world testing necessitates both more informative…
We study the problem of designing consistent sequential two-sample tests in a nonparametric setting. Guided by the principle of testing by betting, we reframe this task into that of selecting a sequence of payoff functions that maximize the…
We congratulate Waudby-Smith and Ramdas for their interesting paper \cite{waudbysmith2022estimating} in generating confidence intervals and time-uniform confidence sequences for mean estimation with bounded observations. Their methodology…
Social Explainable AI (SAI) is a new direction in artificial intelligence that emphasises decentralisation, transparency, social context, and focus on the human users. SAI research is still at an early stage. Consequently, it concentrates…
Hypothesis testing via e-variables can be framed as a sequential betting game, where a player each round picks an e-variable. A good player's strategy results in an effective statistical test that rejects the null hypothesis as soon as…
We provide practical, efficient, and nonparametric methods for auditing the fairness of deployed classification and regression models. Whereas previous work relies on a fixed-sample size, our methods are sequential and allow for the…
This work contains the mathematical exploration of a few prototypical games in which central concepts from statistics and probability theory naturally emerge. The first two kinds of games are termed Fisher and Bayesian games, which are…
Delayed outcomes are ubiquitous in online experimentation. When such a temporal dimension is present, treatment influences not only the outcome value but also the outcome timing, which can move in opposite directions. Motivated by the…
We design sequential tests for a large class of nonparametric null hypotheses based on elicitable and identifiable functionals. Such functionals are defined in terms of scoring functions and identification functions, which are ideal…