Related papers: Forecast dominance testing via sign randomization

Textual representations based on pre-trained language models are key, especially in few-shot learning scenarios. What makes a representation good for text classification? Is it due to the geometric properties of the space or because it is…

Given an imperfect predictor, we exploit additional features at test time to improve the predictions made, without retraining and without knowledge of the prediction function. This scenario arises if training labels or data are proprietary,…

We ask: Can focusing on likely classes of a single, in-domain sample improve model predictions? Prior work argued ``no''. We put forward a novel rationale in favor of ``yes'': Sharedness of features among classes indicates their reliability…

Stochastic dominance has been studied extensively, particularly in the finance and economics literature. In this paper, we obtain two results. First, necessary conditions for higher-order inverse stochastic dominance are developed. These…

We provide methods to validate and compare sensor outputs, or inference algorithms applied to sensor data, by adapting statistical scoring rules. The reported output should either be in the form of a prediction interval or of a parameter…

Statistical modeling plays a fundamental role in understanding the underlying mechanism of massive data (statistical inference) and predicting the future (statistical prediction). Although all models are wrong, researchers try their best to…

We consider the hypothesis testing problem of detecting a shift between the means of two multivariate normal distributions in the high-dimensional setting, allowing for the data dimension p to exceed the sample size n. Specifically, we…

Stochastic dominance of a random variable by a convex combination of its independent copies has recently been shown to hold within the relatively narrow class of distributions with concave odds function, and later extended to broader…

Statistical hypothesis testing serves as statistical evidence for scientific innovation. However, if the reported results are intentionally biased, hypothesis testing no longer controls the rate of false discovery. In particular, we study…

In this paper we study goodness-of-fit testing of single-index models. The large sample behavior of certain score-type test statistics is investigated. As a by-product, we obtain asymptotically distribution-free maximin tests for a large…

The paper discusses a test for the hypothesis that a random sample comes from the Cauchy distribution. The test statistics is derived from a characterization and is based on the characteristic function. Properties of the test are discussed…

We consider the problem of testing the equality of conditional distributions of a response variable given a vector of covariates between two populations. Such a hypothesis testing problem can be motivated from various machine learning and…

A user-focused verification approach for evaluating probability forecasts of binary outcomes (also known as probabilistic classifiers) is demonstrated that is (i) based on proper scoring rules, (ii) focuses on user decision thresholds, and…

This paper introduces a statistical test inferring whether a variable allows separating two classes by means of a single critical value. Its test statistic is the prediction error of a nonparametric threshold classifier. While this approach…

In causal inference, we can consider a situation in which treatment on one unit affects others, i.e., interference exists. In the presence of interference, we cannot perform a classical randomization test directly because a null hypothesis…

We present partial strategyproofness, a new, relaxed notion of strategyproofness for studying the incentive properties of non-strategyproof assignment mechanisms. Informally, a mechanism is partially strategyproof if it makes truthful…

We propose new statistical tests, in high-dimensional settings, for testing the independence of two random vectors and their conditional independence given a third random vector. The key idea is simple, i.e., we first transform each…

Probabilistic forecasts in the form of probability distributions over future events have become popular in several fields including meteorology, hydrology, economics, and demography. In typical applications, many alternative statistical…

Although unbiasedness is a basic property of a good test, many tests on vector parameters or scalar parameters against two-sided alternatives are not finite-sample unbiased. This was already noticed by Sugiura [Ann. Inst. Statist. Math. 17…

Football forecasting models traditionally rate teams on past match results, that is based on the number of goals scored. Goals, however, involve a high element of chance and thus past results often do not reflect the performances of the…