Related papers: Type I Error Rates are Not Usually Inflated
Replication is complicated in psychological research because studies of a given psychological phenomenon can never be direct or exact replications of one another, and thus effect sizes vary from one study of the phenomenon to the next--an…
Despite its central role in modern cosmology, doubts are often expressed as to whether cosmological inflation is really a falsifiable theory. We distinguish two facets of inflation, one as a theory of initial conditions for the hot big bang…
We theoretically analyze the problem of testing for $p$-hacking based on distributions of $p$-values across multiple studies. We provide general results for when such distributions have testable restrictions (are non-increasing) under the…
We investigate which of the exotic Doppler peak features found for textures and cosmic strings are generic novelties pertaining to defects. We find that the ``out of phase'' texture signature is an accident. Generic defects, when they…
The inflated beta regression model aims to enable the modeling of responses in the intervals $(0,1]$, $[0,1)$ or $[0,1]$. In this model, hypothesis testing is often performed based on the likelihood ratio statistic. The critical values are…
Clustering is part of unsupervised analysis methods that consist in grouping samples into homogeneous and separate subgroups of observations also called clusters. To interpret the clusters, statistical hypothesis testing is often used to…
The cell biology literature is littered with erroneously tiny P values, often the result of evaluating individual cells as independent samples. Because readers use P values and error bars to infer whether a reported difference would likely…
During lab studies of text entry methods it is typical to observer very few errors in participants' typing - users tend to type very carefully in labs. This is a problem when investigating methods to support error awareness or correction as…
Counterfactual explanations are usually obtained by identifying the smallest change made to an input to change a prediction made by a fixed model (hereafter called sparse methods). Recent work, however, has revitalized an old insight: there…
The problem of causal inference is to determine if a given probability distribution on observed variables is compatible with some causal structure. The difficult case is when the causal structure includes latent variables. We here introduce…
A pervasive issue in statistical hypothesis testing is that the reported $p$-values are biased downward by data "peeking" -- the practice of reporting only progressively extreme values of the test statistic as more data samples are…
Recent advances in generative models facilitate the creation of synthetic data to be made available for research in privacy-sensitive contexts. However, the analysis of synthetic data raises a unique set of methodological challenges. In…
RCTs sometimes test interventions that aim to improve existing services targeted to a subset of individuals identified after randomization. Accordingly, the treatment could affect the composition of service recipients and the offered…
We review the recent progress regarding the loop corrections to the correlation functions in the inflationary universe. A naive perturbation theory predicts that loop corrections generated during inflation suffer from various infrared (IR)…
In empirical work it is common to estimate parameters of models and report associated standard errors that account for "clustering" of units, where clusters are defined by factors such as geography. Clustering adjustments are typically…
Inverse normal transformations applied to the partially overlapping samples t-tests by Derrick et.al. (2017) are considered for their Type I error robustness and power. The inverse normal transformation solutions proposed in this paper are…
Cluster analysis is a fundamental research issue in statistics and machine learning. In many modern clustering methods, we need to determine whether two subsets of samples come from the same cluster. Since these subsets are usually…
Research on reasoning in language models (LMs) predominantly focuses on improving the correctness of their outputs. But some important applications require modeling reasoning patterns that are incorrect. For example, automated systems that…
When assessing the presence of an exposure causal effect on a given outcome, it is well known that classical measurement error of the exposure can reduce the power of a test of the null hypothesis in question, although its type I error rate…
In a recent simulation study, Goodman et al. (2019) compare several methods with regard to their type I and type II error rates in case of a thick null hypothesis that includes all values that are practically equivalent to the point null…