Related papers: Interpretable hypothesis tests
This paper raises concerns about the advantages of using statistical significance tests in research assessments as has recently been suggested in the debate about proper normalization procedures for citation indicators. Statistical…
Current machine learning models are evaluated through behavioral snapshots, with benchmark accuracies, win rates and outcome-based metrics. Model explanations and evaluations, however, are fundamentally intertwined: understanding why a…
Statistical hypothesis testing, as formalized by 20th Century statisticians and taught in college statistics courses, has been a cornerstone of 100 years of scientific progress. Nevertheless, the methodology is increasingly questioned in…
A fundamental assumption of classical hypothesis testing is that the significance threshold $\alpha$ is chosen independently from the data. The validity of confidence intervals likewise relies on choosing $\alpha$ beforehand. We point out…
In this introductory note, I describe my particular view of the notion of ontological commitments as honest and pragmatic working hypotheses that assume the existence (out there) of certain entities represented by the symbols in our theory.…
This paper explores conditions of existence of different types of consistent tests. New links of these types of consistency are also established. The existence of discernible (strong consistent) tests follows from the existence of pointwise…
The best mathematical arguments against a realistic interpretation of quantum mechanics - that gives definite but partially unknown values to all observables - are analysed and shown to be based on reasoning that is not compelling. This…
There are many ways we can not know. Even in systems that we created ourselves, as, for example, systems in mathematical logic, Go\"edel and Tarski's theorems impose limits on what we can know. As we try to speak of the real world, things…
As machine learning and algorithmic decision making systems are increasingly being leveraged in high-stakes human-in-the-loop settings, there is a pressing need to understand the rationale of their predictions. Researchers have responded to…
Supervised machine learning models boast remarkable predictive capabilities. But can you trust your model? Will it work in deployment? What else can it tell you about the world? We want models to be not only good, but interpretable. And yet…
In Bayesian statistics the precise point-null hypothesis $\theta=\theta_0$ can be tested by checking whether $\theta_0$ is contained in a credible set. This permits testing of $\theta=\theta_0$ without having to put prior probabilities on…
The choice of priors may become an insoluble problem if priors and Bayes' rule are not seen and accepted in the framework of subjectivism. Therefore, the meaning and the role of subjectivity in science is considered and defended from the…
Every scientific endeavour consists of (at least) two components: A hypothesis on the one hand and data on the other. There is always a more or less abstract level - some theory, a set of concepts, certain relations of ideas - and a…
When statisticians quarrel about hypothesis testing, the debate usually focus on which method is the correct one. The fundamental question of whether we should test hypothesis at all tends to be forgotten. This lack of debate has its roots…
The intrinsic difficulties in building realistic climate models and in providing complete, reliable and meaningful observational datasets, and the conceptual impossibility of testing theories against data imply that the usual Galilean…
It is difficult to neutrally evaluate the risks posed by large-scale leading-edge science experiments. Traditional risk assessment is problematic in this context for multiple reasons. Also, such experiments can be insulated from challenge…
We consider the problem of hypotheses testing with the basic simple hypothesis: observed sequence of points corresponds to stationary Poisson process with known intensity. The alternatives are stationary self-exciting point processes. We…
One can argue that one of the main roles of the subject of statistics is to characterize what the evidence in collected data says about questions of scientific interest. There are two broad questions that we will refer to as the estimation…
The process of doing Science in condition of uncertainty is illustrated with a toy experiment in which the inferential and the forecasting aspects are both present. The fundamental aspects of probabilistic reasoning, also relevant in real…
We consider the problems of hypothesis testing on a probability measure of independent sample, on solution of ill-posed problem, on deconvolution problem and on Poisson mean measure. For all these setups necessary conditions and sufficient…