Related papers: Significance Tests in Climate Science
Determining and measuring cause-effect relationships is fundamental to most scientific studies of natural phenomena. The notion of causation is distinctly different from correlation which only looks at association of trends or patterns in…
Statistical significance testing is widely accepted as a means to assess how well a difference in effectiveness reflects an actual difference between systems, as opposed to random noise because of the selection of topics. According to…
I present a critique of the methods used in a typical paper. This leads to three broad conclusions about the conventional use of statistical methods. First, results are often reported in an unnecessarily obscure manner. Second, the null…
Experimental research on behavior and cognition frequently rests on stimulus or subject selection where not all characteristics can be fully controlled, even when attempting strict matching. For example, when contrasting patients to…
Although hypothesis tests play a prominent role in Science, their interpretation can be challenging. Three issues are (i) the difficulty in making an assertive decision based on the output of an hypothesis test, (ii) the logical…
This special volume of Statistical Sciences presents some innovative, if not provocative, ideas in the area of reliability, or perhaps more appropriately named, integrated system assessment. In this age of exponential growth in science,…
The sample median is often used in statistical analyses of physical or astronomical data wherein a central value must be found from samples polluted by elements which do not belong to the population of interest or when the underlying…
In most circumstances, probability sampling is the only way to ensure unbiased inference about population quantities where a complete census is not possible. As we enter the era of 'big data', however, nonprobability samples, whose sampling…
Despite their importance in supporting experimental conclusions, standard statistical tests are often inadequate for research areas, like the life sciences, where the typical sample size is small and the test assumptions difficult to…
In their recent comment, published in Nature, Jeffrey T.Leek and Roger D.Peng discuss how P-values are widely abused in null hypothesis significance testing . We agree completely with them and in this short comment we discuss the importance…
Significance testing aims to determine whether a proposition about the population distribution is the truth or not given observations. However, traditional significance testing often needs to derive the distribution of the testing…
Impact of science is one of the most important topics in scientometrics. Recent developments show a fundamental change in impact measurements from impact on science to impact on society. Since impact measurement is currently in a state of…
The value of the likelihood is occasionally used by high energy physicists as a statistic to measure goodness-of-fit in unbinned maximum likelihood fits. Simple examples are presented that illustrate why this (seemingly intuitive) method…
Statistical analysis is often used to evaluate the evidence for or against scientific hypotheses, and various statistics (e.g., p-values, likelihood ratios, Bayes factors) are interpreted as measures of evidence strength. Here I consider…
It is quite common in modern research, for a researcher to test many hypotheses. The statistical (frequentist) hypothesis testing framework, does not scale with the number of hypotheses in the sense that naively performing many hypothesis…
This essay, written by a statistician and a climate scientist, describes our view of the gap that exists between current practice in mainstream climate science, and the practical needs of policymakers charged with exploring possible…
The development of scientometric indicators and methods for evaluative purposes, requires a multitude of assumptions, conventions, limitations, and caveats. Given this, we cannot permit ambiguities in the key concepts forming the basis of…
Many measurements at collider experiments study physics candidates that are a subset of a collision event. The presence of multiple such candidates in a given event can cause raw biases which are large compared to typical statistical…
In narrative synthesis of evidence, it can be the case that the only quantitative measures available concerning the efficacy of an intervention is the direction of the effect, i.e. whether it is positive or negative. In such situations, the…
We define some appropriate statistical quantities that indicate the physical significance (reliability) of confidence intervals in the framework of both Frequentist and Bayesian statistical theories. We consider the expectation value of the…