Why bother with Bayesian t-tests?
Abstract
Given the well-known and fundamental problems with hypothesis testing via classical (point-form) significance tests, there has been a general move to alternative approaches, often focused on the Bayesian t-test. We show that the Bayesian t-test approach does not address the observed problems with classical significance testing, that Bayesian and classical t-tests are mathematically equivalent and linearly related in order of magnitude (so that the Bayesian t-test providing no further information beyond that given by point-form significance tests), and that Bayesian t-tests are subject to serious risks of misinterpretation, in some cases more problematic than seen for classical tests (with, for example, a negative sample mean in an experiment giving strong Bayesian t-test evidence in favour of a positive population mean). We do not suggest a return to the classical, point-form significance approach to hypothesis testing. Instead we argue for an alternative distributional approach to significance testing, which addresses the observed problems with classical hypothesis testing and provides a natural link between the Bayesian and frequentist approaches.
Keywords
Cite
@article{arxiv.2211.02613,
title = {Why bother with Bayesian t-tests?},
author = {Fintan Costello and Paul Watts},
journal= {arXiv preprint arXiv:2211.02613},
year = {2022}
}