English

Precision and Decisiveness as Goals: Reliable Sequential Hypothesis Testing with a Dual Stopping Criterion

Methodology 2026-08-05 v1

Abstract

Sequential hypothesis testing offers flexibility over fixed-sample designs, but stopping rules coupled to decision criteria risk confirmation bias through early peeking. The HDI+ROPE algorithm exemplifies this: it stops as soon as the posterior Highest Density Interval (HDI) falls entirely outside a Region of Practical Equivalence (ROPE) around the null (rejecting it) or entirely inside, which enables positive acceptance of the null, something Null Hypothesis Significance Testing cannot do. However, this coupling means a wide HDI can satisfy the criterion on early, unrepresentative samples, incurring a systematic false-positive cost. Fully decoupled methods, such as ``Precision is the Goal'' (PitG), eliminate this bias by halting only once a target HDI width ω\omega is reached; yet, by divorcing the stopping rule from the decision criterion, PitG frequently yields inconclusive outcomes, particularly when the null hypothesis is true. We propose ``Decisive Precision is the Goal'' (DPitG), which requires both the precision target and a conclusive verdict to be satisfied simultaneously. In fair coin simulations (ω=0.08\omega=0.08, ROPE=0.5±0.05=0.5\pm0.05), DPitG reduces the PitG inconclusive rate from 62% to 2% at a median cost of only 5% more samples, with zero false positives; HDI+ROPE achieves comparable conclusiveness only at the cost of false positives. Across the ω\omega range tested, DPitG conclusiveness remains above 97% while PitG's varies widely. We provide a closed-form planning formula NV/ω2N\propto V/\omega^2 (where VV is the observation variance), an online interactive calculator, and open-source code. Demonstrated on single-group binary data, the framework extends readily to continuous outcomes and two-group comparisons. DPitG's fully pre-specified stopping rule is compatible with pre-registration standards and is the method of choice whenever a reliable verdict is required.

Keywords

Cite

@article{arxiv.2608.05301,
  title  = {Precision and Decisiveness as Goals: Reliable Sequential Hypothesis Testing with a Dual Stopping Criterion},
  author = {Eyal A. Kazin},
  journal= {arXiv preprint arXiv:2608.05301},
  year   = {2026}
}

Comments

Main section 17 pages and 9 figures, Seven Abstracts in 7 pages and 1 figure