A comparative study of two-sample hypothesis tests in the presence of long-term survivors

Time-to-event data with long-term survivors (L-TS), subjects who never experience the event, have been reported in multiple areas of oncology as therapies have improved. Conventional two-sample tests ignore L-TS, but alternatives have been developed in the cure models literature. Because L-TS can induce non-proportional hazards (non-PH), non-PH candidates also exist. However, there has not been a comprehensive comparison of these candidates. Additionally, follow-up is an important consideration for data with L-TS, but there has been limited study of the impact of follow-up time on performance of two-sample tests with L-TS. We conducted a neutral simulation study of the impact of sample size and follow-up time on type I error and power across varying effect sizes for conventional methods, methods adapted for non-PH, and a correctly-specified parametric model. When one or both groups lack L-TS, log-rank tests and one non-PH method typically have the highest power, but order varies. Surprisingly, when both groups have L-TS, these tests have non-monotonic power as a function of follow-up time, while parametric models have monotonic increasing power and the highest power at the longest follow-up time. While absolute power differs, patterns over follow-up are consistent across sample sizes. To address this for practitioners, we devise a numerical approach to predict the potential for non-monotonicity during study planning. We conclude that na\"ive use of conventional methods can have counterintuitive properties in settings with L-TS, and this work provides knowledge and a tool to anticipate and address these issues.