Survival-inferred fragility of statistical significance in phase III oncology trials
Ethan Bernard Ludmir, Yufei Liu, David Bomze, Christine Lin, Avital M. Miller, Tomer Meirson, Timothy A. Lin, Jonathan Ofer, Molly Blue El Alam, Pavlos Msaouel, Zachary Ryan McCawa, Ramez Kouzy, Alex Koong, Joseph Abi Jaoude, Mohannad Owiwi, Roshal R. Patel, Alexander Dean Sherry
- aUniversity of North Carolina at Chapel Hill
Papers open inside SciRah from the original publisher or repository; SciRah never hosts them.
Abstract
In phase III oncology trials, superiority is defined by statistical significance using P thresholds. However, this approach has been criticized because P is continuous. Here, we reconstruct patient-level data for 230 phase III oncology trials to model the robustness of statistical significance by estimating the survival-inferred fragility index (SIFI), defined as the smallest number of patients changing arms that alters the statistical significance interpretation. The median SIFI was 8 patients (IQR 4–19), representing 1.4% of enrollments (IQR 0.7%–3%). As a continuous statistic, P—but not the significance interpretation—was correlated with SIFI. Moreover, overall survival endpoints were more fragile than surrogate endpoints. Taken together, while phase III oncology trials are intended to robustly inform patient care, shifting the assignment of a few patients is often sufficient to upend the statistical significance interpretation. This vulnerability underscores the need for more robust strategies to identify superiority in oncology.