This site uses cookies

By using this site, you consent to our use of cookies. You can view our terms and conditions for more information.

Sequential ANOVA: An Efficient Alternative to Fixed Sample Designs

Meike Steinhilber
University of Mainz ~ Department of Analysis and Modeling of Complex Data
Anna-Lena Schubert
University of Mainz
Dr. Martin Schnuerch
University of Mannheim ~ Statistical Modeling in Psychology

The replication crisis has shown that researchers often collect insufficient sample sizes for their studies or use questionable research practices such as data peeking. Sequential testing procedures are one possible solution to these shortcomings. The Sequential Probability Ratio Tests (SPRTs) offer interesting possibilities within Frequentist Statistics. Here, a likelihood ratio is calculated as a test statistic, which can be computed continuously by adding new data points. After each iterative step of data collection, SPRTs decide whether the evidence is sufficient to accept the null or alternative hypothesis or whether more data points are needed. SPRTs control for alpha and beta error rates and require, as a test specification, a minimum expected effect size. We show in simulation studies that the one-way sequential ANOVA is very efficient compared to a classical fixed ANOVA. In 87% of the simulated cases, the sequential samples are smaller than the fixed samples. On average, 56% of the data can be saved using the sequential design. However, sequential designs show biases in effect size estimation. Thus, we want to discuss the benefits and limitations of sequential testing. The R package sprtt can be used to calculate sequential versions of t-tests and one-way ANOVAs.



sequential testing
sequential probability ratio tests
replication crisis

There is nothing here yet. Be the first to create a thread.

Cite this as:

Steinhilber, M., Schubert, A.-L., & Schnuerch, M. (2023, July). Sequential ANOVA: An Efficient Alternative to Fixed Sample Designs. Abstract published at MathPsych/ICCM/EMPG 2023. Via