Generalized Sequential Probability Ratio Test for Families of Hypotheses [pdf]

4 days ago (sites.stat.columbia.edu)

Great. How do I use this in my life to make things better?

  • The SPRT is probably already making your life better: it's used to decrease the cost of medical trails, optimize classifications in high-stakes examinations (i.e. for medical certifications), detect defective manufacturing processes, etc. It sounds like this paper extends the method to groups of hypotheses, whereas the basic version is limited to a null hypothesis and an alternative hypothesis.

  • You can search for the "peeking" problem in A/B testing.

    SPRT also very likely helped win a major war that involved many nations.

  • This helps with determining when have you observed enough data to make a decision.

    A/B tests, monitoring metrics, health, quality control all use this.

    If you use LLMs, you might use this to determine if a model update or prompt change impacts results using fewer tokens.

  • Implement a statistical software suite that ubiquitously uses this framework instead of the usual hierarchical mixed modeling tools whose assumptions often don't match what experiments were actually done.