Comment by ryan-duve

> A lot of sites don't have enough traffic to get statistical significance with this in a reasonable amount of time

What's nice about AB testing is the decision can be made on point estimates, provided the two choices don't have different operational "costs". You don't need to know that A is better than B, you just need to pick one and the point estimate gives the best answer with the available data.

I don't know of a way to determine whether A is better than B with statistical significance without letting the experiment run, in practice, for way too long.