Comment by hinkley

It is a universal truth that people fuck up statistical math.

    There are three kinds of lies: lies, damn lies, and statistics

If you aren’t testing at exactly 50/50 - and you can’t because my plan for visiting a site and for how long will never be equivalent to your plan, then any other factors that can affect conversion rate will cause one partition to go up faster than the other. You have to test at a level of Amazon to get statistical significance anyway.

And as many if us have told people until they’re blue in the face: we (you) are not a FAANG company and pretending to be one won’t work.