Comment by equark

This is the most important critique of A/B testing. It far outweighs the traditional hoopla about simultaneous inference and Bonferonni corrections.

Epsilon greedy does well on k-armed bandit problems, but in most applications you likely can do significantly better by customizing the strategy to individual users. That's a contextual bandit and there are simple strategies that to pretty well here too. For instance:

http://hunch.net/?p=298

http://hunch.net/~exploration_learning/main.pdf

http://web.mit.edu/hauser/www/Papers/Hauser_Urban_Liberali_B...