Comment by micheljansen

Although I can imagine this works very well for ecommerce websites and other things where there is a very obvious single measure of success, like for example:

  * the user clicked the button
  * the user signed up
  * the user put something in their cart
  * the user paid X

In this case, it's easy to create the feedback loop that is required for this testing method.

However, in the real world, things are not always that simple. What if you want to optimise:

  * the percentage of users that returns to the site 
  * the time that users spend on your site
  * the number of pages that they view in a session

I'm sure some of these metrics can also be plugged back into the bandit algorithm, but it's a lot more complicated.