Comment by Jare

It will adjust itself in the sense that while it is still enabled and testing (a suboptimal state for your product), it will let your product perform better than plain old A/B.

At some point you still step in and decide based on the data, especially if you detect degenerate cases, and move on to the next experiment.