Comment by yorwba

As long as variance is nonzero, you won't get the exact same result twice for the same benchmark, so one of the numbers has to be higher and the other lower. If watermarking has no effect on the distribution (by construction, it should have no effect), that's a 50% chance the watermarked model gets the higher number. In this case, it happened 5 out of 8 times, which is hardly unusual.