Comment by Herring
13 days ago
Yeah and he claims a pass rate of 99.96%. At that point you might be running into bugs in the original implementation.
13 days ago
Yeah and he claims a pass rate of 99.96%. At that point you might be running into bugs in the original implementation.
Not really. Due to combinatorial explosion some path is hard to hit randomly in this kind of source code. I would have preferred if after 2M random battles the reference implementation had 99% code coverage, than 99% pass rate.
I don't know anything about Pokemon, but I briefly looked at the code. "weather" seemed like a self contained thing I could potentially understand. Looking at https://github.com/vjeux/pokemon-showdown-rs/blob/master/src...
> NOTE: ignoringAbility() and abilityState.ending not fully implemented
So it is almost certain even after 99.96% pass rate, it didn't hit battle with weather suppressing Pokemon but with ability ignored. Code coverage driven testing loop would have found and fixed this one easily.
Good catch. I should really look at the code before commenting on it.