Comment by IanCal
20 hours ago
Hypothesis keeps a database of failures to use locally and you can add a decorator to mark a specific case that failed. So you run it, see the failure, add it as a specific case and then that’s committed to the codebase.
The randomness can bite a little if that test failure happens on an unrelated branch, but it’s not much different to someone just discovering a bug.
edit - here's the relevant part of the hypothesis guide https://hypothesis.readthedocs.io/en/latest/tutorial/replayi...
You can also cache the DB across CI runs, which will reduce the randomness (ie failures won’t just disappear between runs).