Comment by chriswarbo

Hypothesis is a search algorithm for finding ways that we have misunderstood our code (whether that be in the codebase, or the claims we have made about it in the spec/tests). Conceptually, that's the inverse of "a unit test that sometimes fails": it's a search that sometimes succeeds at finding out that we're wrong. That's infinity% more effective than a search procedure which is hard-coded to check a single example; especially when that example is chosen by the same dev (with the same misconceptions!) that implemented the code-under-test!

Now, as for what would actually happen in that situation, when using Hypothesis:

- As others have indicated, Hypothesis keeps a database of the failures it's found. Committing that database to your project repo would act like a ratchet: once they've successfully found a problem, they will continue to do so (until it's fixed).

- If you don't want to commit your database (to avoid churn/conflicts), then the same would happen per working-copy. This is fine for dev machines, where repo directories can live for a long time; though not so useful for CI, if the working-copy gets nuked after each run.

- Even if we disable the failure database, property failures will always show the inputs which caused it (the "counterexample"), along with the random seed used to generate it. Either of those is enough to reproduce that exact run, giving us a ready-made regression test we can copy/paste to make the failure permanent (until it's fixed). As others have said, Hypothesis properties can be decorated with `@example(foo)` to ensure a particular input is always checked.

- Even if we disable the failure database, and don't copy the counterexample as a regression test, property checkers like Hypothesis will still "shrink" any counterexamples they find. Your example of apostrophes causing problems is trivial for Hypothesis to shrink, so if/when it happens to find some counterexample, it will always manage to shrink that down to the string "'"; essentially telling us that "employee names fail on apostrophe", which we could either fix right now, or stick in our bug tracker, or whatever.