Comment by zingar
13 hours ago
I’d love to hear from you or someone in your shoes: what are some patterns or examples of tests that are made redundant by types?
“It has a field of type X” has never been a useful test for me, my tests are always more like:
“if I send message X I get return value or action Y”
… with my admittedly limited experience of types I don’t see how they replicate this.
Therefore it looks like I’d only be “replacing” tests that I’d never write in the first place.
What am I missing?
We have some tests that ensure the interface is correct - that the correct type of args are passed say from a batch process to a mailer and a mail object is returned.
For these tests we don’t care about the content only that something didn’t get incorrectly set or the mailer interface changed.
Now if the developer changes the Mailer to require a user object the compiler tells us there is an error. Sorbet will error and say “hey you need to update your code here and here by adding a User object”
Before we would have had test coverage for that - or maybe not and missed the error.
One of the big advantages of types is documenting what is *not* allowed. This brings a clarity to the developers and additionally ensure what is not allowed does not happen.
Unit tests typically test for behaviours. This could be both positive and negative tests. But we often test only a subset of possibilities just because how people generally think (more positive cases than negative cases). Theoretically we can do all those tests with unit testing. But we need to ask ourselves honestly, do we have that kind of test coverage as SQLLite? If yes, do we have that for very large codebases?
Just to clarify, are you saying SQLLite is a good example that we should emulate?
SQLite is known for having a lot of testing. Per their docs around 600x as much test as application code.
https://sqlite.org/testing.html
First one that pops to mind is some old python code; the parameter that came in on some functions could be a single string or a list of them. Lots of bugs where arg[0] was a character rather than a string. So tests had to be written showing both being passed in.