Comment by 9rx
8 days ago
> Tests are a description
As is a spec. "Description" is literally found in the dictionary definition. Which stands to reason as tests are merely a way to write a spec. They are the same thing.
> The thing is it is very very difficult to know if your tests have complete coverage.
There is no way to avoid that, though. Like you point out, not even formal proofs, the closest speccing methodology we know of to try and avoid this, is immune.
> Tests are an approximation of your spec.
Specs are an approximation of what you actually want, sure, but that does not change that tests are the spec. There are other ways to write a spec, of course, but if you went down that road you wouldn't also have tests. That would be not only pointless, but a nightmare due to not having a single source of truth which causes all kinds of social (and sometimes technical) problems.
I disagree. It's, like you say, one description of your spec but that's not the spec.
Well that's the thing, there is no single source of truth. A single source of truth is for religion, not code.
The point of saying this is to ensure you don't fall prey to fooling yourself. You're the easiest person for you to fool, after all. You should always carry some doubt. Not so much it is debilitating, but enough to keep you from being too arrogant. You need to constantly check that your documentation is aligned to your specs and that your specs are aligned to your goals. If you cannot see how these are different things then it's impossible to check your alignment and you've fooled yourself.
> You need to constantly check that your documentation is aligned to your specs
Documentation, tests, and specs are all ultimately different words for the same thing.
You do have to check that your implementation and documentation/spec/tests are aligned, which can be a lot of work if you do so by hand, but that's why we invented automatic methods. Formal verification is theoretically best (that we know of) at this, but a huge pain in the ass for humans to write, so that is why virtually everyone has adopted tests instead. It is a reasonable tradeoff between comfort in writing documentation while still providing sufficient automatic guarantees that the documentation is true.
> If you cannot see how these are different things
If you see them as different things, you are either pointlessly repeating yourself over and over or inventing information that is, at best, worthless (but often actively harmful).
You're still misunderstanding and missing the layer of abstraction, which is what I'm (and others are) talking about
We have 3 objects: doc, test, spec. How do you prove they are the same thing?
You are arguing that they all point to the same address.
I'm arguing they all have the same parent.
I think it's pretty trivial to show that they aren't identical, so I'll give two examples (I'm sure you can figure out a few more trivial ones):
Yes, you should simplify things as much as possible but be careful to not simplify further
3 replies →