Comment by wesselbindt

You are not wrong but if we go deeper we will find more nuances.

> tests should test behaviour and not structure.

Of what?

The whole idea of software architecture, and underlying of my messages, is to structure the code so that it is easy, but before easy, possible, to work at the right abstraction level.

Which allows to tests the behaviour of components and not their structure.

Trivial example, say your code read from a socket and manage the bytes with some CPU operations and write them to another socket. (You may recognise it is basically what a compressor do)

The way in which you manage read and write to the sockets will make dramatically simpler or much more difficult to write good tests.

If you adopt strategies like sans-io, you will see that the testing is almost trivial.

If all the logic sprawl up from the read syscall in a loop, you will notice how more challenging testing becomes.

---

To answer your question, the way I let LLMs write code is very DI (dependency injection) based.

A class never instantiates another class - all the dependencies are passed to the constructor. Including time. Including whatever DB iteraction.

The reason why I prefer this is that I can test each component at every level of abstraction. I don't have to. But I can.

My current approach is to force coverage higher than say 80% as default and then when a bug is discovered drill down to the components.

The question "for which functions/modules/classes should I write tests?" and the question "how do I structure my functions/modules/classes?" are very closely related questions, which I think is what the other comment was getting at.

That follows largely from what you're saying though, so I think the two of you agree, you're just coming at the same thing from different perspectives. Like you say, you want to avoid/manage coupling tests to implementation details, and typically the best way to do that is to bundle a significant, testable chunk of code into a single module, define a clear public interface to that module, and then test that interface. The internals of that module are then free to change, but the test interface should stay the same.

But the corollary of that is that you can't just design your modules independently of your tests, you need to design them so that they are testable. Which is why the previous comment links testing to code architecture/design.

Regarding dependencies like that, the best situation is where you can either:

(a) design a module so that the dependency is injected with a clear interface (not just "here's an instance of libredis.rs" but "here's a series of callbacks for saving data, those callbacks might save to Redis, but they might be in-memory only, or they might save everything to the blockchain, either way it doesn't matter").

(b) just spin up an instance of Redis and test directly against that — it should be quick enough, and there's plenty of ways to ensure that the tests don't affect each other even while accessing a shared resource.

> I'm not sure I fully understand. In my view, it's pretty evident that tests should test behavior and not structure.

Not OP, but I think I know what OP has in mind. There are some trivial ways in which code can be made easier to test as well as more complex (let's call them architectural) ways.

Examples of trivial conveniences include: making tested functionality accessible from the test (i.e. private class methods in Java present a problem for testing), coding against an interface rather than an implementation (this allows substituting a test implementation for a production implementation that can fish out some bugs), having internal to the code correctness checks that don't necessarily influence program execution (simply having runtime type checks, like it's done in eg. Java, compared to eg. C makes testing easier and more productive). Of course, there's more, these are just very common and easy to understand.

The architectural properties that improve the ability to test a program might be: the so-called "observability" programmed into the product, i.e. the product generates metrics data that's not part of the desired output. The product is split into modules with formally defined interfaces, which allows testing modules in isolation and lowers the number of possible combinations to test.

Improving testability isn't necessarily a good thing because it's likely to negatively impact complexity, size or speed of the program. So, depending on what's more important for any given program, the testing strategy will be different...