Comment by pmontra
8 hours ago
> In Elixir tests, each test runs in a database transaction that rolls back at the end. Tests run async without hitting each other. No test data persists.
And it confuses Claude.
This way of running tests is also what Rails does, and AFAIK Django too. Tests are isolated and can be run in random order. Actually, Rails randomizes the order so if the are tests that for any reason depend on the order of execution, they will eventually fail. To help debug those cases, it prints the seed and it can be used to rerun those tests deterministically, including the calls to methods returning random values.
I thought that this is how all test frameworks work in 2026.
Some database tests can't be done within transactions. With postgres, I create a copy of the database via WITH TEMPLATE, and each test runs in its own copy of the database. Then it can use or avoid transactions as it pleases, because the whole thing is local to that one tests anyway.
I did too, and I've had a challenging time convincing people outside of those ecosystems that this is possible, reasonable, we've been doing it for over a decade.
Story of my life in so many dimensions.
[flagged]
> And it confuses Claude.
I've never had this problem.
Why not just write to the db? Just make every test independent, use uuids / random ids for ids.
> Just make every test independent
That's easier said than done. Simple example: API that returns a count of all users in the database. The obvious correct implementation that will work would be just to `select count(*) from users`. But if some other test touches users table beforehand, it won't work. There is no uuid to latch onto here.
That’s why you run each test in a transaction with proper isolation level, and don’t commit the transaction— roll it back when the test ends. No test ever interferes with another that way.
3 replies →
Frankly this is the better solution for async tests. If the app can handle multiple users interacting with it simultaneously, then it can handle multiple tests. If it can’t, then the dev has bigger problems.
As for assertions, it’s not that hard to think of a better way to check if you made an insertion or not into the db without writing “assert user_count() == 0”
I don’t disagree with you, but there are diminishing returns on making your test suite complex. To make async test work properly, you need to know what you’re doing in regards to message passing, OTP, mocks, shared memory, blah blah blah. It can get really complicated, and it is still isn’t a substitute for real user traffic. You’re going to have to rely on hiring experienced Elixir developers (small talent pool), allow for long onboarding time (expensive), or provide extensive training (difficult). Personally for most cases, writing a sync test suite and just optimizing to keep it not to slow is probably more practical in the long term.
[dead]