Comment by mrkeen
3 days ago
> Your logs are lying to you. Not maliciously. They're just not equipped to tell the truth.
The best way to equip logs to tell the truth is to have other parts of the system consume them as their source of truth.
Firstly: "what the system does" and "what the logs say" can't be two different things.
Secondly: developers can't put less info into the logs than they should, because their feature simply won't work without it.
That doesn't sound like a good plan. You're coupling logging with business logic. I don't want to have to think if i change a debug string am i going to break something.
You're also assuming your log infrastructure is a lot more durable than most are. Generally, logging is not a guaranteed action. Writing a log message is not normally something where you wait for a disk sync before proceeding. Dropping a log message here or there is not a fatal error. Logs get rotated and deleted automatically. They are designed for retroactive use and best effort event recording, not assumed to be a flawless record of everything the system did.
> You're also assuming your log infrastructure is a lot more durable than most are.
Make actions, not assumptions. Instead of using a one machine storage system, distribute that storage across many machines. Then stop deleting them.
> Dropping a log message here or there is not a fatal error.
I would try to reallocate my effort budget to things that actually need to work.
Drop logging completely, and come back to it once you have a flawless record of everything the system did. The reconsider whether you need it.
> You're coupling logging with business logic
Yes, the system shall not report that "User null was created" if it was actually "User 123 that was created".
String? Not a chance, make a proper type-safe struct. UserCreated { "id": 123}
> I don't want to have to think if i change a debug string am i going to break something.
Good point, you should probably have a unit test somewhere.
Your logic wouldn't be dependent on a debug string, but some enum in a structured field. Ex, event_type: CREATED_TRANSACTION.
Seeing logging as debugging is flawed imo. A log is technically just a record of what happened in your database.