Comment by ants_everywhere
1 day ago
There's also a problem c)
c) Work tends to proceed in drafts/iterations, with earlier initial ideas refined and processed over time. You can record the reasons behind the initial ideas, but eventually those ideas become implicit background and new ideas are diffs against them. It's hard to make all of that explicit and keep it up to date.
You'd think something like wikis would help here because you can see the history. But eventually as you move around in the design space you can make moves that seem obvious but cut across multiple dimensions simultaneously. And it's hard to note something like that well in a wiki commit.
A long form lab notebook (or doc) is often useful here.
That is a good point. The more documentation there is, the harder it is to keep up to date. It also perhaps works against getting the best design, because nobody want to go back and change all the documentation. An argument for keeping the documentation fairly minimal, perhaps.
I like making good docs, and i appreciate good docs when i find it. I think alot can be done to make docs easier to maintain and more useful.
When approaching a large cryptic codebase, the intention and assumptions are usually more important than the details. And these are really quite simple to write down.
- Goals a Problems
- Constraints
- Chosen high level approach
- Relevant files, functions and resources.
- Future concerns
There is the worry about maintenence, but i think the history itself may be valuable.
If a design change happens, or a new major thing is added on-top, make a new document and timestamp it.
Over time this becomes a repository and history of the design and current state of the system.
Its harder to read and understand than a properly updated doc that is reflective of the current state, but its orders of magnitude better than not making docs because its annoying.
This problem can be solved by making documentation a natural side effect of the actual work.
One way is by recording video calls where the decisions are made with transcriptions which can be interpreted by LLMs.
Another is generating docs from tests.
Yet another is making a rule that answers to code review questions must be turned into code comments.
There's also a huge amount of untapped potential for turning conversations in slack into documentation.
The issue with wikis, etc. is yea, that it isnt a side effect of the work itself. It is work, and thankless work at that.
All good points. It is perhaps a bit easier with software, where you can put comments and hyperlink directly in the code. It is a bit harder with a big, long-lived physical artefacts, such as a nuclear reactors.
The broader point was to make documentation a side effect of the work rather than a separate task. The opportunities to do this are not obvious but they are there.
I imagine this principle can probably be applied to nuclear power plants too but I dont really know how they work or what software they use so it's hard to come up with examples.