Comment by ehnto
5 days ago
My two insights are:
Legacy code is difficult to work with because the reasons "why" something has to be some way are missing. The "whys" can change over time, be interwoven with business requirements not in the codebase, and compound/nest with other whys that have come and gone. Often only humans in the business know, and you have to figure out who and what to ask.
Second insight is that LLMs have so far been very bad at holding on to the thread of winding, changing requirements, which is something it would have to be good at in order to revisit old codebases. Even one it wrote itself. It would need some kind of historical changes to business requirements database, which I have never seen a business keep, it'sjust the collective human knowledge of the business. Otherwise it'll make changes without accounting for historic requirements as well as your new ones.
I think there are probably some workarounds, like getting it to write lengthy documentation to refer to. Long term perhaps it just gets smart enough. Legacy codebases are something I specialise in working with somewhat, I spent a long time consulting so I was often parachuting into unfamiliar codebases and trying to quickly understand years to a decade or more, of evolving requirements in a business/codebase. I have a lot more to say on the topic but those are two of my insights.
Good point. Whys are lost because whys are often never written down. World around us always changes, so the whys change too.
This may have something to so with the AI "frame-problem".We can't document all "whys" in the current "frame" because they depend on other "frames" like the current legislation. We don't write down "this will work because ..." . We just assume it, and asume many other causally related things.
In other wrods it would very hard or impossible to document "what must not change" for our system to continue to be useful. For instance Sun must still exist and air must be breathable for this system to be useful.