Comment by jacobedawson
7 days ago
I feel as though that world model strongly correlates with memory - the experience of having jumped to a conclusion early and full-steaming ahead, only to be bitten by constraints and problems later down the track.
Part of that is critical thinking and projecting forward / simulating potential issues, and part of that is that memory which in humans we probably would see as "wisdom".
I don't know if that's a fundamental limitation of LLMs, or, rather, that this can be solved moving forward with better memory systems, harnesses, and context windows.
Yeah, I think it's more so learning from experiences that didn't scale. E.g. I started out with Notepad, and wrote everything for the website I wanted to build in a single large massive php file. I of course don't do that anymore, but it was a step by step iterative progress to move to where I am now. Although I still miss how easy it was to see changes locally, deploy to prod quickly, and make hotfixes in prod. I sometimes think maybe I should go back to php.