Comment by TeMPOraL
2 days ago
This just means we can't take the (often bad) shortcut of guessing what the code does because we feel we can predict what the author was thinking. Instead, we have to treat existing code as a natural sciences exercise - use the same methods we apply to the world around us, which has its logic and structure, but no human intent behind it.
GP is saying that "the search space" of "the natural world" is much bigger than the search space of "stuff created on purpose by humans". In both cases, there will be guesswork, but in one case less.
I think there is something to this, but am even more concerned by the pure scale. Consider 100k, 1M, and 10M LOC human legacy projects (already a big problem, usually!), and now compare to the likely outcome of 100M, 1B, 10B, etc LOC llm projects. That's where you may start to see the outcomes I reference in my OP.
> GP is saying that "the search space" of "the natural world" is much bigger than the search space of "stuff created on purpose by humans". In both cases, there will be guesswork, but in one case less.
I agree. What I am saying is, we have effective tools and processes for each of these search spaces, so code no longer being "created on purpose by humans" is not the end of the world.
> we have effective tools and processes for each of these search spaces
Do we? Science (the tool for the natural world) is limited and there's so much we don't understand.
The recent superhuman achievements by llms in math are promising, but what we've seen so far isn't enough imo to YOLO building arbitrarily large systems in code and just trust that the llms will be able to handle it. Especially when the stakes are high (human lives, etc).
2 replies →