Comment by koolba
19 hours ago
Except you can’t do that unless the runtime itself can reason about what’s being executed.
Otherwise you can get a python one liner that execs a different script engine.
19 hours ago
Except you can’t do that unless the runtime itself can reason about what’s being executed.
Otherwise you can get a python one liner that execs a different script engine.
You absolutely can, that’s what your harness is for. You don’t need your environment to “reason” about things when deterministic tools exist - You have a really fancy hammer, but that doesn’t make everything a nail.
To offer a possible example: What would the game Zork™ look like with an LLM? Assume we do not want to let players sweet-talk the system into letting them teleport to the end.
The LLM's job would be to channel "I perambulate in the direction of the Arctic circle" into go(north). You saved writing the grammar parser, but you still need to write the game world.
But what if we used the fancy hammer to change the shape of everything to be a nail? And what if we build the handle of the fancy hammer with a fancy hammer? With all of this we could build a very good fancy hammer manufacturing company.
Yet the hammer seller continues to scream everything is a nail and their hammer will replace your entire job eventually. So are you telling me the hammer seller is lying or am I the one using it wrong?
I mean you don’t _have_ to make python available to the agent. Nor bash.