Comment by Normal_gaussian

2 hours ago

Commands like /goal and similar are the more complex version of this; you write a prompt like it was a singular iteration, it runs one iteration, then runs an "evaluator" to determine if the goal has been reached, then runs the prompt again with a little extra to make it go again - and so on. The evaluator is just an LLM with most of the result or context looking at the original goal and the state and answering the question "has the goal been met".

The interesting part of this: while some leading implementations use the same LLM and context for the evaluator, some call out to a different context, some to a tuned LLM and different context; so which is better? many blog-scale benchmarks are calling it a toss-up that is highly dependent on the primary model.