Comment by logicallee

20 hours ago

This is a solid and very interesting paper! The authors were kind enough to publish the complete prompt for it (appendix B.1 on page 18), so anyone can try their approach with any LLM and see the results.

It says to read the complete history. Would that be analogous to reading all of one's chat threads, or just the history of the relevant chat thread that it's a part of?

  • That seems to refer specifically to a provided history for it to read - presumably, this would just have the project it's running in. ("Variables (‘$node_dir‘, ‘$history_dir‘, ‘$baseline_dir‘, ‘$eval_program‘, ‘$problem_file‘) are filled in by the calling system.", meaning that a history directory for it to read is provided by the harness.)