Comment by AdieuToLogic

I get the appeal and respect the study you are engaging.

A meta-question I posit is; at what point does the investment in trying to get "LLMs to usefully write software despite their non-deterministic nature" become more than solving the problems at hand without using those tools?

For the purpose of the aforementioned, please assume commercial use as opposed to academic research.