Comment by capnrefsmmat

6 days ago

> Well, if you’re a novice, don’t do that.

I agree, and it sounds like you're getting great results, but they're all going to do it. Ask anyone who grades their homework.

Heck, it's even common among expert users. Here's a study that interviewed scientists who use LLMs to assist with tasks in their research: https://doi.org/10.1145/3706598.3713668

Only a few interviewees said they read the code through to verify it does what they intend. The most common strategy was to just run the code and see if it appears to do the right thing, then declare victory. Scientific codebases rarely have unit tests, so this was purely a visual inspection of output, not any kind of verification.