Comment by otabdeveloper4
5 days ago
"Good" is the context of LLMs means "plausible". Not "correct".
If you can't code then the distinction is lost on you, but in fact the "correct" part is why programmers get paid. If "plausible" were good enough then the profession of programmer wouldn't exist.
Not necessarily. If the RL objective is passing tests then in the context of LLMs it means "correct", or at least "correct based on the tests".
Unfortunately that doesn't solve the problem in any way. We don't have an Oracle machine for testing software.
If we did, we could autogenerate code even without an LLM.