Comment by JoshTriplett
11 hours ago
LLMs are unusually good at Rust; it's an optimization target. And the constraints provided by "successfully compile with the Rust compiler" make it work well for agent iteration.
(I have mixed feelings about that, but empirically it holds true.)
Yes, I found these agents to be better at producing acceptable Rust than at producing acceptable Python code.
In addition to the Rust compiler, you can also tell them to make clippy happy. Both in normal mode or if you are feeling nitpicky, you can also tell them to make clippy::pedantic happy.
Can you give an example where an LLM produced low quality Python code? Python is such a simple language. This seems hard to imagine. Plus, the amount of open source Python that LLMs can be trained upon is enormous.
I am looking at an MR a colleague threw at me right now where the code quality produced is atrocious.
Multiple redefinitions of enums - except they aren't enum but random lists of strings - that also diverge in different files.
Things consistently typed as `Any` or `dict[str, Any]` even though the functions clearly are expecting specifics, not `Any`
This is the worst MR I've had to review yet in my life. Absolute garbage.