Comment by ben_w
1 month ago
To an extent. Even though they're clearly improving*, they also definitely look better than they actually are.
* this time last year they couldn't write compilable source code for a compiler for a toy language, I know because I tried
This time last year they could definitely write compilable source code for a compiler for a toy language if you bootstrapped the implementation. If you, e.g., had it write an interpreter and use the source code as a comptime argument (I used Zig as the backend -- Futamura transforms and all that), everything worked swimmingly. I wasn't even using agents; ChatGPT with a big context window was sufficient to write most of the compiler for some language for embedded tensor shenanigans I was hacking on.
Used to need the "if", now SOTA doesn't.
SOTA today has a different set of caveats, of course.