Comment by UncleOxidant
19 hours ago
It's way better than Opus 4.5. See my stress test elsewhere in this thread (C -> microcode compiler with optimizations written in OCaml, generates microcode + Verilog output) - it's doing better than Opus 4.6/4.8 were when I was using them to code the original C version of this compiler early in the year. It's come up with optimizations that Claude didn't come up with back then.
Did opus have an oracle for the original compiler? If not, then it's expected that qwen would do a better job.
As is known, LLMs are amazing when verification is cheap, and much less useful when it's expensive.
> Did opus have an oracle for the original compiler?
It actually did. There was an older yacc-based implementation that had a lot of architectural limitations. We decided that it was better to start over with a better architecture, but we did have that older one around to compare generated microcode initially.