Comment by copperx

I just realized how much better is 3.5 in some cases. I asked ChatGPT to improve a script using a fairly obscure API by adding a few features and it got it on the first try.

Then ... I realized I had picked 3.5 by mistake, so I went back and copied and pasted the same prompt into GPT4 and it failed horribly, hallucinating functions that don't exist in that API.

I did a few other tests and yes, GPT 3.5 tends to be better at coding (less mistakes / hallucinations). Actually, all the 3.5 code was flawless, whereas all the 4 had major problems, as if it was reasoning incorrectly.

GPT4 was incredibly better when it first came out, and I was gaslighted by many articles / blog posts that claim that the degraded performance is in our imagination.

Fortunately, 3.5 still has a bit of that magic.