Comment by embedding-shape
5 hours ago
For all we know, you both could comparing using a Nokia 3310 and a workstation PC based on the hardware, but you both just say "this computer is better than that computer".
There are a ton of models out there, ran in a ton of different ways, that can be used in different ways with different harnesses, and people use different workflows. There is just so many variables involved, that I don't think it's neither fair nor accurate for anyone to claim "This is obviously better" or "This is obviously impossible".
I've been in situations where I hit my head against some hard to find bug for days, then I put "AI" (but what? No one knows) to it and it solves it in 20 minutes. I've also asked "AI" to do trivial work that it still somehow fucked up, even if I could probably have asked a non-programmer friend to do it and they'd be able to.
The variance is great, and the fact that system/developer/user prompts matter a lot for what the responses you get, makes it even harder to fairly compare things like this without having the actual chat logs in front of you.
> The variance is great
this strikes me as a very important thing to reflect on. when the automobile was invented, was the apparent benefit so incredibly variable?
Is this a trick question? Yes it was. A horse could go over any terrain while a car could only really go over very specific terrain designed for it. We had to terraform the world in order to make the automobile so beneficial. And it turned out that this terraforming had many unintended consequences. It's actually a pretty apt comparison to LLMs.