Comment by jonplackett
2 years ago
OK I get that everyone’s hype sensitive and I absolutely remain to be convinced on Gemini’s actual ability
BUT
The fact this wasn’t realtime or with voice is not the issue. Voice to text could absolutely capture this conversation easily. And from what I’ve seen Gemini seems quicker than GPT4
Being able to respond quicker and via voice chat is not actually a big deal.
The underlying performance of the model is what we should be focussing on
I disagree.
One of the major issues in LLMs is the economics; a lot of people suspect ChatGPT loses money on every user, or at least every heavy user, because they've got a big model and A100 GPUs are expensive and in short supply.
They're kinda reluctant to have customers, with API rate limits galore, and I've heard people claiming ChatGPT has lost the performance crown having switched to a cheaper-to-run model.
If google had a model that operated on video in realtime, that would imply they've got a model that performs well, and is also very fast or that their 'TPUs' outperform the A100 quite a bit, either of which would be a big step forward.
Even if you'd be inclined to shrug off the fact that this wasn't real-time voice and video-based, which you shouldn't because the underlying implications would be huge for performance, there's still the matter that the prompts used. The prompts shown are miles apart and significantly misrepresent the performance of the underlying model.
It goes from a model being able to infer a great deal at the level of human intelligence to a model that needs to be fed essential details, and that doesn't do much inferring.
I get the feeling that many here on HN who are just shrugging it off don't realize how much of the “demo” was faked. Here’s a decent article that goes into it some more: https://techcrunch.com/2023/12/07/googles-best-gemini-demo-w...
Exactly.
Corporate tech demo exaggerates actual capabilities and smoothes over rough edges? Impossible, this is unprecedented!!
The Apple vs Google brand war is so tiresome. Let's focus on the tech.
To be clear - I’m not saying this makes Gemini good. Just that it isn’t bad for these reasons!