Comment by obiefernandez
2 years ago
> For Gemini Ultra, we’re currently completing extensive trust and safety checks, including red-teaming by trusted external parties, and further refining the model using fine-tuning and reinforcement learning from human feedback (RLHF) before making it broadly available.
> As part of this process, we’ll make Gemini Ultra available to select customers, developers, partners and safety and responsibility experts for early experimentation and feedback before rolling it out to developers and enterprise customers early next year.
Finally, some competition for GPT4 API!!! This is such good news.
It won't be available to regular devs until Q2 next year probably (January for selected partners). So they are roughly a year behind OpenAI - and that is assuming their model is not overtrained to just pass the tests slightly better than GPT4
> and that is assuming their model is not overtrained to just pass the tests slightly better than GPT4
You are assuming GPT4 didn't do the exact same!
Seriously, it's been like this for a while, with LLMs any benchmark other than human feedback is useless. I guess we'll see how Gemini performs when it's released next year and we get independent groups comparing them.
>So they are roughly a year behind OpenAI
Possibly by that time GPT5 will already be out.
I would not underestimate DeepMind with its access to Google's resources and private data which OpenAI lacks, even with Microsoft's help.
They already caught up and surpassed GPT-4 and OpenAI's availability and APIs are very unstable and all that matters is that and the cost per token.
Possibly, but I doubt it! I'd expect a response by OpenAI via GPT-4V improvements soon though.
>Finally, some competition for GPT4 API!!! This is such good news.
Save your enthusiasm for after it launches; Google's got a habit of over-promising when it comes to AI.
Everything they published thus far in the generative AI space has been abysmal in quality compared to the competition. I'd be hella surprised if this reaches GPT-4 levels of quality...
I'm a GPT4 subscriber and a Google GSuite work subscriber. I've been using the latest Bard this morning to write and refine python code, and it's just as good if not slightly better than GPT4. I asked it to refine some obtuse code with lots of chaining, and it did an admirable job writing accurate comments and explaining the chained logic. It's ridiculously anecdotal of course, but I used Bard for all of 5 minutes last time they announced. This time seems different.
Manifold has this at 69%, so here's an opportunity to take some people's internet points: https://manifold.markets/brubsby/will-googles-gemini-beat-gp...
3 replies →
I’m curious which instances of overpromising you’re referring to.
Like how much they hyped up Bard, which when released turned out to be barely competitive with GPT3.5. E.g. https://www.reuters.com/technology/google-ai-chatbot-bard-of...
8 replies →
In terms of AI? Last year? A better question is what wasn't an overpromise?
Heh, I read that and had the opposite conclusion.
When I was reading the benchmarks and seeing how Gemini Ultra was outperforming GPT-4 I thought, "Finally, some competition for GPT4"!
But when I got to that part, that's when I realized that it could potentially be caught in release hell and not actually see the light of day or significant use. Google, for better or worse, has more of a brand reputation to maintain and is more risk averse, so even if Gemini Ultra can, in theory, outperform GPT4, users might not get a chance to access it for a while.
"Google, for better or worse, has more of a brand reputation to maintain"
You think this is why Google is so far behind?
It absolutely is. Googlers here will know that there was an internal version of ChatGPT that got canned because of halucinations.
2 replies →
Absolutely I do. Internally they have some incredible stuff, but the leadership is terrified of letting normies try it out because of the (real or perceived I don't know) damage to the brand that would happen if it said something racist or misogynist, etc.
4 replies →
It's by no means the whole story, but Google's very significant aversion to brand risk is definitely part of why they move slowly.
(Speaking as someone who's worked on launching several somewhat risky technologies at Google.)
Well not exactly. Not coming out until later when presumably GPT4 will have grown as much as well. So far each time, Google has failed to catch up to OpenAI. Hopefully they do however eventually.
Even if they're perpetually a year behind, that's a strong competitive spur to keep OpenAI on the move.
> presumably GPT4 will have grown as much as well
Most of the comments I see on Hacker News claim ChatGPT is getting worse at different things (though I don't believe those claims).
I’m afraid it won’t be nearly as good as GPT4, because of how lax Open AI can be with intellectual property. Google will not be able to train their model on Libgen or Opensubtitles, because they can’t afford the risk.
won't be available for regular devs until probably Q2 next year, OpenAI will have probably released GPT5 or whatever new model by then. And GPT4 was done training in 2022, the fact Google is thumping their chest about being 2 years behind a much smaller company is kind of pathetic
That's actually not a bad achievement considering Google is now the new IBM.