← Back to context Comment by MillionOClock 2 years ago I wonder the same too, also is Gemini Pro supposed to beat GPT-4? 1 comment MillionOClock Reply m3at 2 years ago I thought so too but no, Gemini _Ultra_ is supposed to beat GPT-4. The one available through Bard is Gemini Pro. The wording sure is confusing!Looking at the technical report [1], on selected benchmarks, rounded score in %:Dataset | Gemini Ultra | Gemini Pro | GPT-4MMLU | 90 | 79 | 87BIG-Bench-Hard | 84 | 75 | 83HellaSwag | 88 | 85 | 95Natural2Code | 75 | 70 | 74WMT23 | 74 | 72 | 74[1] https://storage.googleapis.com/deepmind-media/gemini/gemini_...
m3at 2 years ago I thought so too but no, Gemini _Ultra_ is supposed to beat GPT-4. The one available through Bard is Gemini Pro. The wording sure is confusing!Looking at the technical report [1], on selected benchmarks, rounded score in %:Dataset | Gemini Ultra | Gemini Pro | GPT-4MMLU | 90 | 79 | 87BIG-Bench-Hard | 84 | 75 | 83HellaSwag | 88 | 85 | 95Natural2Code | 75 | 70 | 74WMT23 | 74 | 72 | 74[1] https://storage.googleapis.com/deepmind-media/gemini/gemini_...
I thought so too but no, Gemini _Ultra_ is supposed to beat GPT-4. The one available through Bard is Gemini Pro. The wording sure is confusing!
Looking at the technical report [1], on selected benchmarks, rounded score in %:
Dataset | Gemini Ultra | Gemini Pro | GPT-4
MMLU | 90 | 79 | 87
BIG-Bench-Hard | 84 | 75 | 83
HellaSwag | 88 | 85 | 95
Natural2Code | 75 | 70 | 74
WMT23 | 74 | 72 | 74
[1] https://storage.googleapis.com/deepmind-media/gemini/gemini_...