← Back to context Comment by xiphias2 2 years ago It will be SOTA maybe when Gemini Ultra is available. GPT-4 is still SOTA. 2 comments xiphias2 Reply philomath_mn 2 years ago Usually SOTA status is established when the benchmark paper is released (probably after some review). But GPT4 is the current generally-available-SOTA silveraxe93 2 years ago They also compare to RLHFed GPT-4, which reduces capabilities, while their model seems to be pre-RLHF. So I'd expect those numbers to be a bit inflated compared to public release.
philomath_mn 2 years ago Usually SOTA status is established when the benchmark paper is released (probably after some review). But GPT4 is the current generally-available-SOTA
silveraxe93 2 years ago They also compare to RLHFed GPT-4, which reduces capabilities, while their model seems to be pre-RLHF. So I'd expect those numbers to be a bit inflated compared to public release.
Usually SOTA status is established when the benchmark paper is released (probably after some review). But GPT4 is the current generally-available-SOTA
They also compare to RLHFed GPT-4, which reduces capabilities, while their model seems to be pre-RLHF. So I'd expect those numbers to be a bit inflated compared to public release.