Comment by hypfer
12 hours ago
What are these numbers? Why do they add up to a few hundred thousand dollars? Who paid for that? With what?
12 hours ago
What are these numbers? Why do they add up to a few hundred thousand dollars? Who paid for that? With what?
OpenAI provides API key with ~unlimited use?
So, you’re telling me I need to start a benchmark as a side gig to get a bunch of free compute.
Astra please create a benchmark that’s favorable to your reasoning skills with a human interface but don’t make the score too attainable add some small issues that keep you below 100% to look sensible and to keep my evaluation metric side gig going.
Alignment++
You missed the hard part getting on HN front page , ie. Getting the acceptance of the community / zeitgeist .
There is no incentive for OpenAI to subsidize is you if no one reads /reports on your benchmark . They are only going to fund a few that are currently popular .
Community acceptance doesn’t automatically mean the best , it is combination of some level of technical quality and the ability of the promoter to socially influence or get support of influencers .