So, you’re telling me I need to start a benchmark as a side gig to get a bunch of free compute.
Astra please create a benchmark that’s favorable to your reasoning skills with a human interface but don’t make the score too attainable add some small issues that keep you below 100% to look sensible and to keep my evaluation metric side gig going.
You missed the hard part getting on HN front page , ie. Getting the acceptance of the community / zeitgeist .
There is no incentive for OpenAI to subsidize is you if no one reads /reports on your benchmark . They are only going to fund a few that are currently popular .
Community acceptance doesn’t automatically mean the best , it is combination of some level of technical quality and the ability of the promoter to socially influence or get support of influencers .
So, you’re telling me I need to start a benchmark as a side gig to get a bunch of free compute.
Astra please create a benchmark that’s favorable to your reasoning skills with a human interface but don’t make the score too attainable add some small issues that keep you below 100% to look sensible and to keep my evaluation metric side gig going.
Alignment++
You missed the hard part getting on HN front page , ie. Getting the acceptance of the community / zeitgeist .
There is no incentive for OpenAI to subsidize is you if no one reads /reports on your benchmark . They are only going to fund a few that are currently popular .
Community acceptance doesn’t automatically mean the best , it is combination of some level of technical quality and the ability of the promoter to socially influence or get support of influencers .