Comment by kristofferR
7 hours ago
What's with the deceptive graph on top? Not including Astra can't have been an oversight, did the model compare poorly to it?
7 hours ago
What's with the deceptive graph on top? Not including Astra can't have been an oversight, did the model compare poorly to it?
https://openai.com/index/our-decision-on-cursor-following-it...
This explains why. Mentioned in another comment, but cursorbench explicitly tests with Cursor as the harness, and OpenAI doesn't allow them to use Astra in Cursor.
> with a proposed shutoff date of November 12, 2026
That said, I don't expect them to benchmark Astra in their Cursor harness given the situation.
Cursor never added Astra to its consumer subscription plans. And it's likely exactly because of this announcement. Why would they add support for a model they would have to remove shortly after?
shutoff for existing models, new models stopped as of that announcement, astra will never be on cursor.
That's not accurate. OpenAI doesn't allow Grok to provide Astra to Cursor customers anymore, but it doesn't ban anyone from using Astra via alternative harnesses.
If Cursor wanted to include Astra in CursorBench nothing would stop them, they could easily have spent half an hour vibecoding in OpenAI API key support - if it hadn't been convenient to neglect to do that.
Even if they could do that (workaround to include Astra in CursorBench), that has no practical consequences for Cursor users and that's what I as a Cursor user (what I use for dev, though I use ChatGPT for non-dev stuff) care about.
2 replies →
If the benchmark is "what's the best model to use in your Cursor subscription", why would they do that? OpenAI knew what the effects of their decision were. Hard for me to have sympathy for either party here, honestly, and I say that as someone who is a customer of both.
Deceptive? An extremely quick google search would answer your question. OpenAI pulled out of Cursor before they released Astra so it never got that benchmark.
Pulled out from letting them resell Astra access, that's not a limitation on running a benchmark.
I wonder if that means that SpaceX evals show that they consider astra better than fable or that they hate Sam&co so much they don't want to show their stuff.
https://openai.com/index/our-decision-on-cursor-following-it...
Its because of this. You can't use Astra in Cursor, and cursorbench uses cursor as the harness. They can't actually benchmark it using their harness hence why its not included.
They have Astra in other benchmarks lower on the page. They just don't want to show it winning
1 reply →
They can benchmark it because you can use an openai api key with cursor. Astra is just not included in the cursor plan.
Elon posted on X that Grok 4.7 is behind Claude and OpenAI for agentic coding:
https://x.com/elonmusk/status/2102082011233931762?s=20
so it's likely about usage in Cursor specifically.
https://xxcancel.com/elonmusk/status/2102082011233931762?s=2...
this is exactly it.