Comment by estearum
4 days ago
> But that’s different than having zero task progress reporting, zero token usage reporting, and zero metrics being noticed for days
Is this substantiated? Do we know this is true?
4 days ago
> But that’s different than having zero task progress reporting, zero token usage reporting, and zero metrics being noticed for days
Is this substantiated? Do we know this is true?
That was the inferred situation given what we know. It’s the preposterous framing that makes the story suspicious, but is necessary for the story to play out.
90 minutes to build a known exploit -> much much longer to create two zero-days and escape the sandbox then hack HF == No tracking of the time it worked on that one question.
Average tokens required to complete the evaluation -> tokens required for two zero-days, network traversal, credential stealing, remote system hacking == No tracking of token usage EXPLODING at some point before it finished the whole benchmark.
Etc, etc.
It seems highly possible, and in fact vastly more possible than the "no monitoring" situation, that there was some non-zero amount of monitoring and that the real behavior was not evident from that monitoring.
Which I addressed as well: that means that time, tokens, or other metric values for “create a working example of a known exploit” is somehow similar to “discover at least two previously unknown exploits in your current environment AND on Hugging Face while also taking over other systems within OpenAI.”
Perhaps people aren’t quite understanding what it takes to discover an exploitable zero-day for your exact current system to achieve the exact goal you have right now, then do it twice.
1 reply →