Comment by jtrn
11 hours ago
The correct title: Researcher brakes one specific stubborn historic enigma message with good help from Astra.
Stubborn for a long time because the message used a completely different key from the rest of that day's traffic. Everyone assumed it shared the daily key. The original transcription had errors. The left rotor turned over at letter 72, which is rare and breaks standard crib attacks.
What is cool, if true, is that it was a 2 day collab between the Leffer and Astra. To me this shows the importance of human in the loop, was still all also showing how immensely power of llm tools. But I think it’s getting a bit silly how much anrticles ignores the driving force (the person) in breakthroughs like this.
From TFA:
"However, the most astonishing thing about this break is that the GPT–6 Astra did it entirely on its own."
Follow up bit adds more context:
"Carter Leffer only directed GPT–6 Astra to see if it could break any of the unbroken Enigma messages published on the Crypto Cellar Research web page."
keep going
awesome! keep going
great work! keep going
/goal See if you can break any of the unbroken Enigma messages published on the Crypto Cellar Research web page.
3 replies →
> to me this shows the importance of human in the loop
1 reply →
Make me proud is current SOTA
Las Vegas Algorithm: A randomized, non-deterministic algorithm that is 100% accurate but has a variable runtime.
Now we just need this as a service. Another LLM that would encourage your agent like a cheerleader and provide emotional support and reassurance if necessary
I agree with this. I think the researchers who's harnessing the llm's power should be credited more than the model itself. We also need to understand the thought process and the prompts that are given to the model so we can learn and collab to ensure humanity's progress as much as the llm itself.
> I think the researchers who's harnessing the llm's power should be credited more than the model itself.
Even when the report literally says the LLM did it on its own?
Let's not over-correct in the direction of knowing better than the first party.
> the report literally says the LLM did it on its own
Not mention it also says this...
> We are still analysing the GPT–6 Astra logs to see exactly how it executed the break.
1 reply →
‘knowing what problems are worth solving —- priceless”.
For everything else, there’s Astracard
It also decided which problem to solve:
"After analysing the unbroken messages on the website, it decided that the most promising message was Nr. 172, MVUEH and it also quickly suspected that the plaintext of Nr. 173, SIPVX ..."
[dead]
Even if this is true, we must avoid falling into the trap of Kasparov of betting on Centaur Chess.
Just like with Kasparov's Centaur Chess, the idea of a 'human in the loop' is just a necessity due to current limitations.
There will hopefully (?) come a time one day when human beings provide only ultimate value judgments, and everything else is done by machines. Or it may not.
But I don't think betting your ego on the idea that you will be useful in the loop for very long is very wise.
While this is a reasonable analogy, engines became better than humans in the late 1990s, and engines became better than centaurs in the early 2020s. Could AI-powered mathematics improve faster than the ~25 years it took for chess? The AI labs are certainly hoping it does, but that's far from a guarantee.
This is a bit too future-oriented. Let's not mix up current capabilities and speculation about future capabilities. For the time being, collaboration works well. What the future brings is uncertain.
I'm still in my 20's so I feel some necessity to be future-oriented.
I do expect centaurs to outperform other systems for many types of tasks for years to come (and am kind of betting on this to keep getting paid).
But what I'm talking about is ego. It your ego is tied up with (a) your intelligence or (b) your ability to perform task X; you will probably be humbled this century.
2 replies →
> This is a bit too future-oriented.
This? Still? After everything?
Buddy, you're living in the future. In a science fiction novel. Please get used to it.
3 replies →
It seems I was wrong in this instance with regard to the "colab" part.
I found that Leffen even said the explanatory website took about 99 times more effort than the codebreaking itself. And he said that he set the direction and pushed, and the model did the execution. How much steering "pushed forward" involved is not disclosed anywhere, but in this instance, it seems to be more a case of "Human pointed at hard task and AI did an awesome job mostly by itself." Tho how much he was a simple meat-ralph-loop is not entirely clear.
8 months ago I got to the top of highload.fun using GPT-5 and Opus 4.5, and a lot of human interaction.
Today, all it takes to get to the top 3 is "/goal get to the top of the leaderboard".
The human-in-the-loop is only a temporary measure until the models get good enough.
> The correct title: Researcher brakes one specific stubborn historic enigma message with good help from Astra.
That's not correct for the content.
"However, the most astonishing thing about this break is that the GPT–6 Astra did it entirely on its own. Carter Leffer only directed GPT–6 Astra to see if it could break any of the unbroken Enigma messages published on the Crypto Cellar Research web page."
Your suggested title is more misleading, so what’s the value in using it?
*breaks, and also, your conclusion ignores the words typed by the article's author in the piece you presumably read, where it is reported that Astra did it mostly on its own.
Therefore Astra could also have done this comment better
that the left rotor turned over at 72 - was that a bug or a feature?
What's amazing is this comment is complete bullshit, and yet is #1.
Don't people actually read anymore?
Your "corrected" title is much less descriptive of what actually happened
also the entirety of the research that went into breaking enigma in the first place is in the training dataset
Come on, the trivializations start to sound quite unfounded now. Yes, a human was needed, but no, it wasn't a "collaboration"
So the LLM would have done all of this on its own? Why is it ok to acknowledge the human was needed but it’s not a collaboration? Is there a defined percentage of ownership required to make the word collaboration valid?
We focus on the tool because that is what makes it novel.
Hearing a guy built his home in a week with the power of nails and a hammer would have been novel in era of mortise and tenon.
I actually found an article about raving about how fast nail production was thanks to machining advances in 1790 and that it would bring great value: https://digital.libraries.psu.edu/digital/collection/pabookn...
It’s seems to me humans haven’t changed, just which machines we praise.
If you get someone to build you a house and they do it on their own, does that not count because they wouldn't have done it if you didn't pay them to do it?
Technically you built it yourself and the builder was just a minor collaborator?
3 replies →
From the article:
"However, the most astonishing thing about this break is that the GPT–6 Astra did it entirely on its own. Carter Leffer only directed GPT–6 Astra to see if it could break any of the unbroken Enigma messages published on the Crypto Cellar Research web page."
I mean... I'm all for collaboration but I think this case is pretty clear, no?
6 replies →
Humans are also needed to code in C++ and compile it, then the computer does the rest.
Now we just code in english and the computer does the rest.
1 reply →
What do you even mean it wasn't a collaboration. At any meaningful level LLMs just plain out suck when left unguided.
The shortcomings should really be obvious by now to anyone honest. And the marketing distortion being oushed out is just tiresome and detrimental for all of us.
A cyclist pedaling up a mountain isn't a "collaboration" between a bicycle and a human. This is the same. You don't see feral bicycles roaming the land. All models are ultimately built and run by humans, with human-provided instructions. And as with any program, it's garbage in, garbage out.
More apt analogy here: a cyclist pushing a bicycle down the mountain and seeing it somehow get down the whole track without falling down, is not a collaboration between a bicycle and a human. The human was not involved beyond giving the initial push.
1 reply →
The bicycle -- a simple method of transport powered entirely by humans -- used analogically to prove a point about [clears throat] automation.
I think this sounds somewhat less silly in English because "automotive" and "automative" don't have the same hyper-visible affinity, but all the same; you may want to consider the car as a more viable analogand.
If generating an image doesn't make you an artist, generating a solution doesn't make you a researcher.
Well, it makes them an AI researcher maybe but not a cryptographer anyway.
edit: I think that's going to be my go to on "you aren't an artist" from now on. "No! I'm an AI researcher!"