Comment by john_strinlai
4 days ago
you arent kidding!
>"think AI makes you super hacker? think again: https://arxiv.org/abs/2507.09089"
not that im particularly pro-ai, but it seems a bit silly to rely on a paper about early-2025 ai when people are using late-2026 ai, given how different they are.
The interesting thing about that paper, I would have said, was the _perception_ gap; people believed that it was making them more productive even when it was doing the opposite. That effect presumably hasn’t gone away.
how are they different? did the algorithms upon which they operate suddenly change?
> how are they different? did the algorithms upon which they operate suddenly change?
Just like most technologies AI algorithms and tools get continuously refined/improved over time.
One thing modern AI has gotten especially good at in the last year is reverse engineering, it went from struggling and frequently making mistakes when analyzing binaries even with say a specialized Ghidra MCP integration to being able to very quickly and accurately analyze even heavily stripped BROM binaries with almost no user guidance or specialized environments. Binaries that would previously take days or weeks to analyze can be analyzed by a modern AI in minutes to hours.
The difference is massive and obvious to anyone who has been working with these tools over these time periods.
Presumably yes. It is safe to assume that algorithms running the model changed, the training procedures changed, and the scale changed. Unfortunately the AI companies refuse to disclose the details of any of this, which is a harm to human knowledge (hopefully a temporary one). Yet the results speak for themselves.
For me it's only been in the last month with Opus 5.5 that I've been seeing code quality I actually respect (though this is with substantial human effort in the loop). Though this experience is from work where I don't have access to non-Anthropic models, nor Fable.
Exactly, same with personal transportation. They still need to be fed oats, right?
>how are they different?
i do not believe you are asking that question in good faith.
and, if you are the leah who wrote the policy, i am positive this would be a completely fruitless discussion as you are obviously entrenched in your beliefs. i am not under the delusion that anything i could say, show, or prove to you would change your mind.
bots are unworthy of any faith.
1 reply →
It takes time to research and publish. It'll always be some slightly old model. Is this argument just a permanent get out of jail free card? Add 10 to each year, will we be seeing that said in the future?
i don't think models will continue to improve at the rate they have been, so eventually it will be less of an issue. there are also questions to answer and studies to be done that aren't as sensitive about the specific model.
however, productivity is sensitive to the model, and models have significantly improved in the last 1.5-2 years. sometimes things move quickly and research gets outdated. ai is not the first field where this has happened.
the research is still valuable, it just has to be acknowledged that it is specific to a point in time. using it for an argument 2 years later just shows that the leah is being intellectually dishonest, and it weakens the overall message.