Comment by aniviacat
3 days ago
> the difference between Early-2025 AI and Late-2026 AI is absolutely massive
If you only accept papers that were published in reputable venues and analyze LLMs that are less than a month old, you are left with zero useful papers (since writing a useful paper takes longer than a month). It is impossible to have an intellectual discussion if all scientific material is disregarded.
> If you only accept papers that were published in reputable venues and analyze LLMs that are less than a month old, you are left with zero useful papers (since writing a useful paper takes longer than a month).
Why would one look only at formal published scientific papers? Especially if the only publications are known to be quite outdated and highly likely to be no longer particularly relevant anymore.
> It is impossible to have an intellectual discussion if all scientific material is disregarded.
One can certainly have an intellectual discussion even for areas without relevant formal published scientific papers.
Certainly one can do so if the subject matter is in the area of your own expertise, that can even lead to scientific papers being written.
I'm an experienced open source developer leveraging AI tools for linux kernel, u-boot and buildroot contributions in addition to reverse engineering, so clearly this subject is well within my area of expertise regardless of what some outdated published research says.
I probably wouldn't even disagree with that paper's assessment at the time, however the situation is very different now. I have noticed that there is an issue with many experienced open source developers simply having not figured out how to properly leverage AI for open source projects they work on. Probably one reason for this is that productive workflows with AI are often very different from previous workflows and it takes some practice to figure out how to best use AI tools, these best practice can also change over time.