Comment by Supermancho
6 hours ago
The potential threats section reads like panic, rather than a critique of AI. I can see where #2 has some legs, if I thought tradition was sacrosanct.
1. AI proofs might be incorrect and difficult to demonstrate why. This implies they are not like human proofs in these qualities.
2. AI proofs are difficult to attribute correctly, because they don't follow established traditions. Nothing to do with the math, but ok.
3. Mathematicians without AI (for political or practical reasons) will not necessarily be able to participate in AI-assisted research. This history of Mathematics is littered with people having uneven access.
4. People/orgs are publishing that AI found things are fact before they are properly evaluated. Same issue.
5. All these things are bad, because AI might muddy the field with lots of unknowns.
This appears to be a very bad faith post that intentionally misrepresents what is being said.
1. pertains to the quantity of output adding stress to review processes; LLMs can feasibly produce a million plausible but incorrect 'proofs' in the time that a human can produce one. We already see this effect in software development, with bug bounty programs shutting down and open-source software rejecting AI contributions or closing altogether because LLMs flood review channels with an amount of spam for which there is no sufficient amount of human bandwidth to handle.
2. is nothing about "following established traditions" but rather the general concept of crediting people for their prior work, unless you think that "not plagiarising" is a trifling established tradition.
3. is more or less accurate to the point they made, but "it has historically been this way" isn't a compelling justification for "it should always be this way and also it's okay if it gets worse"
4. An existing issue being made 100x more common is a point worth bringing attention to even if it already existed, actually
5. said nothing that could possibly be interpreted in the vein of "muddying the field with lots of unknowns" at all. Point 5 was actually about economic incentives and the risk of mathematic research becoming beholden to tech monopolies
>2. is nothing about "following established traditions" but rather the general concept of crediting people for their prior work, unless you think that "not plagiarising" is a trifling established tradition.
But that is the nature of establishment, when something is a sufficiently firmly established tradition, people see it as a truism.
Crediting people is a social convention. Plagiarism is a social construct. It can be useful, in many areas of science, to reference to support your arguments. This is less important in proofs, because a proof is a proof, but references aid in understanding.
These are all reasons to reference and attribute that benefit the writer, and could be done voluntarily. The notion of a duty to reference or attribute has no impact on the validity of the claims being made. It is a collective decision to proportion prestige.
Turning the duty to do so into an unquestioned truism means it has to be done regardless of whether it accurately represents any property of merit.
There are many instances where prestige delivered grossly mismatches what an impartial observer would consider a fair balance of effort and ability.
We should at least recognise that this is so because we have chosen to let it be this way.
I did say in another reply already that if this was the person I replied to's position, they should stand on it and argue for it. I don't believe it's some kind of sacred cow that is beyond discussing. My problem with what they were saying is that they weren't solidly taking a stance and trying to make such an argument, but rather taking cheap shots by quoting things in misleading ways. Tackle the argument head on if this is what you believe, don't try to mislead people who might not've (fully) read the article by misstating the article's position in an attempt to make it look weaker than it actually is. If your argument depends on misrepresenting your opponent's position rather than standing on your own convictions, well, it doesn't speak much for your convictions.
In the research papers of any branch of science and technology, crediting the previous work is not some social convention, but it is very useful information for readers.
Despite the fact that there are some dishonest researchers, who attempt to create the illusion that their work is revolutionary and does not owe anything or only very little to predecessors, the truth is that almost anything novel that is published today contains only a few percent of truly new ideas that are grafted on a big body that combines many results extracted for older works.
If a new research paper separates properly its new elements from the old elements, which are properly attributed, enabling the search of the original sources, that can help a lot the readers to understand it.
I consider the fact that when you ask a question to an LLM, it is unable to accurately provide the source of the answer, as their greatest defect.
I'm not sure it's constructive to explain our differences, point by point. eg
> 2. is nothing about "following established traditions"
> undermine the traditional system of attribution
Literally does.
Suffice to say, I find your interpretations to be surprising and disconnected and it has not changed my views.
The actual thesis of point 2 is about plagiarism, and the thesis would remain the same if the sentence you quoted were removed completely. Your portrayal of it moves the out-of-context snippet to the forefont of the argument and makes it sound like an issue of "tradition for tradition's sake" or something similarly indefensible, but you refuse to engage with the real argument being made, hence why I suspect you are acting in bad faith. Are you suggesting that not attributing credit to work you've copied from is the way things should be going forward? If you are, then argue that point and make it earnestly. Instead you continue to avoid any substantial discussion of the points raised and only went for a cheap "gotcha".
1 reply →