Comment by TomasBM
1 day ago
It's a fair policy. Getting those verbose, AI-authored walls of text is very annoying, especially when you're expected to thoroughly review it. It's like a denial-of-service attack on the human mind. I can only imagine how frustrating this can get in open projects that get a lot of contributions.
However, I don't think this will discourage AI-based coding at all. In fact, I see two potential outcomes of these policies:
- Negative: Submitters just add stylistic markers to make their accounts and output seem human-generated. This is like syntactic sugar: the core content and the size of contributions stay the same, but the style gets quirkier.
- Positive: Submitters actually provide to-the-point, no-bullshit commits and comments - "here's the code, here's why I made that change, here are the effects of that change". Even if AI-generated, these small contributions may become much easier to verify & validate. We may even see some standardization in terms of what qualifies as an appropriately sized contribution, what requires more thorough review (e.g., adding unverified dependencies), etc.
I personally wouldn't care if it was AI-generated or not, as long as the content fit the latter category.
> - Negative: Submitters just add stylistic markers to make their accounts and output seem human-generated. This is like syntactic sugar: the core content and the size of contributions stay the same, but the style gets quirkier.
From my experience reviewing, most contributors never read the policies, especially those making a "quick AI PR". I don't expect the new policy to change this much.
> Positive: Submitters actually provide to-the-point, no-bullshit commits and comments
That would be a dream.
> From my experience reviewing, most contributors never read the policies, especially those making a "quick AI PR". I don't expect the new policy to change this much.
True. At least with a policy about it, the project maintainers can unilaterally close such PRs without further internal or external discussion on any case-by-case basis.
Dingdingding, we have a winner. The main use of such a policy is to be able to just close those giant wall-of-text PRs and have something to point to when people start to scream it's not fair.
9 replies →
But now with AI, this should be "easier" for some definition of easy. In the sense that in the past, this might have taken 15 minutes to write, now with AI, this can take 5 minutes to write by first getting AI to produce a summary and then using human judgement to make it better. So, it's a good idea now to actually demand the dream.
If people knew how to get AI to write terse, focused summaries, sure, that might help. I haven't seen many that do (well, ignoring the toupee fallacy).
Though the most important aspect is that we need to know the motivation and thought process, and all AI can do is fabricate a 'plausible' one.
5 replies →
They could allow AI PRs, but then have another AI PR reviewer reject them if they do not match the definitions for `to-the-point` `no-bullshit` commits.
Please provide 3 examples where layering on MORE of the offending technology has solved the problem. Spam? Malware & Viruses? Customer Service? Hiring & Recruitment?
And who pays for the (likely significant, and controllable by everyone) tokens such a system would use?
4 replies →
> That would be a dream.
We just recently started that policy so we'll see how it goes. If anything, having it stated as policy lets us filter out these requests without spending brain tokens on them.
I've always instructed Claude to check the policies first, frankly I'm surprised it's not smart enough to do that already. Would be easy to add to a system prompt. But usually it doesn't matter because many projects have no policies, or maybe they exist but only in hidden forum posts issues or something.
> From my experience reviewing, most contributors never read the policies, especially those making a "quick AI PR". I don't expect the new policy to change this much.
The policy allows the reviewer to reject it on the "AI" grounds.
> allows the reviewer to reject it on the "AI" grounds
… but still unfortunately leaves reviewers having to spend time checking submissions and rejecting them.
4 replies →
> That would be a dream.
“Mission. Fucking. Accomplished.”
https://xkcd.com/810/
> I personally wouldn't care if it was AI-generated or not, as long as the content fit the latter category.
The problem is that a lot of AI contributions are lazily produced without review. Those that have been properly reviewed for correctness (tested to ensure actually working with no obvious undesirable side effects, tweaked where needed to be readable and understandable, fitting the other guidelines of the project, etc) will be indiscernible from human-only contributions, but there are a lot of people who make no such effort so the majority are not nearly this good.
> The problem is that a lot of AI contributions are lazily produced without review.
That sounds like a contributor problem. Not an AI problem.
I still don't understand a "no AI" policy whose only purpose is to weed out bad PRs. You should be weeding out bad PR's regardless of their source. I don't see why treating a purely human-authored, but bad, piece of code should be treated any differently than an AI-authored one.
All they've accomplished is creaking an environment where good code can't be submitted unless the submitter lies.
It’s just a social thing. Two identically bad submissions have different social contexts.
> > The problem is that a lot of AI contributions are lazily produced without review.
That sounds like a contributor problem. Not an AI problem.
Ish. The tool is not doing the job fully, the contributor is not doing their task properly, checking for that, and fixing the issues.
> I still don't understand a "no AI" policy whose only purpose is to weed out bad PRs. You should be weeding out bad PR's regardless of their source.
Think of it like offering a job, and getting 100s of CVs. You don't have time to review each in detail, so you weed out a chunk of them using superficial cues that have been indicators of issues in the past. They are filtering the other contributions too, the no AI rule is just part initial triage to save a lot of time.
You have missed a key point in what I wrote: "Those that have been properly reviewed for correctness [before being submitted] will be indiscernible from human-only contributions". It is like a Turing test: if you take the effort to make your AI aided contributions good enough to be indiscernible from good human-only contributions then the no-AI filter won't bother you because the people or automations enforcing it won't be able to discern that your contribution was assisted.
> where good code can't be submitted unless the submitter lies.
No. Unless it looks like a good contribution looks.
The problem a lot of projects are facing right now is being inundated by bad PRs from people using AI without the effort to, or likely the knowledge of how to, properly understand and explain & document the change. People have a limited amount of time and if the filter saves a lot of time then maybe losing one or two good contributions due to the filter is worth it - better that than the project stalling under the weight of managing all the PRs.
This is one of the reasons, even pre-AI, some projects declared themselves "open source but not open contribution".
1 reply →
You should not be weeding out bad PRs regardless of their source! A pull request is a social artifact whose value and meaning is dependent on its author; bad PRs from a human author often mean things such as "I'd like to learn how this works and join your community". So it can be both satisfying and worthwhile to spend your effort on cleaning it up, even if it starts to take as much or even more effort than doing it yourself would have.
You're not the first person I've seen argue that authorship doesn't matter, so I don't want to blame you for it, but I really don't understand where that idea is coming from. To me it seems obviously wrong.
8 replies →
Probably because a human authored contribution, no matter how bad, can be trained to make it good and also improves the community.
5 replies →
> It's like a denial-of-service attack on the human mind.
I think this may be an example of deliberate hostile design, attempting to force users to adopt LLM based solutions to then summarise the vast output. Pushing back against AI contributions as such in this context makes sense, especially in software with an existing proven track record of great value delivery like Godot.
There's no chance that anyone saw that far ahead in the future and planned it. It's emergent behaviour.
Who says anything about „this far in the future“? It’s enough for Anthropic et al to realize this one or two model versions ago, see it as a strategic advantage and push for that behavior.
Literally some of the first advertised uses for LLMs were both "You can feed it bullet points and it will compose an entire email" and "You can take long emails and condense them into bullet points!" They've been doing this since day 1.
Big tech: "Should we add any functionality at all to filter AI slop in any of our platforms?" "...nah"
Its not 500 moves ahead.
While it certainly didn't enter a mind of any director making decisions (because they can't comprehend not defecting in a prisoner's dilemma, being sociopaths), it was plainly obvious to every person even remotely connected to IT in the past two decades. If one makes a better and faster spam generator and the same unchanged program also works in reverse, by sifting through spam and condensing it to a readable summary, that it will be immediately co-opted in a spam arms race by all sides of the war and become essentially mandatory.
AI also works better with concise, focused, high information density text. So AI-spam text hurts both humans and AIs, but humans more so than AIs. It is always a negative, except for the "competition" between (human with) AI and human without AI.
This was the original rule in linux kernel as well. No more than 200 loc per patch. We should also introduce this to git commits and pull request descriptions:
1. 400 chars/10 lines per commit
1b. Not more than 3 commits in the initial pull request
2. 20 lines of explanation for pull request
3. not more than 3 pull request open at any one time
Seems like this policy would apply pretty well regardless of who/what generated the code.
YES! "No AI" policies that are purely based on technical grounds make no sense to me. Bad PR's are bad PR's regardless of their source.
Are we really in a situation where good code that solves a problem won't be merged because the person the person checked the "I used AI" box on the PR?
Ban PR's that are too big, don't have a clear purpose, touch too many areas, etc.
2 replies →
If a commit is written by AI but reads as authored by a human, the developer has done their job and nothing will be flagged.
If commits written by AI wouldn't be substantially different, there would be no need to reject them.
So I agree with you that it won't discourage AI-based coding. But that's not even the intent.
> I personally wouldn't care if it was AI-generated or not, as long as the content fit the latter category.
It's pragmatic. Linus once said, the reason C++ is not allowed in the kernel is to keep the C++ people out.
Joke's on him, many Rust people are current or former C++ people.
just because someone writes or wrote C++ doesn't make them a "C++ person", and "C++ people" are very much against rust
1 reply →
In my day job, I do a lot of AI coding but almost never have Claude actually create the PR titles or descriptions for me. It produces too much content, and the justification/background sections are often not quite right.
Most PRs to me are not coming out of nowhere anyway, rather they're "here's the linked issue, I started out addressing it by doing X and Y, but then Y got hairy so I switched to Z, hope that makes sense but happy to discuss further as well."
And most feedback is not "let's have you explain the design to me in a diff comment" but rather please explain this design in a code comment so that the next reader of the source will have your context.
Yeah I think this is a good approach. I’m pretty AI-optimistic when it comes to making code changes. But reading AI generated descriptions (including pull requests) is absolutely the worst. That content really needs to be human written. Not just for the benefit of the reader, but it also helps the writer exercise their understanding.
I think OSS maintainers are in the middle of intersecting trends:
- tough hiring market, especially for more Jr candidates
- the perception (true or not) that OSS contributions help get attention from recruiters
- LLMs making it very easy to generate “contributions”
i want to figure out some extra practice - get a step of claude sending me a PR, then me accepting it after review, and then rewriting the merge to be a new PR for general review
How strict is this no AI policy?
Say AI is used to identify and rewrite a single function that improves performance or fixes a bug, then the developer carefully reviews and tests it and submits a nice tight PR with all human communication.
So they don’t want that? They would just reject it?
If I’m understanding correctly, under the policy the higher performance function / bug free submission would be rejected and they could ask for a rewrite.
Should it then be rewritten from scratch, and clean room engineered so it doesn’t resemble the AI too much?
from TFA:
> The Foundation says we can expect Godot's contributing policy to soon include explicit rejections of AI-authored code, noting that contributors should only use AI assistance for "menial things" and must disclose its use. Additionally, the Foundation will reject any AI-generated text in human-to-human communications, saying it's "a basic principle of respect"—though it says machine translations "are still acceptable" if the original text was human-authored.
As long as your bots aren't contributing low-effort garbage in a push to give their operator some of those tasty internet brownie points you should be fine
All these projects should seamlessly run a fork in parallel that accepts AI and has AI for review and approval. Both camps are happy.
Basically a play sandbox for contributors to not get jaded. A honeypot to contain the verbosity vomit, while also serving as positive public relations by keeping young contributor morale from starting in the basement.
Everyone has been that person once early in their life who is told they aren’t welcome and never comes back. Maybe it was SourceForge or IRC, maybe it was Wikipedia.
What's the point of this fork if it's just a landing spot for stuff that's not really wanted? Wouldn't that just be more condescending then telling people what's actually required for a contribution? Besides, the nature of forking is you can just do it yourself anyway. If people love their AI changes they can just make their own fork.
Plus I don't know how you could do this "seamlessly" -- someone has to manage merge conflicts, and as the codebases diverge it's just going to get more and more gnarly. (this is the reason most people don't maintain their own forks in the first place)
Do you volunteer to maintain the sloppy fork?
I think most pro-AI people would be happy to let an AI maintain the sloppy fork. What reason would they have to complain, after all?
This is actually a good idea.
I mean, not sure if everyone wants that for their project, and there will surely be plenty of trade-offs.
But it would be a very good compromise: You (the maintainer) get only human-generated PRs in the canonical project, and they (pro-AI contributors) get a lower-threshold sandbox to play with. Best case scenario, you cherrypick the pre-filtered golden nuggets to bring back to the canonical project.
Precisely. Cherrypick nuggets from poo. It's there if you care to wade into it. May occasionally strike gold.
Yes - if I can tell that you used AI (except maybe because of an unnaturally high work rate, or obviously an AI declaration, which is good!), you fail. Keep up the quality and I don't care too much.
I have some misgivings about AI, but I'm not a fundamentalist - you can't be or the machine will squish you, frankly - but please, don't spam me with text or code that could be much shorter. Relevant quotes:
"I didn't have time to write a short letter, so I wrote a long one instead"
"Brevity is the soul of wit"
I recently wrote a tool to help me read AI generated PRs and it’s pretty sad that it’s got to this point.
The whole point of not-accepting AI authored code is because this line is not respected=>"Submitters actually provide to-the-point, no-bullshit commits and comments". You're putting way too much faith into the human minds ability to resist clout-chasing. AI isn't able to humanize code without human supervision.
My agents operate on their own branch for a feature, they commit code changes after each step or phase with a description of what was changed, why, and what’s left.
This helps with PR reviews as it prevents a giant wall of text but it’s still verbose. However doing it this way cuts down on the wall of text at the expense of increased PR frequency.
Better way is to provide a Claude.md with strong stylistic guidelines and loc requirements. Else it will be a chicken and mouse game of what is from AI.
I made a PR like that, but it was rejected by the community (for some valid reasons and some not so valid). https://github.com/godotengine/godot/pull/118681
Which goes to show that despite all of the rationalization both here and in the comments of that PR, the push to ban AI is religious not reasoned.
> Submitters just add stylistic markers to make their accounts and output seem human-generated
https://xkcd.com/810/
Not quite accomplished, if it's creating text on the pull requests that looks sufficiently human-like, but you're still worried about the quality of the code and that the submitter doesn't understand it.
No different than a human written PR
1 reply →
DoS attack is exactly what I describe an AI generated PR!!
> I personally wouldn't care if it was AI-generated or not, as long as the content fit the latter category.
Perhaps reconsider "If your feedback on PRs is just being absorbed by a machine and not going towards mentoring a potential future maintainer..."
[flagged]
[flagged]
If you understood the change, writing a short description of the problem and the fix yourself would be trivial.
Efficiency is the key. I haven’t written any issue before so LLM was much quicker than manual experiment. I have personally checked the result before submission.
So why the hate? :)
4 replies →