Comment by jillesvangurp
7 hours ago
It's a good rationale. But it points the finger at a real bottleneck in open source development: the burden of manually reviewing contributions. And the need to automate that with AI as well. Reviews were already becoming a problem before AI. Lots of projects have been dealing with a large influx of contributions from inexperienced developers from all over the world looking to boost their CVs by increasing their Github statistics. It's the same dynamic that destroyed Stackoverflow. Which, thanks to AI has been largely sidelined now. And now that AI is there, those same inexperienced developers are using that at scale to generate even more garbage contributions.
Doing manual reviews of everything is very labor intensive and not scalable. However, AIs are pretty good at doing code reviews and verifying adherence to guard rails, contributor guidelines, and other rules. It's not perfect, but it's an underused tool. Both by reviewers and contributors. If your contribution obviously doesn't comply with the guidelines, it should be rejected automatically. The word "obviously" here translates into "easy to detect with some AI system".
Projects should be using a lot of scrutiny for contributions by new contributors. And most of that scrutiny should be automated. They should reserve their attention for things that make it past automated checks for contribution quality, contributor reputability, adherence to whatever rules are in place, etc. Reputability is a good way to ensure that contributions from reputable sources get priority. If your reputation is not great, you should expect more scrutiny and a lower priority.
I don't know Zig, but I think that is not the problem here. Not exactly. The real question is: why spending all those efforts to grow and align a pool of contributors if contributions are cheap and correct? Code review is not just about checking if what it says it does, and if it does it according to the guidelines. The review is a touch point to discuss where the project is heading and how to get there. That is the most important part in the long run. As a collective human effort, it needs coordination. Some of it is via the review process (especially for those not part if the core team that draft the roadmap). One could document all those micro decisions with the rational, but it might end up be a wakamole game. IMO, projects which allow AI usage need to spend way more effort in coordination (and quality insurance).
> The real question is: why spending all those efforts to grow and align a pool of contributors if contributions are cheap and correct?
Until the contributions are cheap and correct, you need valuable contributors more than you need the contributions.
You point would be valid when we get to a point of contributions all being both correct and cheap. Right now they are only cheap.
You still have to review everything manually again anyway. It's a compiler for a language, bugs and bad architecture decisions cost a lot. They moved to codeberg, so there are less garbage PRs now. They try to grow a culture where you expected to deliver good code in the PRs so the review takes less time.
It takes like 5 minutes to spot garbage PRs manually. LLM can flood you with a wall of text where only half of the stuff make sense. Also, they can't really spot bad architecture. It's a compiler in an unpopular language, don't forget that.
> [you can] stop accepting imperfect PRs in order to maximize ROI from your work, but that’s not what we do in the Zig project
The real bottle neck when you want to grow is connecting with the right people. An LLM is not helping with that if you want to build a community. When you use LLM to skip the need to understand a problem how are you ever going to get a reputation that I can trust?
The post is not about reputation it about seeing how people respond and work with you in a community.
EDIT: I see that you frame it as a help and a tool and sure it might work, but I feel like it is just another obstacle.
> the burden of manually reviewing contributions... [a]nd the need to automate that with AI as well.
I suggest we also automate the distribution and the use of software with AI as well, and then just all go to the beach and sip on some cocktails or something.
Or in other words: Good luck with that.