← Back to context

Comment by vga1

10 hours ago

How would you differentiate a 3000 line LLM commit made by the best models and good AI processes from a 3000 line commit made by the best human developer?

edit Okay, I set the bar too high here with "best human developer" and vague "good AI processes". My bad. Yes, LLM is not quite there yet.

18 comments

vga1

Reply
IshKebab  10 hours ago

It's still fairly obvious just by skimming the code. The best AI models are still quite far from the best human developers in ability and especially in code quality.

  • matwood  9 hours ago

    When the best AI models are the same or better than the best[1] human developers, what then?

    We're already at the point talking about best vs. best.

    • IshKebab  8 hours ago

      If that happens and we have a way of reliably knowing if some code is produced to that high quality, then I think we probably can accept that AI coding is the only sensible option.

      We definitely are not close to that point though and it's unclear if/when we will get there.

      3 replies →

    • irishcoffee  8 hours ago

      How can AI possibly be better than “the best” when the corpus of training data now includes its own slop in addition to all the code by new devs/lazy devs/bad devs scattered all over the internet? Law of averages applies here.

      3 replies →

maccard  6 hours ago

The post that inspired this post [0] says:

> So while one could in theory be a valid contributor that makes use of LLMs, from the perspective of contributor poker it’s simply irrational for us to bet on LLM users while there’s a huge pool of other contributors that don’t present this risk factor.

> The people who remarked on how it’s impossible to know if a contribution comes from an LLM or not have completely missed the point of this policy and are clearly unaware of contributor poker.

The point isn't about the 3000 line PR, it's about do we think the submitter is going to stick around.

[0] https://kristoff.it/blog/contributor-poker-and-ai/