Comment by mohsen1
21 days ago
> Qodo takes a different approach by starting with real, merged PRs
Merged PRs being considered good code?
21 days ago
> Qodo takes a different approach by starting with real, merged PRs
Merged PRs being considered good code?
What do you suggest they use for ground truth?
I thought about this quite a bit. There are some nuggets in the open source code:
- vX.X.1 releases. when software was considered perfect but author had to write a fast follow up fix. very real bugs with real fixes
- Reverts. I'm sure anyone doing AI code review pays attention to this already. This is a sign of bad changes, but as important.
- PRs that delete a lot of code. A good change is often deleting code and making things simpler
For the first, your signal would be weak, for those events are rare. I don't think deleting and reverting is a signal of quality. Rather, it demonstrates bad changes, as you said. This does not tell the model what good code is, just what it is not.