Comment by traceroute66

The point I'm making is that companies who are serious enough to want to keep their code-base in-house and off the various online repo services are also the kind of companies who are strict about what you can and cannot do with LLMs (if they permit use of LLMs at all).

So it does not make sense that the same companies would then magically sign-off on allowing their entire codebase to be spoon-fed into a whole bunch of LLMs for benchmarking.