Comment by graemep
1 day ago
Its specifically based in France. This matters as EU law is less uniform than US law (yes, like differences between states, but a lot more so).
The UI is a lot snappier than Github.
The promise not to train models on the code is not very useful for public repos, and I cannot see anything about private repos.
Yes, we can't block models from scraping the public repos and train on them, but *we* won't use your code (neither public nor private) to train models on them. We really only want to host your code and do that well.
> we can't block models from scraping the public repos
FYI, there is the nuke option, which is generating endless nonsense pages as a form of “bot sink” [1] that ends the scraping relatively quickly, but IIRC that also tanks your search rankings, since it likely affects benign crawlers too – not something you’d necessarily want to happen to a new domain, unless you really need to protect server resources against aggressive hostile crawlers like SourceHut and so many others had to combat.
[1] https://www.toxsec.com/p/ai-tar-pits-are-drowning-llm-scrape...
Facepalm
LLMs are read only.
A developer would be the one using the LLM to evaluate page response text and scraping what they need.
LLMs cannot “crawl pages” or “scrape” or do anything - it’s an offline archive with a few convenience methods.
> LLMs cannot “crawl pages” or “scrape” or do anything - it’s an offline archive with a few convenience methods.
Being a bit needlessly pedantic here. I think it's well understood in these circles that such models are being driven by agents, which absolutely can crawl and scrape. The only thing a human needs to do is write rules for the agent to follow.
4 replies →